On Fri, Apr 15, 2022 at 12:25 AM Linus Torvalds
[off-list ref] wrote:

quoted

And it could be that if I have 150k of those smallish allocations, a
server with lots of active users might have millions. Not having
looked at where they come from, maybe that isn't the case, but it
*might* be.

Maybe adding something like a

        static int warn_every_1k = 0;
        WARN_ON(size < 32 && (1023 & ++warn_every_1k));

to kmalloc() would give us a statistical view of "lots of these small
allocations" thing, and we could add GFP_NODMA to them. There probably
aren't that many places that have those small allocations, and it's
certainly safer to annotate "this is not for DMA" than have the
requirement that all DMA allocations must be marked.

I think finding out the allocations is one of the most common examples
for ftrace. I followed the instructions from
https://www.kernel.org/doc/Documentation/trace/events.txt to
show me a histogram of all allocations under 256 bytes, which
(one kernel compile later) gives me something like

$echo 'hist:key=call_site.sym-offset:val=bytes_req:sort=bytes_req.descending
if bytes_req<256' >   \
/sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
$ make -skj30
...
$ head  /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
{ call_site: [ffffffffc04e457f]
btrfs_delete_delayed_dir_index+0xbf/0x1e0 [btrfs]       } hitcount:
 146914  bytes_req:   16454368
{ call_site: [ffffffffbbe601a3] generic_file_buffered_read+0x463/0x4a0
                 } hitcount:      98187  bytes_req:   14906232
{ call_site: [ffffffffc0497b81] btrfs_buffered_write+0x131/0x7e0
[btrfs]                } hitcount:     156513  bytes_req:   10038544
{ call_site: [ffffffffc05125c9] btrfs_alloc_block_rsv+0x29/0x60
[btrfs]                 } hitcount:     155044  bytes_req:    8682464
{ call_site: [ffffffffbbfe7272] kernfs_fop_open+0xc2/0x290
                 } hitcount:      38764  bytes_req:    5892128
{ call_site: [ffffffffbbfb6ea2] load_elf_binary+0x242/0xed0
                 } hitcount:      58276  bytes_req:    3729664
{ call_site: [ffffffffc04b52d0] __btrfs_map_block+0x1f0/0xb60 [btrfs]
                 } hitcount:      29289  bytes_req:    3521656
{ call_site: [ffffffffbbf7ac7e] inotify_handle_inode_event+0x7e/0x210
                 } hitcount:      61688  bytes_req:    2986992
{ call_site: [ffffffffbbf2fa35] alloc_pipe_info+0x65/0x230
                 } hitcount:      13139  bytes_req:    2312464
{ call_site: [ffffffffbc0cd3ec] security_task_alloc+0x9c/0x100
                 } hitcount:      60475  bytes_req:    2177100
{ call_site: [ffffffffbc0cd5f6] security_prepare_creds+0x76/0xa0
                 } hitcount:     266124  bytes_req:    2128992
{ call_site: [ffffffffbbfe710e] kernfs_get_open_node+0x7e/0x120
                 } hitcount:      38764  bytes_req:    1860672
{ call_site: [ffffffffc04e1fbd] btrfs_alloc_delayed_item+0x1d/0x50
[btrfs]              } hitcount:      11859  bytes_req:    1833383
{ call_site: [ffffffffc046595d] split_item+0x8d/0x2e0 [btrfs]
                 } hitcount:      14049  bytes_req:    1716288
{ call_site: [ffffffffbbfb6dbc] load_elf_binary+0x15c/0xed0
                 } hitcount:      58276  bytes_req:    1631728
{ call_site: [ffffffffbbf40e79] __d_alloc+0x179/0x1f0
                 } hitcount:      24814  bytes_req:    1280649
{ call_site: [ffffffffbbf5203f] single_open+0x2f/0xa0
                 } hitcount:      34541  bytes_req:    1105312
{ call_site: [ffffffffc047ad0a] btrfs_wq_submit_bio+0x4a/0xe0 [btrfs]
                 } hitcount:       7746  bytes_req:    1053456
{ call_site: [ffffffffbc519e95] xhci_urb_enqueue+0xf5/0x3c0
                 } hitcount:       5511  bytes_req:     484968
{ call_site: [ffffffffc0482935] btrfs_opendir+0x25/0x70 [btrfs]
                 } hitcount:      60245  bytes_req:     481960
{ call_site: [ffffffffc04c44ff] overwrite_item+0x1cf/0x5c0 [btrfs]
                 } hitcount:       7378  bytes_req:     364305
{ call_site: [ffffffffc04c4514] overwrite_item+0x1e4/0x5c0 [btrfs]
                 } hitcount:       7378  bytes_req:     364305
{ call_site: [ffffffffc04e207f] btrfs_wq_run_delayed_node+0x2f/0x80
[btrfs]             } hitcount:       3427  bytes_req:     356408
{ call_site: [ffffffffbbe7e96d] shmem_symlink+0xbd/0x250
                 } hitcount:       5169  bytes_req:     242943
{ call_site: [ffffffffc03e0526] hid_input_field+0x56/0x290 [hid]
                 } hitcount:      11004  bytes_req:     175760

I think these are all safe for the GFP_NODMA approach you suggest, maybe
not the xhci_urb_enqueue one.

          Arnd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel