Re: [PATCH RFC 0/7] add socket to netdev page frag recycling support
From: Yunsheng Lin <hidden>
Date: 2021-08-18 09:38:20
Also in:
bpf, lkml
On 2021/8/18 16:57, Eric Dumazet wrote:
On Wed, Aug 18, 2021 at 5:33 AM Yunsheng Lin [off-list ref] wrote:quoted
This patchset adds the socket to netdev page frag recycling support based on the busy polling and page pool infrastructure.I really do not see how this can scale to thousands of sockets. tcp_mem[] defaults to ~ 9 % of physical memory. If you now run tests with thousands of sockets, their skbs will consume Gigabytes of memory on typical servers, now backed by order-0 pages (instead of current order-3 pages) So IOMMU costs will actually be much bigger.
As the page allocator support bulk allocating now, see: https://elixir.bootlin.com/linux/latest/source/net/core/page_pool.c#L252 if the DMA also support batch mapping/unmapping, maybe having a small-sized page pool for thousands of sockets may not be a problem? Christoph Hellwig mentioned the batch DMA operation support in below thread: https://www.spinics.net/lists/netdev/msg666715.html if the batched DMA operation is supported, maybe having the page pool is mainly benefit the case of small number of socket?
Are we planning to use Gigabyte sized page pools for NIC ? Have you tried instead to make TCP frags twice bigger ?
Not yet.
This would require less IOMMU mappings. (Note: This could require some mm help, since PAGE_ALLOC_COSTLY_ORDER is currently 3, not 4)
I am not familiar with mm yet, but I will take a look about that:)
quoted hunk ↗ jump to hunk
diff --git a/net/core/sock.c b/net/core/sock.c index a3eea6e0b30a7d43793f567ffa526092c03e3546..6b66b51b61be9f198f6f1c4a3d81b57fa327986a100644--- a/net/core/sock.c +++ b/net/core/sock.c@@ -2560,7 +2560,7 @@ static void sk_leave_memory_pressure(struct sock *sk) } } -#define SKB_FRAG_PAGE_ORDER get_order(32768) +#define SKB_FRAG_PAGE_ORDER get_order(65536) DEFINE_STATIC_KEY_FALSE(net_high_order_alloc_disable_key); /**quoted