Re: [PATCH net-next 1/3] net: allow > 0 order atomic page alloc in skb_page_frag_refill
From: Eric Dumazet <hidden>
Date: 2014-01-03 22:54:53
Also in:
virtualization
On Fri, 2014-01-03 at 17:47 -0500, Debabrata Banerjee wrote:
quoted hunk ↗ jump to hunk
quoted
quoted
On Thu, 2014-01-02 at 16:56 -0800, Eric Dumazet wrote: Hmm... it looks like I missed __GFP_NORETRYdiff --git a/net/core/sock.c b/net/core/sock.c index 5393b4b719d7..5f42a4d70cb2 100644 --- a/net/core/sock.c +++ b/net/core/sock.c@@ -1872,7 +1872,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t prio) gfp_t gfp = prio; if (order) - gfp |= __GFP_COMP | __GFP_NOWARN; + gfp |= __GFP_COMP | __GFP_NOWARN | __GFP_NORETRY; pfrag->page = alloc_pages(gfp, order); if (likely(pfrag->page)) { pfrag->offset = 0;There is another patch needed (looks like good stable fixes):diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 06e72d3..d42d48c 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c@@ -378,7 +378,7 @@ refill: gfp_t gfp = gfp_mask; if (order) - gfp |= __GFP_COMP | __GFP_NOWARN; + gfp |= __GFP_COMP | __GFP_NOWARN |__GFP_NORETRY; nc->frag.page = alloc_pages(gfp, order); if (likely(nc->frag.page)) break;
This is in GFP_ATOMIC cases, I dont think it can ever start compaction.
This reduces the really pathological compact/reclaim behavior but doesn't fix it. Actually it still really quite bad because the whole thing loops until it gets to order-0 so it's effectively trying the allocation 4 times anyway. I typically see non-zero order allocations very rarely without these two pieces of code. I hotpatched a running system to get results from this quickly. Even setting the max order to order-1 I still see bad behavior. If anything this behavior should be conditional until this is ironed out. Performance data: http://pastebin.ubuntu.com/6687527/
It seems that you shoot the messenger : If memory is fragmented, then one order-1 allocation is going to start compaction. It can be a simple fork(). If your workload never fork(), then yes, you never needed compaction. It doesn't really matter to say that which memory allocation triggered compaction, which is a normal step in mm layer. If you believe its badly done, you should ask to mm guys to fix/improve it, not netdev... We are not trying to optimize the kernel behavior for hosts in deep memory pressure. Using order-3 pages in TCP stack improves performance for 99% of the hosts, there might be something wrong on your side ?