Re: QUESTION: can netdev_alloc_skb() errors be reduced by tuning?

Please dont top post, we prefer other way around :)

starlight@binnacle.cx a écrit :

Eric,

Great thought--thank you.  Running a similar server with 
82571/e1000e and it does not exhibit the problem.  'e1000e' has 
default copybreak=256 while 'ixgbe' has no copybreak.  Rational 
given is

   http://osdir.com/ml/linux.drivers.e1000.devel/2008-01/msg00103.html

But the comparion is a bit apples-and-oranges since the 'e1000e' 
system is dual Opteron 2354 while the 'ixgbe' system is Xeon 
E5430 (a painful choice thus far).  Also 'e1000e' system passes 
data via a PACKET socket while the 'ixgbe' system passes data 
via UDP (a configurable option).

I'm not fully up on how this all works: am I to understand that 
the error could result from RX ring-queue buffers not freeing 
quickly enough because they have a use-count held non-zero as
the packet travels the stack?

Well, error is normal in stress situation, when no more kernel
memory is available.

cat /proc/net/udp

can show you (in last column) sockets where packets where dropped
by UDP stack if their receive queue was full.

I've just doubled some SLAB tuneables that seem relevant, but 
if the cause is the aforementioned, this won't help.  Will
have the answer on the tweaks by the end of Tuesday.

David

copybreak in drivers themselves is nice because driver can recycle
its rx skbs much faster, but that is suboptimal in forwarding (routers)
workloads. Its also a lot of duplicated code in every driver.

So we could do the skb trimming (ie : reallocating the data portion to exactly
the size of packet) in core network stack, when we know packet must be handled
by an application, and not dropped or forwarded by kernel.

Because of slab rounding, this reallocation should be done only if resulting data
portion is really smaller (50 %) than original skb.

At 04:26 AM 6/16/2009 +0200, Eric Dumazet wrote:

quoted

152691992335/724246449 = 210 bytes per rx packet in average

It could make sense to add copybreak feature in this driver to 
reduce memory needs, but that also would consume more cpu 
cycles, and slow down forwarding setups.

Maybe this packet trimming could be done generically in UDP 
stack input path, before queueing packet into a receive queue, 
if amount of available memory is under a given threshold.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Eric Dumazet [off-list ref] wrote:

Because of slab rounding, this reallocation should be done only if resulting data
portion is really smaller (50 %) than original skb.

If we're going to do this in the core then we should only do it
in the spots where the packet may be held indefinitely.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} [off-list ref]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>