Hi
I am posting this question to "netdev" mailing list because I could no longer find "linux-net" mailing list.
I find that the Linux TCP stack consumes huge amount of CPU if the MTU of an interface is set to 2400 and it is receiving 1000 byte Ethernet packets. On the other hand, if the MTU is set to 1500, the CPU consumption is reduced drastically. Increased CPU usage causes network throughput to drop considerably (from 800-900 Mbps to 200 Mbps). My kernel version is fedora core 6 and we are using 1 Gig NICs (Intel 82546GB and Broadcom NetXtreme BCM5721):
Linux he7700-tg 2.6.22.14-72.fc6 #1 SMP Wed Nov 21 14:10:25 EST 2007 x86_64 x86_64 x86_64 GNU/Linux
I do not know the TCP buffer management internals and how they are affected by MTU. Is there some FAQ/information online or do I have to open up the source code and try to identify the source of the problem. I guess I can also try newer versions of the kernel and see if the issue has been resolved.
-Pawan
On 08/16/2011 11:33 AM, Pawan Singh wrote:
Hi
I am posting this question to "netdev" mailing list because I could
no longer find "linux-net" mailing list.
I find that the Linux TCP stack consumes huge amount of CPU if the
MTU of an interface is set to 2400 and it is receiving 1000 byte
Ethernet packets. On the other hand, if the MTU is set to 1500, the
CPU consumption is reduced drastically. Increased CPU usage causes
network throughput to drop considerably (from 800-900 Mbps to 200
Mbps). My kernel version is fedora core 6 and we are using 1 Gig NICs
(Intel 82546GB and Broadcom NetXtreme BCM5721):
Linux he7700-tg 2.6.22.14-72.fc6 #1 SMP Wed Nov 21 14:10:25 EST 2007
x86_64 x86_64 x86_64 GNU/Linux
Based on my experience, you are best off using much more contemporary kernels when bringing things to the attention of the netdev list. The list is much more concerned with current bits than those from years past.
Issues with bits from years past are probably best addressed towards the distro(s) using them, if any.
Looking into getting a oprofile profile of the behaviour, particularly if it still exists in a contemporary kernel would probably be a good idea.
Also good ideas would be checking the netstat statistics before and after a test run, and looking at packet traces. You want to look for things like stats on collapsing buffers and packets being dropped.
rick jones
I do not know the TCP buffer management internals and how they are
affected by MTU. Is there some FAQ/information online or do I have to
open up the source code and try to identify the source of the
problem. I guess I can also try newer versions of the kernel and see
if the issue has been resolved.
-Pawan
Le mardi 16 août 2011 à 11:33 -0700, Pawan Singh a écrit :
Hi
I am posting this question to "netdev" mailing list because I could no
longer find "linux-net" mailing list.
I find that the Linux TCP stack consumes huge amount of CPU if the MTU
of an interface is set to 2400 and it is receiving 1000 byte Ethernet
packets. On the other hand, if the MTU is set to 1500, the CPU
consumption is reduced drastically. Increased CPU usage causes network
throughput to drop considerably (from 800-900 Mbps to 200 Mbps). My
kernel version is fedora core 6 and we are using 1 Gig NICs (Intel
82546GB and Broadcom NetXtreme BCM5721):
Linux he7700-tg 2.6.22.14-72.fc6 #1 SMP Wed Nov 21 14:10:25 EST 2007
x86_64 x86_64 x86_64 GNU/Linux
I do not know the TCP buffer management internals and how they are
affected by MTU. Is there some FAQ/information online or do I have to
open up the source code and try to identify the source of the problem.
I guess I can also try newer versions of the kernel and see if the
issue has been resolved.
I suspect you have a CPU problem on the receiver side, not on transmit
one ?
MTU has a direct impact on skb 'truesize'. As the linux tcp stack takes
care of not using too much kernel ram for the sole use of one socket,
you might hit a per socket limit faster with big MTU (and not filled RX
buffers)
When this limit is hit, tcp stack tries to 'collapse' consecutive skbs
to remove the unused parts of skbs. This is a very expensive process.
With a recent kernel, you could use 'perf top' tool and easily spot the
hot paths in kernel.