RE: linux-next network throughput performance regression

-----Original Message-----
From: David Miller [mailto:davem@davemloft.net]
Sent: Monday, November 9, 2015 10:53
To: Dexuan Cui <decui@microsoft.com>
Cc: eric.dumazet@gmail.com; dsa@cumulusnetworks.com; Simon Xiao
[off-list ref]; netdev@vger.kernel.org; Haiyang Zhang
[off-list ref]; linux-kernel@vger.kernel.org;
devel@linuxdriverproject.org
Subject: Re: linux-next network throughput performance regression

From: Dexuan Cui <decui@microsoft.com>
Date: Mon, 9 Nov 2015 02:39:24 +0000

quoted

quoted

Throughput on a single TCP flow for a 40G NIC can be tricky to tune.

Why is a single TCP flow trickier than multiple TCP flows?
IMO it should be easier to analyze the issue of a single TCP flow?

Because a single TCP flow can only use one of the many TX queues
that such modern NICs have.

The single TX queue becomes the bottleneck.

Whereas if you have several TCP flows, all of them can use independant
TX queues on the NIC in parallel to fill the link with traffic.

That's why.

Thanks, David!
I understand 1 TX queue is the bottleneck (however in Simon's
test, TX=1 => 36.7Gb/s, TX=8 => 37.7 Gb/s, so it looks the TX=1 bottleneck
is not so obvious).
I'm just wondering how the bottleneck became much narrower with
recent linux-next in Simon's result (36.7 Gb/s vs. 18.2 Gb/s). IMO there
must be some latency somewhere.

Thanks,
-- Dexuan