On Thu, 2012-02-16 at 14:40 +0100, Eric Dumazet wrote:

Le jeudi 16 février 2012 à 08:29 +0100, Eric Dumazet a écrit :

quoted

So I took the time to setup a netem in my lab (and had to fix netem by
the way). Gigabit link.

On sender :
tc qdisc add dev vlan.103 root netem delay 50ms

And netperf session is a bit strange, since receiver window is about
1Mbytes (17097*64) but sender never uses half of it.

cwnd is ~408

$ ss -emoi  dst 192.168.20.108
State      Recv-Q Send-Q                                 Local
Address:Port                                     Peer Address:Port   
ESTAB      0      450976
192.168.20.110:52017
192.168.20.108:44169    timer:(on,235ms,0) ino:21711 sk:f24d2d00
	 mem:(r0,w698880,f206336,t0) ts sack ecn bic wscale:6,8 rto:252
rtt:52/0.75 cwnd:408 send 90.9Mbps rcv_space:14600

$ tc -s -d qdisc show dev vlan.103
qdisc netem 8009: root refcnt 2 limit 1000 delay 50.0ms
 Sent 15149167430 bytes 10017438 pkt (dropped 0, overlimits 0 requeues
0) 
 rate 74878Kbit 6193pps backlog 476760b 316p requeues 0 

Its seems a problem with GRO.

Receiver opens its window each time it sends an ACK, without taking care
how many segments were ACKed.

In these traces, I give the ACKS sent by receiver, first trace GRO on,
second with GRO off

We can see that with GRO off, receiver opens its window way faster
(against number of received bytes)

(REC & SENDER are both 3.0 kernels, so problem is quite old)

[...]

I'm aware of this problem and I believe it exists with most
implementations of LRO.

The out-of-tree version of the sfc driver has its own soft-LRO
implementation (SSR) which does some limited connection tracking to
detect slow start and disable aggregation temporarily.  I've been
meaning to look into enhancing GRO to match SSR, but haven't got round
to it yet.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.