a wrinkle to consider when benchmarking with LRO

At the risk of pointing-out the known/obvious...

I have been evaluating a system and its ability to source data across a 10G 
link.  My number of 10G sink's is a triffle minimal.  To maximize the sinks' 
ability to sink data I enabled large receive offload.  The kernel on the sinks 
was 2.6.21.1.  The kernel on the source was "another OS."

While TSO doesn't especially change what is seen on the wire, LRO can.  When the 
NIC/driver combines the segments, the receiving TCP sends fewer ACKs.  In short, 
LRO is an implicit/backdoor ACK avoidance heuristic.

So, the sending TCP will receive fewer ACKs and will have a correspondingly 
lower CPU overhead and so will be able to source data faster than if it were 
sending to a system without LRO enabled.

I'm not trying to say this is a bad thing - in fact, I have gone on record a 
number of times supporting good ACK avoidance heuristics - I just wanted to make 
sure this behaviour I was seeing was on the record somewhere as it may not be 
obvious to everyone.

happy benchmarking,

rick jones