non-growing tcp window size

<html>
<body>
To whom it might concern,<br><br>
We have observed bandwidth problems of TCP when running benchmarks which
gradually increase its payload and having (most of its) data flowing
unidirectional. The degradation is observed between two dual XEON/E7500
machines, using Gbe and a cross-over cable. The Gbe is eth1 and has an
MTU of 9000. eth0 is a FE with an MTU of 1500. The machines is running
linux 2.4.20, and the NIC in question is an Intel Corp. 82544GC Gigabit
Ethernet Controller (rev 02).<br><br>
During an attempt to change TCP parameter settings, and also
systematically changing socket rx/tx buffer sizes, I discovered that once
in a while, the benchmark ran well. The OK run was not deterministic, and
had nothing to do with change of the parameter settings. Hence, I let the
machine work during the week-end, and took tcpdumps which I saved for the
successful run. An analysis of the &quot;OK&quot; run vs. the
&quot;BAD&quot; run, discovers a couple of interesting things. The
problem is that the advertised window of the receiver does not increase.
The second problem, which also affects the &quot;OK&quot; run, is that
the ratio of packets sent (from src to dst) and the number of packets
received (#advertisements) is 1 for the &quot;BAD&quot; scenario
(actually this is probably a consequence of the window being only ~2xMTU
size). Surprisingly, it is as large as 0.5 for the &quot;OK&quot;
scenario. IMHO, this violates RFC813, which states:<br><br>
<dl>
<dd>&nbsp;&nbsp; There are two reasons&nbsp; for&nbsp; prompt&nbsp;
acknowledgement.&nbsp;&nbsp;&nbsp; One&nbsp; is&nbsp; to 
<dd>prevent&nbsp; retransmission.&nbsp; We will discuss later how to
determine whether 
<dd>unnecessary&nbsp; retransmission&nbsp; is&nbsp;
occurring.&nbsp;&nbsp;&nbsp; The&nbsp; other&nbsp;&nbsp;
reason&nbsp;&nbsp; one 
<dd>acknowledges&nbsp; promptly&nbsp; is&nbsp; to permit further data to
be sent.&nbsp; However, 
<dd>the previous section makes quite clear that it is not&nbsp;
always&nbsp; desirable 
<dd>to send a little bit of data, even though the receiver may have room
for 
<dd>it.&nbsp;&nbsp;&nbsp; Therefore,&nbsp; one&nbsp; can&nbsp;
state&nbsp; a&nbsp; general&nbsp; rule&nbsp; that&nbsp; under normal 
<dd>operation, <font color="#FF0000">the receiver of data need not,&nbsp;
and&nbsp; for&nbsp; efficiency&nbsp; reasons 
<dd>should&nbsp; not,&nbsp; acknowledge&nbsp; the data unless either the
acknowledgement is 
<dd>intended to produce an increased useable window</font>, is
necessary&nbsp; in&nbsp; order 
<dd>to&nbsp; prevent&nbsp; retransmission&nbsp; or&nbsp; is&nbsp;
being&nbsp; sent&nbsp; as&nbsp; part&nbsp; of a reverse 
<dd>direction segment being sent for some other reason.&nbsp; We will
consider an 
<dd>algorithm to achieve these goals. 
</dl>The two tcpdumps has been analyzed by a simple awk scripts, which
gives information for every 1/10 of a second of the runtime:<br><br>
time: elapsed time relative to the first packet<br>
MB/s: sum of TCP payload (based on TCP sequence numbers) *1e-6 / delta
time<br>
avg_len: average packet length (TCP payload) sent<br>
nsent: no of packets sent from src to dst<br>
avg_win: average window size advertised by the src<br>
adv/sent: ratio between #advertisements (sum of prompt acks and
piggybacked acks) and #packets sent<br><br>
The extract of the analysis is enclosed as &quot;ok.txt&quot; and
&quot;bad.txt&quot;. Also, the information is presented as graphs in
&quot;bw_winsiz.png&quot; and &quot;bw_adv-sent.png&quot;. In the graphs,
the &quot;OK&quot; data uses the bottom x-axis, the &quot;BAD&quot; data
uses the upper x-axis (the &quot;BAD&quot; run-time is longer due to the
lower bandwidth). Bandwidth uses the left y-axes, whereas the average
advertised window size and the ratio of the #advertisements and #packets
sent uses the rightmost x-axis.<br><br>
I do hope this information is useful 4u and that the problem can be
fixed. Since I do not read the mailing lists, I would appreciate a note
back by email of someone triggers on this.<br><br>
Cheers, Håkon<br><br>
<x-sigsep><p></x-sigsep>
--<br>
Håkon Bugge; VP Product Development; Scali AS;<br>
<a href="mailto:hob@scali.no" eudora="autourl">mailto:hob@scali.no</a>;
<a href="http://www.scali.com/" eudora="autourl">http://www.scali.com</a>;
fax: +47 22 62 89 51;<br>
Voice: +47 22 62 89 50; Cellular (Europe+US): +47 924 84 514;<br>
Visiting Addr: Olaf Helsets vei 6, Bogerud, N-0621 Oslo, Norway;<br>
Mail Addr:&nbsp; Scali AS, Postboks 150, Oppsal, N-0619&nbsp; Oslo,
Norway;<br>
</body>
</html>