RE: [PATCH net-next] TLP: Don't reschedule PTO when there's one outstanding TLP retransmission.

-----Original Message-----
From: Yuchung Cheng [mailto:ycheng@google.com]
Sent: Thursday, July 27, 2017 12:45 AM
To: maowenan
Cc: Neal Cardwell; Netdev; David Miller; weiyongjun (A); Chenweilong; Nandita
Dukkipati; Wangkefeng (Kevin)
Subject: Re: [PATCH net-next] TLP: Don't reschedule PTO when there's one
outstanding TLP retransmission.

On Tue, Jul 25, 2017 at 7:12 PM, maowenan [off-list ref]
wrote:

quoted

quoted

-----Original Message-----
From: Neal Cardwell [mailto:ncardwell@google.com]
Sent: Tuesday, July 25, 2017 9:30 PM
To: maowenan
Cc: Netdev; David Miller; weiyongjun (A); Chenweilong; Yuchung
Cheng; Nandita Dukkipati
Subject: Re: [PATCH net-next] TLP: Don't reschedule PTO when there's
one outstanding TLP retransmission.

On Tue, Jul 25, 2017 at 4:35 AM, Mao Wenan [off-list ref]
wrote:

quoted

If there is one TLP probe went out(TLP use the write_queue_tail
packet as TLP probe, we assume this first TLP probe named A), and
this TLP probe was not acked by receive side.

Then the transmit side sent the next two packetes out(named B,C),
but unfortunately these two packets are also not acked by receive side.

And then there is one data packet with ack_seq A arrive, in
tcp_ack() will call tcp_schedule_loss_probe() to rearm PTO, the
handler
tcp_send_loss_probe() pass if(tp->tlp_high_seq)(because there is
one outstanding TLP named A,tp->tlp_high_seq is not zero), so the
new TLP probe can't be went out and need to rearm the RTO
timer(timeout is relative to the transmit time of the write queue head).

After this, another data packet with ack_seq A is received, if the
tlp_time_stamp is after rto_time_stamp, it will reset the TLP
timeout with delta value, which is before previous RTO timeout, so
PTO is rearm and previous RTO is cleared. This TLP probe also
can't be sent out because of tp->tlp_high_seq != 0, so there is no
way(or need very long time)to retransmit the packet because of TLP A is

lost.

quoted

quoted

quoted

This fix is not to pass the if(tp->tlp_high_seq) in
tcp_schedule_loss_probe() when TLP PTO is after RTO, It is no need
to reschedule PTO when there is one outstanding TLP
retransmission, so if the TLP A is lost then RTO can retransmit
that packet, and
tp->tlp_high_seq will be set to 0. After this TLP will go to the
tp->normal work

process.

quoted

Signed-off-by: Mao Wenan <redacted>

Thanks for posting this. This is a pretty involved scenario. To help
document/test precisely what the behavior is before and after your
patch, would you be able to post a packetdrill (
https://github.com/google/packetdrill ) test case for this scenario?

Can I ask if you saw this scenario in an actual trace, or noticed
this by inspection?

[Mao Wenan] It is my actual product scenario, from some debug trace
info I found that PTO is always rescheduled before RTO, this is PTO is
trigged by receiving one tcp_ack packets. After this patch, the
product works well and there is no previous issue happen. The packet
can retransmit with RTO if TLP probe is lost.
I will say sorry that my local test can't reproduce because i can't
build successfully the same complex scenario, I don't have mature test
case to reproduce now but I will do my best to try in local test environment.

tcpdump output of an example trace also helps. Feel free to remove IP address,
ports, or payload information. We only need the timing and TCP header info.

Ok, please check as below. Server: 192.169.0.18, client: 192.169.0.17
......
20:36:12.239562 IP 192.169.0.18.1234 > 192.169.0.17.56246: P 501216544:501216616(72) ack 2308880351 win 92
20:36:12.240034 IP 192.169.0.18.1234 > 192.169.0.17.56246: P 501216616:501216688(72) ack 2308880351 win 92
20:36:12.260764 IP 192.169.0.18.1234 > 192.169.0.17.56246: P 501216616:501216688(72) ack 2308880351 win 92     /* first TLP probe A, seq: 501216616 */ 
20:36:13.018020 IP 192.169.0.17.56246 > 192.169.0.18.1234: . ack 501216616 win 1970                         
20:36:13.018386 IP 192.169.0.18.1234 > 192.169.0.17.56246: P 501216688:501216760(72) ack 2308880351 win 92     /* server send packet B */
20:36:13.022448 IP 192.169.0.18.1234 > 192.169.0.17.56246: P 501216760:501216832(72) ack 2308880351 win 92     /* server send packet C */
20:36:13.033813 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880351:2308880437(86) ack 501216616 win 1970  /* receive client data packet ack 501216616 */
20:36:13.034213 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880437:2308880523(86) ack 501216616 win 1970  /* receive client data packet ack 501216616 */
20:36:13.034633 IP 192.169.0.18.1234 > 192.169.0.17.56246: . ack 2308880523 win 92                            /* server pure ack client */
20:36:13.039512 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880523:2308880659(136) ack 501216616 win 1970
20:36:13.040083 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880659:2308880795(136) ack 501216616 win 1970
20:36:13.040491 IP 192.169.0.18.1234 > 192.169.0.17.56246: . ack 2308880795 win 92
20:36:13.041161 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880795:2308880931(136) ack 501216616 win 1970
20:36:13.042828 IP 192.169.0.17.56246 > 192.169.0.18.1234: P 2308880931:2308881032(101) ack 501216616 win 1970
20:36:13.043198 IP 192.169.0.18.1234 > 192.169.0.17.56246: . ack 2308881032 win 92
......

quoted

Thank you.

quoted

thanks,
neal