Re: [PATCH net] net: tcp: don't allocate fast clones for fastopen SYN

On Thu, Mar 4, 2021 at 10:20 PM Eric Dumazet [off-list ref] wrote:

On Thu, Mar 4, 2021 at 10:08 PM Jonathan Lemon [off-list ref] wrote:

quoted

On Thu, Mar 04, 2021 at 08:41:45PM +0100, Eric Dumazet wrote:

quoted

On Thu, Mar 4, 2021 at 8:06 PM Jakub Kicinski [off-list ref] wrote:

quoted

On Thu, 4 Mar 2021 13:51:15 +0100 Eric Dumazet wrote:

quoted

I think we are over thinking this really (especially if the fix needs
a change in core networking or drivers)

We can reuse TSQ logic to have a chance to recover when the clone is
eventually freed.
This will be more generic, not only for the SYN+data of FastOpen.

Can you please test the following patch ?

#7 - Eric comes up with something much better :)


But so far doesn't seem to quite do it, I'm looking but maybe you'll
know right away (FWIW testing a v5.6 backport but I don't think TSQ
changed?):

On __tcp_retransmit_skb kretprobe:

==> Hit TFO case ret:-16 ca_state:0 skb:ffff888fdb4bac00!

First hit:
        __tcp_retransmit_skb+1
        tcp_rcv_state_process+2488
        tcp_v6_do_rcv+405
        tcp_v6_rcv+2984
        ip6_protocol_deliver_rcu+180
        ip6_input_finish+17

Successful hit:
        __tcp_retransmit_skb+1
        tcp_retransmit_skb+18
        tcp_retransmit_timer+716
        tcp_write_timer_handler+136
        tcp_write_timer+141
        call_timer_fn+43

 skb:ffff888fdb4bac00 --- delay:51642us bytes_acked:1

Humm maybe one of the conditions used in tcp_tsq_write() does not hold...

if (tp->lost_out > tp->retrans_out &&
    tp->snd_cwnd > tcp_packets_in_flight(tp)) {
    tcp_mstamp_refresh(tp);
    tcp_xmit_retransmit_queue(sk);
}

Maybe FastOpen case is 'special' and tp->lost_out is wrong.

Something like this?  (completely untested)
--
Jonathan

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 69a545db80d2..92bc9b0f4955 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c

@@ -5995,8 +5995,10 @@ static bool tcp_rcv_fastopen_synack(struct sock *sk, struct sk_buff *synack,
                else
                        tp->fastopen_client_fail = TFO_DATA_NOT_ACKED;
                skb_rbtree_walk_from(data) {
+                       tcp_mark_skb_lost(sk, data);
                        if (__tcp_retransmit_skb(sk, data, 1))
                                break;
+                       tp->retrans_out += tcp_skb_pcount(data);
                }
                tcp_rearm_rto(sk);
                NET_INC_STATS(sock_net(sk),

Yes, but the hard part is testing this ;)

Once we properly mark the skb lost, we can call regular
tcp_xmit_retransmit_queue() to not have to care of tp->retrans_out

Not that we really need to make sure tcp_xmit_retransmit_queue() can
be called anyway from TSQ handler if TX completion is delayed.

I was testing this (note how I also removed the tcp_rearm_rto(sk) call)

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 6f450e577975c7be9537338c8a4c0673d7d36c4c..9ef92ca55e530f76ad793d7342442c4ec06165f7

100644

--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c

@@ -6471,11 +6471,10 @@ static bool tcp_rcv_fastopen_synack(struct

sock *sk, struct sk_buff *synack,
        tcp_fastopen_cache_set(sk, mss, cookie, syn_drop, try_exp);

        if (data) { /* Retransmit unacked data in SYN */
-               skb_rbtree_walk_from(data) {
-                       if (__tcp_retransmit_skb(sk, data, 1))
-                               break;
-               }
-               tcp_rearm_rto(sk);
+               skb_rbtree_walk_from(data)
+                       tcp_mark_skb_lost(sk, data);
+
+               tcp_xmit_retransmit_queue(sk);
                NET_INC_STATS(sock_net(sk),
                                LINUX_MIB_TCPFASTOPENACTIVEFAIL);
                return true;