Re: [bisected] xfrm: TCP connection initiating PMTU discovery stalls on v3.
From: Wolfgang Walter <hidden>
Date: 2014-12-10 18:34:52
Am Dienstag, 9. Dezember 2014, 13:40:19 schrieb Eric Dumazet:
quoted hunk ↗ jump to hunk
On Tue, 2014-12-09 at 21:36 +0100, Wolfgang Walter wrote:quoted
How would that be done? I found no way to disable it especially for xfrm. I disabled gso for the interface which serves the ipsec traffic but this does not help. tcp still uses gso for the esp tunnel. I put a view printk's in net/xfrm/xfrm_output.c and net/ipv4/tcp_output.c. (I try to understand where in the xfrm transformation gso is handeled). What I can say yet is: xfrm_output() is used with ipsec (esp) tunnel mode but at I never see gso packets here. xfrm_output_gso() is never called. Everytime tcp_set_skb_tso_segs() is called for a tcp connection over the esp- tunnel and it is a gso case then the tcp connection hangs. Those packets always have skb->len 1398 and mss_now is 1374. I see a call of xfrm_output() afterwards but for a packet of skb->len 52 (maybe ACK from other direction?). As long as the tcp-connection over the ipsec-tunnel works and if I send bulk traffic xfrm_output() is called 3 times with packet skb->len 1426 and then one time with 78 (maybe other direction?), don't know if that is of any interest. With non-ipsec-traffic gso works fine: in this case the skb->len() varies a lot and mss_now is always 1288.Presumably something happens so that sk_can_gso() returns false. But apparently, this happens _after_ tcp_sendmsg(), ie a bit too late. Note that after https://patchwork.ozlabs.org/patch/418506/ is applied, fix would simply be :diff --git a/net/core/sock.c b/net/core/sock.cindex 9a56b2000c3f374fb95aedada3327447816a9512..678ef8393680dc781445c5f121719ea8e a7bb7c1 100644 --- a/net/core/sock.c+++ b/net/core/sock.c@@ -1585,6 +1585,8 @@ void sk_setup_caps(struct sock *sk, struct dst_entry*dst) sk->sk_gso_max_size = dst->dev->gso_max_size; sk->sk_gso_max_segs = dst->dev->gso_max_segs; } + } else { + sk->sk_gso_max_segs = 1; } } EXPORT_SYMBOL_GPL(sk_setup_caps);
Ok. I tried that. I backported https://patchwork.ozlabs.org/patch/418506/ to 3.14.26 and then applied this patch. Still hangs. At that stage sk_can_gso(sk) is true. The reason is that the interface of *dst in sk_setup_caps() is the interface which serves the ipsec traffic. There is a difference, though. If I disable gso for the interface which serves the ipsec traffic then the hang disappears now. This seems to be because of the patch above: sk_can_gso(sk) in sk_setup_caps then returns false and the ne else branch is taken. So there may be a hidden bug in the non-gso case. Back to xfrm case with hang: I see: a call of sk_setup_caps() which takes the path sk->sk_route_caps |= NETIF_F_SG | NETIF_F_HW_CSUM; ... So gso is on. When the hang happens sk_setup_caps is called from inet_sk_rebuild_header(). Now the path sk->sk_route_caps &= ~NETIF_F_GSO_MASK; is taken as dst->header_len is now non zero. This is the reason why later calls of sk_can_gso() return false. I'll try to change the above patch to
@@ -1585,6 +1585,8 @@ void sk_setup_caps(struct sock *sk, struct dst_entry*dst) sk->sk_gso_max_size = dst->dev->gso_max_size;
sk->sk_gso_max_segs = dst->dev->gso_max_segs;
}
}
+ if (sk_can_gso(sk)) {
+ sk->sk_gso_max_segs = 1;
}
}
EXPORT_SYMBOL_GPL(sk_setup_caps);
so that the case that GSO is disabled because of dst->header_len != 0 sets
sk_gso_max_segs, too.
Regards,
--
Wolfgang Walter
Studentenwerk München
Anstalt des öffentlichen Rechts