Re: [REGRESSION] Warning in tcp_fastretrans_alert() of net/ipv4/tcp_input.c
From: Oleksandr Natalenko <hidden>
Date: 2017-09-17 18:43:25
Hi. Just to note that it looks like disabling RACK and re-enabling FACK prevents warning from happening: net.ipv4.tcp_fack = 1 net.ipv4.tcp_recovery = 0 Hope I get semantics of these tunables right. On pátek 15. září 2017 21:04:36 CEST Oleksandr Natalenko wrote:
Hello. With net.ipv4.tcp_fack set to 0 the warning still appears: === » sysctl net.ipv4.tcp_fack net.ipv4.tcp_fack = 0 » LC_TIME=C dmesg -T | grep WARNING [Fri Sep 15 20:40:30 2017] WARNING: CPU: 1 PID: 711 at net/ipv4/tcp_input.c: 2826 tcp_fastretrans_alert+0x7c8/0x990 [Fri Sep 15 20:40:30 2017] WARNING: CPU: 0 PID: 711 at net/ipv4/tcp_input.c: 2826 tcp_fastretrans_alert+0x7c8/0x990 [Fri Sep 15 20:48:37 2017] WARNING: CPU: 1 PID: 711 at net/ipv4/tcp_input.c: 2826 tcp_fastretrans_alert+0x7c8/0x990 [Fri Sep 15 20:48:55 2017] WARNING: CPU: 0 PID: 711 at net/ipv4/tcp_input.c: 2826 tcp_fastretrans_alert+0x7c8/0x990 » ps -up 711 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 711 4.3 0.0 0 0 ? S 18:12 7:23 [irq/123- enp3s0] === Any suggestions? On pátek 15. září 2017 16:03:00 CEST Neal Cardwell wrote:quoted
Thanks for testing that. That is a very useful data point. I was able to cook up a packetdrill test that could put the connection in CA_Disorder with retransmitted packets out, but not in CA_Open. So we do not yet have a test case to reproduce this. We do not see this warning on our fleet at Google. One significant difference I see between our environment and yours is that it seems you run with FACK enabled: net.ipv4.tcp_fack = 1 Note that FACK was disabled by default (since it was replaced by RACK) between kernel v4.10 and v4.11. And this is exactly the time when this bug started manifesting itself for you and some others, but not our fleet. So my new working hypothesis would be that this warning is due to a behavior that only shows up in kernels >=4.11 when FACK is enabled. Would you be able to disable FACK ("sysctl net.ipv4.tcp_fack=0" at boot, or net.ipv4.tcp_fack=0 in /etc/sysctl.conf, or equivalent), reboot, and test the kernel for a few days to see if the warning still pops up? thanks, neal [ps: apologies for the previous, mis-formatted post...]