Thread (5 messages) 5 messages, 2 authors, 2025-07-15

Re: [PATCH net-next v4] tcp: extend tcp_retransmit_skb tracepoint with failure reasons

From: <fan.yu9@zte.com.cn>
Date: 2025-07-15 04:35:49
Also in: linux-trace-kernel, lkml

quoted
Solution
========
Adds a "result" field to the tcp_retransmit_skb tracepoint,
enumerating with explicit failure cases:
TCP_RETRANS_ERR_DEFAULT (retransmit terminate unexpectedly)
TCP_RETRANS_IN_HOST_QUEUE (packet still queued in driver)
TCP_RETRANS_END_SEQ_ERROR (invalid end sequence)
TCP_RETRANS_NOMEM (retransmit no memory)
TCP_RETRANS_ROUTE_FAIL (routing failure)
TCP_RETRANS_RCV_ZERO_WINDOW (closed receiver window)
Have you tried to use this or perform some analysis of which of these
reasons actually make sense to add? I'd venture a guess that
IN_HOST_QUEUE will dominate in datacenter. Maybe RCV_ZERO_WINDOW
can happen. Tracing ENOMEM is a waste of time, so is this:
Hi Jakub,
Thanks for the feedback. This patch was motivated by a issue
where TCP retransmissions were silently failing, and we eventually
traced the problem to packets stuck in the driver queue(IN_HOST_QUEUE).

However, the current tcp_retransmit_skb tracepoint lacks visibility
into why retransmissions fail. Our goal is to to make tcp_retransmit_skb
more useful by not only tracking retransmission attempts but also their
outcomes. This aligns with the tracepoint’s original purpose but adds
actionable diagnostics.
         if (unlikely(before(TCP_SKB_CB(skb)->end_seq, tp->snd_una))) {
            >>>>>    WARN_ON_ONCE(1);  <<<<<<<<
-            return -EINVAL;
+            result = TCP_RETRANS_END_SEQ_ERROR;
I agree that some of the result types (e.g., ENOMEM, END_SEQ_ERROR)
may be redundant or unlikely in practice. If we focus only on the most
critical cases, would the following subset be more acceptable?
- TCP_RETRANS_FAIL_QUEUED (packet stuck in host/driver queue)
- TCP_RETRANS_FAIL_ZERO_WINDOW (receiver window closed)
- TCP_RETRANS_FAIL_ROUTE (routing issues)
- TCP_RETRANS_FAIL_DEFAULT (catch-all for unexpected failures)

Best regards.



Original


From: JakubKicinski <kuba@kernel.org>
To: 范雨10344752;
Cc: edumazet@google.com <edumazet@google.com>;kuniyu@amazon.com <redacted>;ncardwell@google.com <ncardwell@google.com>;davem@davemloft.net <davem@davemloft.net>;netdev@vger.kernel.org <redacted>;linux-kernel@vger.kernel.org <redacted>;linux-trace-kernel@vger.kernel.org <redacted>;杨洋10192021;徐鑫10311587;涂强10171646;蒋昆10222859;
Date: 2025年07月15日 07:46
Subject: Re: [PATCH net-next v4] tcp: extend tcp_retransmit_skb tracepoint with failure reasons

On Thu, 10 Jul 2025 10:01:38 +0800 (CST) fan.yu9@zte.com.cn wrote:
Background
==========
When TCP retransmits a packet due to missing ACKs, the
retransmission may fail for various reasons (e.g., packets
stuck in driver queues, sequence errors, or routing issues).
 
The original tcp_retransmit_skb tracepoint:
'commit e086101b150a ("tcp: add a tracepoint for tcp retransmission")' 
lacks visibility into these failure causes, making production
diagnostics difficult.
 
Solution
========
Adds a "result" field to the tcp_retransmit_skb tracepoint,
enumerating with explicit failure cases:
TCP_RETRANS_ERR_DEFAULT (retransmit terminate unexpectedly)
TCP_RETRANS_IN_HOST_QUEUE (packet still queued in driver)
TCP_RETRANS_END_SEQ_ERROR (invalid end sequence)
TCP_RETRANS_NOMEM (retransmit no memory)
TCP_RETRANS_ROUTE_FAIL (routing failure)
TCP_RETRANS_RCV_ZERO_WINDOW (closed receiver window)
 
Have you tried to use this or perform some analysis of which of these
reasons actually make sense to add? I'd venture a guess that
IN_HOST_QUEUE will dominate in datacenter. Maybe RCV_ZERO_WINDOW
can happen. Tracing ENOMEM is a waste of time, so is this:
 
         if (unlikely(before(TCP_SKB_CB(skb)->end_seq, tp->snd_una))) {
            >>>>>    WARN_ON_ONCE(1);  <<<<<<<< 
-            return -EINVAL;
+            result = TCP_RETRANS_END_SEQ_ERROR;
--  
pw-bot: cr
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help