Re: [PATCH net-next v4] tcp: extend tcp_retransmit_skb tracepoint with failure reasons
From: <fan.yu9@zte.com.cn>
Date: 2025-07-15 04:35:49
Also in:
linux-trace-kernel, lkml
quoted
Solution ======== Adds a "result" field to the tcp_retransmit_skb tracepoint, enumerating with explicit failure cases: TCP_RETRANS_ERR_DEFAULT (retransmit terminate unexpectedly) TCP_RETRANS_IN_HOST_QUEUE (packet still queued in driver) TCP_RETRANS_END_SEQ_ERROR (invalid end sequence) TCP_RETRANS_NOMEM (retransmit no memory) TCP_RETRANS_ROUTE_FAIL (routing failure) TCP_RETRANS_RCV_ZERO_WINDOW (closed receiver window)Have you tried to use this or perform some analysis of which of these reasons actually make sense to add? I'd venture a guess that IN_HOST_QUEUE will dominate in datacenter. Maybe RCV_ZERO_WINDOW can happen. Tracing ENOMEM is a waste of time, so is this:
Hi Jakub, Thanks for the feedback. This patch was motivated by a issue where TCP retransmissions were silently failing, and we eventually traced the problem to packets stuck in the driver queue(IN_HOST_QUEUE). However, the current tcp_retransmit_skb tracepoint lacks visibility into why retransmissions fail. Our goal is to to make tcp_retransmit_skb more useful by not only tracking retransmission attempts but also their outcomes. This aligns with the tracepoint’s original purpose but adds actionable diagnostics.
if (unlikely(before(TCP_SKB_CB(skb)->end_seq, tp->snd_una))) {
>>>>> WARN_ON_ONCE(1); <<<<<<<<
- return -EINVAL;
+ result = TCP_RETRANS_END_SEQ_ERROR;I agree that some of the result types (e.g., ENOMEM, END_SEQ_ERROR) may be redundant or unlikely in practice. If we focus only on the most critical cases, would the following subset be more acceptable? - TCP_RETRANS_FAIL_QUEUED (packet stuck in host/driver queue) - TCP_RETRANS_FAIL_ZERO_WINDOW (receiver window closed) - TCP_RETRANS_FAIL_ROUTE (routing issues) - TCP_RETRANS_FAIL_DEFAULT (catch-all for unexpected failures) Best regards. Original From: JakubKicinski <kuba@kernel.org> To: 范雨10344752; Cc: edumazet@google.com <edumazet@google.com>;kuniyu@amazon.com <redacted>;ncardwell@google.com <ncardwell@google.com>;davem@davemloft.net <davem@davemloft.net>;netdev@vger.kernel.org <redacted>;linux-kernel@vger.kernel.org <redacted>;linux-trace-kernel@vger.kernel.org <redacted>;杨洋10192021;徐鑫10311587;涂强10171646;蒋昆10222859; Date: 2025年07月15日 07:46 Subject: Re: [PATCH net-next v4] tcp: extend tcp_retransmit_skb tracepoint with failure reasons On Thu, 10 Jul 2025 10:01:38 +0800 (CST) fan.yu9@zte.com.cn wrote:
Background
==========
When TCP retransmits a packet due to missing ACKs, the
retransmission may fail for various reasons (e.g., packets
stuck in driver queues, sequence errors, or routing issues).
The original tcp_retransmit_skb tracepoint:
'commit e086101b150a ("tcp: add a tracepoint for tcp retransmission")'
lacks visibility into these failure causes, making production
diagnostics difficult.
Solution
========
Adds a "result" field to the tcp_retransmit_skb tracepoint,
enumerating with explicit failure cases:
TCP_RETRANS_ERR_DEFAULT (retransmit terminate unexpectedly)
TCP_RETRANS_IN_HOST_QUEUE (packet still queued in driver)
TCP_RETRANS_END_SEQ_ERROR (invalid end sequence)
TCP_RETRANS_NOMEM (retransmit no memory)
TCP_RETRANS_ROUTE_FAIL (routing failure)
TCP_RETRANS_RCV_ZERO_WINDOW (closed receiver window)
Have you tried to use this or perform some analysis of which of these
reasons actually make sense to add? I'd venture a guess that
IN_HOST_QUEUE will dominate in datacenter. Maybe RCV_ZERO_WINDOW
can happen. Tracing ENOMEM is a waste of time, so is this:
if (unlikely(before(TCP_SKB_CB(skb)->end_seq, tp->snd_una))) {
>>>>> WARN_ON_ONCE(1); <<<<<<<<
- return -EINVAL;
+ result = TCP_RETRANS_END_SEQ_ERROR;
--
pw-bot: cr