Re: [RFC net-next] net: thunderbolt: Allow per-device opt-in Tx E2E flow control
From: Mika Westerberg <mika.westerberg@linux.intel.com>
Date: 2026-09-23 07:00:37
Also in:
lkml
Hi, On Tue, Sep 22, 2026 at 11:43:16PM -0700, Yusheng Zheng wrote:
Commit a8065af3346e ("net: thunderbolt: Enable end-to-end flow control
also in transmit") enabled E2E flow control on Tx rings. It was reverted
after an ASMedia ASM4242 accepted the configuration without returning
credits, leaving its Tx ring wedged. A separate report described similar
Tx completion wedges with multiple native E2E rings on AMD Strix Halo.
The revert kept Rx E2E and suggested a Tx opt-in for known-good
controllers.AMD is also working on a patch that prevents the issue.
On two Intel Meteor Lake-P NHI (8086:7ec2) hosts, the exact patch built against their running 7.3-rc3 kernel was tested in a same-boot A/B/A run. Each arm used a ten-second, four-stream iperf3 transfer. For lab to g485, Tx E2E on, off, then on again produced 15.43, 15.76, and 15.18 Gbit/s, with 0, 29113, and 0 retransmissions. G485 Rx errors rose by 5097 during the off arm and did not rise during either on arm. In the reverse direction, throughput was about 19.41 Gbit/s in all three arms, with 0, 181, and 0 retransmissions; lab Rx errors rose by 14 only during the off arm. The link was brought down and up to change the flag, so link training was not held fixed. The result supports an integrity improvement on this pair, not a general throughput gain or proof for other controllers. Both hosts were returned to their distribution module after testing. Add a per-netdev ethtool private flag named tx-e2e. It defaults to off and can only change while the interface is down. Enabling it requires the existing e2e module parameter. At open, the Tx ring uses E2E only if the peer advertises TBNET_E2E. This bit proves protocol support, not working credit return. Administrators must opt in only on controller pairs known to return credits; a bad controller can stall Tx until the flag is disabled.
If we advertise E2E support over XDomain we must enable it as well otherwise the other side that can be something else that Linux will be disapointed. I think for this, we should come up a solution that deals with the NHI wedge problem directly in the TB driver side (and this is what AMD is working on). Then the Thunderbolt service drivers do not need to care. Also this looks pretty much like LLM generated, so you should add Assisted-by tag.
quoted hunk
Link: https://lore.kernel.org/netdev/20260727123002.25225-1-fy15309206903@gmail.com/ (local) Signed-off-by: Yusheng Zheng <redacted> --- RFC question: Is a per-netdev ethtool private flag an acceptable opt-in for known-good controller pairs, given that the peer protocol bit cannot prove that the controller returns Tx E2E credits? Validation: The patched module built with W=1 against the running 7.3-rc3 headers and ran on both Intel hosts. A same-boot A/B/A test is summarized above. Enabling the flag while the interface was up returned EBUSY; ring flags were observed as FRAME without the opt-in and FRAME|E2E with it. A net-next allmodconfig W=1 build was attempted but failed on GCC 15 -Werror diagnostics in unrelated files including samples/trace_events and kernel/locking; it did not provide a clean whole-tree result. drivers/net/thunderbolt/main.c | 60 ++++++++++++++++++++++++++++++++-- 1 file changed, 58 insertions(+), 2 deletions(-)diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c index d9fb587a62c5..b17ab8e15c1f 100644 --- a/drivers/net/thunderbolt/main.c +++ b/drivers/net/thunderbolt/main.c@@ -162,2 +162,3 @@ struct tbnet_ring { * host + * @tx_e2e: Enable end-to-end flow control on the Tx ring * @local_transmit_path: HopID we are using to send out packets@@ -192,2 +193,3 @@ struct tbnet { bool login_received; + bool tx_e2e; int local_transmit_path;@@ -950,3 +952,7 @@ static int tbnet_open(struct net_device *dev) - ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, - RING_FLAG_FRAME); + flags = RING_FLAG_FRAME; + /* The peer bit advertises E2E protocol support. */ + if (net->tx_e2e && tbnet_e2e && net->svc->prtcstns & TBNET_E2E) + flags |= RING_FLAG_E2E; + + ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags); if (!ring) {@@ -1339,3 +1345,53 @@ +static const char tbnet_priv_flags[][ETH_GSTRING_LEN] = { +#define TBNET_PRIV_FLAG_TX_E2E BIT(0) + "tx-e2e", +}; + +static void tbnet_get_strings(struct net_device *dev, u32 stringset, u8 *data) +{ + if (stringset == ETH_SS_PRIV_FLAGS) + memcpy(data, tbnet_priv_flags, sizeof(tbnet_priv_flags)); +} + +static int tbnet_get_sset_count(struct net_device *dev, int sset) +{ + if (sset == ETH_SS_PRIV_FLAGS) + return ARRAY_SIZE(tbnet_priv_flags); + + return -EOPNOTSUPP; +} + +static u32 tbnet_get_priv_flags(struct net_device *dev) +{ + const struct tbnet *net = netdev_priv(dev); + + return net->tx_e2e ? TBNET_PRIV_FLAG_TX_E2E : 0; +} + +static int tbnet_set_priv_flags(struct net_device *dev, u32 flags) +{ + struct tbnet *net = netdev_priv(dev); + bool tx_e2e; + + if (flags & ~TBNET_PRIV_FLAG_TX_E2E) + return -EINVAL; + + tx_e2e = flags & TBNET_PRIV_FLAG_TX_E2E; + if (net->tx_e2e == tx_e2e) + return 0; + if (netif_running(dev)) + return -EBUSY; + if (tx_e2e && !tbnet_e2e) + return -EOPNOTSUPP; + + net->tx_e2e = tx_e2e; + return 0; +} + static const struct ethtool_ops tbnet_ethtool_ops = { .get_link_ksettings = tbnet_get_link_ksettings, + .get_strings = tbnet_get_strings, + .get_sset_count = tbnet_get_sset_count, + .get_priv_flags = tbnet_get_priv_flags, + .set_priv_flags = tbnet_set_priv_flags, }; --2.55.0