On Mon, Apr 27, 2026 at 06:55:21PM -0700, Benjamin Berman wrote:
The default TBNET_RING_SIZE of 256 and the NAPI_POLL_WEIGHT of 64
implicit in netif_napi_add() are too small for host-to-host Thunderbolt
networking under sustained bulk traffic. Running NCCL all-reduce over
tb-lo on a three-node chain (two TB3 endpoints plus a TB4 Maple Ridge
transit) produces rx_missed_errors at ~1 % of rx_packets on the transit
and ~0.6 % on the endpoints, with rx_packets stalling against a peer's
continuing tx_packets.
Raise TBNET_RING_SIZE to 2048 (8x) and use netif_napi_add_weight() with
a per-NAPI weight of 256 so tbnet_poll() drains more frames per softirq
invocation. With matching sysctls (net.core.netdev_budget=1024,
net.core.netdev_budget_usecs=8000) rx_missed_errors stays below 0.005 %
over a 192 GB all-reduce workload on the same hardware.
Generated-by: Claude Opus 4.7 [off-list ref]
Tested-by: Benjamin Berman <redacted>
Signed-off-by: Benjamin Berman <redacted>
For ring size I don't have any objections. The current ring size 256 is
arbitrary and at the time seemed reasonable.
For the poll weigth there is the comment in netdevice.h:
/* Default NAPI poll() weight
* Device drivers are strongly advised to not use bigger value
*/
#define NAPI_POLL_WEIGHT 64
But if you see improvement using 256 here I'm fine with that unless the
network folks advice otherwise.
Acked-by: Mika Westerberg <mika.westerberg@linux.intel.com>