From: Dimitri Daskalakis <redacted>
Since commit 46bcce5dfd33 ("net/mlx5e: Support ethtool tcp-data-split
settings"), mlx5 no longer reports tcp-data-split from
mlx5e_ethtool_get_ringparam() and the reply carries only the stored
config. With the default config (auto) the attribute is omitted, so the
device looks like it does not support tcp-data-split, even while HW-GRO
is on and SHAMPO is splitting headers.
On mlx5, header-data split is performed by SHAMPO, which is used only
when HW-GRO is enabled. The driver already keeps HW-GRO on while
tcp-data-split is enabled, but nothing enforces the other direction.
A user can disable tcp-data-split and then enable rx-gro-hw, leaving
the device splitting while the config says it is off.
To make the feature handling consistent, report tcp-data-split as on
when SHAMPO is active and off otherwise. Only allow disabling
tcp-data-split while HW-GRO is off, and keep HW-GRO off in
mlx5e_fix_features() when tcp-data-split is already disabled.
Signed-off-by: Dimitri Daskalakis <redacted>
---
Changes in v4:
- Nimrod and sashiko pointed out that not reporting DISABLED for
tcp-data-split makes the feature look unsupported. This conflicted
with my attempt to block users from setting DISABLED, locally sashiko
found several scenarios where ring changes would get blocked.
Instead I took a different approach, if a user sets hw-gro on ->
tcp-data-split on -> hw-gro off mlx forces hw-gro on. I mirrored this
in the opposite direction, i.e force hw-gro off if a user sets
tcp-data-split off -> hw-gro on.
- Clarify extack messages so users know the sequence of commands to run
to enable/disable tcp-data-split.
- Directly going between tcp-data-split on <-> off is intentionally
unsupported. Going from off -> on would require forcing HW-GRO on,
which would require careful validation with features (XDP, XSK) that
cannot support HW-GRO.
- Link to v3: https://lore.kernel.org/netdev/20261009025137.2631215-1-dimitri.daskalakis1@gmail.com/ (local)
Changes in v3:
- Drop redundant hw_features check in mlx5e_ethtool_get_ringparam(). Packet
merge type can only be MLX5E_PACKET_MERGE_SHAMPO when HW GRO is active.
- Address Bjorn's feedback, and drop premature netdev_update_features() call
in mlx5e_ethtool_set_tcp_data_split()
- Link to v2: https://lore.kernel.org/all/20261006225925.568263-1-dimitri.daskalakis1@gmail.com/ (local)
Changes in v2:
- Leave kernel_param->tcp_data_split as ETHTOOL_TCP_DATA_SPLIT_UNKNOWN if the
device does not support HW GRO
- Prevent users from disabling tcp-data-split
- Link to v1: https://lore.kernel.org/all/20260923230521.1267511-1-dimitri.daskalakis1@gmail.com/ (local)
hds.py with GRO off
# Interface: eth0, driver: mlx5_core
TAP version 13
1..13
ok 1 hds.get_hds
ok 2 hds.get_hds_thresh
ok 3 hds.set_hds_disable
ok 4 hds.set_hds_enable # SKIP disabling of HDS not supported by the device
ok 5 hds.set_hds_thresh_random # SKIP hds-thresh-max is too small
ok 6 hds.set_hds_thresh_zero
ok 7 hds.set_hds_thresh_max
ok 8 hds.set_hds_thresh_gt
ok 9 hds.set_xdp
ok 10 hds.enabled_set_xdp # SKIP enabling of HDS not supported by the device
ok 11 hds.ioctl
ok 12 hds.ioctl_set_xdp
ok 13 hds.ioctl_enabled_set_xdp # SKIP enabling of HDS not supported by the device
# Totals: pass:9 fail:0 xfail:0 xpass:0 skip:4 error:0
hds.py with GRO on
# Interface: eth0, driver: mlx5_core
TAP version 13
1..13
ok 1 hds.get_hds
ok 2 hds.get_hds_thresh
ok 3 hds.set_hds_disable # SKIP disabling of HDS not supported by the device
ok 4 hds.set_hds_enable
ok 5 hds.set_hds_thresh_random # SKIP hds-thresh-max is too small
ok 6 hds.set_hds_thresh_zero
ok 7 hds.set_hds_thresh_max
ok 8 hds.set_hds_thresh_gt
# Exception| Traceback (most recent call last):
...
# Exception| CMD: ip link set dev eth0 xdp obj /tmp/ksft-net-drv/net/lib/xdp_dummy.bpf.o sec xdp
# Exception| EXIT: 2
# Exception| STDERR: RTNETLINK answers: Invalid argument
# Exception|
not ok 9 hds.set_xdp
ok 10 hds.enabled_set_xdp
ok 11 hds.ioctl
# Exception| Traceback (most recent call last):
...
# Exception| CMD: ip link set dev eth0 xdp obj /tmp/ksft-net-drv/net/lib/xdp_dummy.bpf.o sec xdp
# Exception| EXIT: 2
# Exception| STDERR: RTNETLINK answers: Invalid argument
# Exception|
not ok 12 hds.ioctl_set_xdp
ok 13 hds.ioctl_enabled_set_xdp
# Totals: pass:9 fail:2 xfail:0 xpass:0 skip:2 error:0
---
.../ethernet/mellanox/mlx5/core/en_ethtool.c | 21 +++++++++++++++----
.../net/ethernet/mellanox/mlx5/core/en_main.c | 7 +++++++
2 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 1f534051cdfe..8e7c748caf2d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -388,6 +388,13 @@ static void mlx5e_get_ringparam(struct net_device *dev,
struct mlx5e_priv *priv = netdev_priv(dev);
mlx5e_ethtool_get_ringparam(priv, param, kernel_param);
+
+ if (dev->hw_features & NETIF_F_GRO_HW)
+ kernel_param->tcp_data_split =
+ priv->channels.params.packet_merge.type ==
+ MLX5E_PACKET_MERGE_SHAMPO ?
+ ETHTOOL_TCP_DATA_SPLIT_ENABLED :
+ ETHTOOL_TCP_DATA_SPLIT_DISABLED;
}
static bool mlx5e_ethtool_set_tcp_data_split(struct mlx5e_priv *priv,
@@ -399,7 +406,14 @@ static bool mlx5e_ethtool_set_tcp_data_split(struct mlx5e_priv *priv,
if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
!(dev->features & NETIF_F_GRO_HW)) {
NL_SET_ERR_MSG_MOD(extack,
- "TCP-data-split is not supported when GRO HW is disabled");
+ "TCP-data-split can not be enabled without HW-GRO, set it to 'auto' and enable HW-GRO first");
+ return false;
+ }
+
+ if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_DISABLED &&
+ dev->features & NETIF_F_GRO_HW) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "TCP-data-split can not be disabled while HW-GRO is on, set it to 'auto' and disable HW-GRO first");
return false;
}
@@ -474,9 +488,8 @@ static int mlx5e_set_ringparam(struct net_device *dev,
if (err)
return err;
- /* Might need to disable HW-GRO if it was kept on due to hds. */
- if (kernel_param->tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_DISABLED &&
- dev->cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_ENABLED)
+ /* Might need to adjust HW-GRO if it was kept on/off due to hds. */
+ if (dev->cfg->hds_config != kernel_param->tcp_data_split)
netdev_update_features(priv->netdev);
return 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index c3c2c5720c6f..e2f657b36824 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -4627,6 +4627,13 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
features |= NETIF_F_GRO_HW;
}
+ /* HW GRO cannot be enabled if header-data split is disabled. */
+ if (cfg && cfg->hds_config == ETHTOOL_TCP_DATA_SPLIT_DISABLED &&
+ features & NETIF_F_GRO_HW) {
+ netdev_warn(netdev, "Disabling HW-GRO, TCP header-data split is disabled\n");
+ features &= ~NETIF_F_GRO_HW;
+ }
+
if (mlx5e_is_uplink_rep(priv)) {
features = mlx5e_fix_uplink_rep_features(netdev, features);
netdev->netns_immutable = true;--
2.52.0