[PATCH net-next V2 0/7] net/mlx5: Improve representor lifecycle and allow switchdev by default
From: Tariq Toukan <tariqt@nvidia.com>
Date: 2026-05-01 04:17:40
Also in:
linux-rdma, lkml
Hi, Find detailed description by Mark below. Regards, Tariq This series addresses three problems that have been present for years, and fixes one representor reload error-unwind case exposed while making the reload path reusable. First, there is no coordination between E-Switch reconfiguration and representor registration. The E-Switch can be mid-way through a mode change or VF count update while mlx5_ib walks in and registers or unregisters representors. Nothing stops them. The race window is small and there is no field report, but it is clearly wrong. Second, loading mlx5_ib while the device is already in switchdev mode does not bring up the IB representors. mlx5_eswitch_register_vport_reps() only stores callbacks; nobody triggers the actual load after registration. Third, deployments that always use switchdev mode still need a manual devlink mode change after probe. That makes automated provisioning more fragile than needed. The series fixes the registration race with a per-E-Switch representor mutex. The lock is introduced first, then LAG shared-FDB and multiport E-Switch transitions are adjusted so auxiliary device rescans and IB representor reloads do not hold ldev->lock while taking the representor lock. This keeps the intermediate commits bisectable before the stricter E-Switch serialization and lock assertions are enabled. After the LAG ordering is fixed, all E-Switch reconfiguration paths that create, destroy, load, or unload representors take the representor mutex. esw_mode_change() deliberately drops the mutex around mlx5_rescan_drivers_locked(), because auxiliary probe and remove paths re-enter mlx5_eswitch_register_vport_reps() and mlx5_eswitch_unregister_vport_reps() on the same thread. The shared-FDB peer IB registration path can hold one E-Switch representor mutex and then register peer representor ops on another E-Switch. The series annotates that case as nested locking so lockdep can distinguish it from recursive locking on the same E-Switch. For the missing IB representors, mlx5_eswitch_register_vport_reps() queues a work item that acquires the devlink lock and loads all relevant representors. This is the change that actually fixes the long-standing bug. The reload path also learns to track which representor types were loaded by the current attempt, so an error does not unload representors that were already active before the retry. Patch 1 is cleanup. LAG and MPESW had the same representor reload sequence duplicated in several places and the copies had started to drift. This consolidates them into one helper. Patch 2 adds the per-E-Switch representor lifecycle lock and helper APIs. Patch 3 adjusts the LAG shared-FDB and multiport E-Switch transitions so auxiliary device rescans and IB representor reloads run without ldev->lock held while taking the representor lock. Patch 4 protects the E-Switch reconfiguration, representor registration and peer IB representor paths with the representor lock. Patch 5 fixes representor load error unwind so only representor types loaded by the current attempt are unloaded on failure. Patch 6 moves the representor load triggered by mlx5_eswitch_register_vport_reps() onto the work queue. This is the patch that fixes IB representors not coming up when mlx5_ib is loaded while the device is already in switchdev mode. Patch 7 adds a driver profile that auto-enables switchdev at device init, for deployments that always operate in switchdev mode and want to avoid a manual devlink command after every probe. V2: Split v1 into two parts: the E-Switch workqueue deadlock fix and the representor lifecycle changes. This is the second part; the first part has already been accepted [1]. Patch 1: Add a cont_on_fail flag so callers can decide whether reload should continue after a failure. Patches 2, 3, 4: Replace the atomic-variable based scheme with a mutex, per Jakub's feedback. Patch 5: New patch that fixes the unwind on representor load failure. Patch 7: Switch from profile 4 to profile 8. Since the profile mainly targets E-Switch handling, keep it separate from the NIC profiles. [1] https://lore.kernel.org/all/20260428051018.219093-1-tariqt@nvidia.com/ (local) Link to V1: https://lore.kernel.org/all/20260409115550.156419-1-tariqt@nvidia.com/ (local) Mark Bloch (7): net/mlx5: Lag: refactor representor reload handling net/mlx5: E-Switch, add representor lifecycle lock net/mlx5: Lag, avoid LAG and representor lock cycles net/mlx5: E-Switch, serialize representor lifecycle net/mlx5: E-Switch, unwind only newly loaded representor types net/mlx5: E-switch, load reps via work queue after registration net/mlx5: Add profile to auto-enable switchdev mode at device init drivers/infiniband/hw/mlx5/ib_rep.c | 6 +- .../net/ethernet/mellanox/mlx5/core/eswitch.c | 10 + .../net/ethernet/mellanox/mlx5/core/eswitch.h | 12 ++ .../mellanox/mlx5/core/eswitch_offloads.c | 186 ++++++++++++++++-- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 171 ++++++++++++---- .../net/ethernet/mellanox/mlx5/core/lag/lag.h | 5 + .../ethernet/mellanox/mlx5/core/lag/mpesw.c | 18 +- .../ethernet/mellanox/mlx5/core/lib/devcom.c | 8 + .../ethernet/mellanox/mlx5/core/lib/devcom.h | 1 + .../net/ethernet/mellanox/mlx5/core/main.c | 43 +++- .../ethernet/mellanox/mlx5/core/sf/devlink.c | 5 + include/linux/mlx5/driver.h | 2 + include/linux/mlx5/eswitch.h | 6 + 13 files changed, 405 insertions(+), 68 deletions(-) base-commit: 4e37987362bcac8909f2d4b4458f3aa645e41641 -- 2.44.0