[PATCH net-next] net/mlx5: Add reset_prepare and reset_done callbacks
flat view
COOLING9d
REVIEWED: 1 (0M)
From: Tariq Toukan <tariqt@nvidia.com>
Date: 2026-10-11 09:50:49
Also in:
linux-rdma, lkml
Subsystem:
mellanox mlx5 ib driver, mellanox mlx5 core vpi driver, networking drivers, the rest · Maintainers:
Leon Romanovsky, Saeed Mahameed, Tariq Toukan, Mark Bloch, Andrew Lunn, "David S. Miller", Eric Dumazet, Jakub Kicinski, Paolo Abeni, Linus Torvalds
1 review trailer.
From: Akiva Goldberger <redacted> Currently, the mlx5_core driver registers only the error_detected, slot_reset and resume PCI error handlers. Add the reset_prepare and reset_done callbacks, which are triggered by the PCI core before and after a PCI reset, respectively. These callbacks let mlx5 quiesce and restore device resources and commands around a PCI reset, so the driver no longer relies on the reset happening while the device is otherwise idle. Additionally, add a device flag to indicate when a PCI reset is in progress to serialize PCI reset flows with other reset flows, namely FW reset. This flag also indicates to the reset_prepare and reset_done callbacks when to do a full unload/reload of the driver. When the callbacks are triggered by the hot-reset FW reset method, they leave the teardown and reload to the FW reset. Signed-off-by: Akiva Goldberger <redacted> Reviewed-by: Moshe Shemesh <redacted> Signed-off-by: Tariq Toukan <tariqt@nvidia.com> --- .../ethernet/mellanox/mlx5/core/fw_reset.c | 7 ++ .../net/ethernet/mellanox/mlx5/core/main.c | 89 ++++++++++++++++++- include/linux/mlx5/driver.h | 1 + 3 files changed, 96 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c
index 7c0585fe1ee7..2ba2f6dac263 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c@@ -501,6 +501,13 @@ static void mlx5_sync_reset_request_event(struct work_struct *work) return; } + if (dev->pci_reset_in_progress) { + err = mlx5_fw_reset_set_reset_sync_nack(dev); + mlx5_core_warn(dev, "PCI reset in progress, Sync FW Update Reset Nack %s", + err ? "Failed" : "Sent"); + goto unlock; + } + if (mlx5_sync_reset_set_reset_requested(dev)) goto unlock;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 1a45474ce03d..fe7c0331b183 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c@@ -2138,10 +2138,97 @@ static void mlx5_pci_resume(struct pci_dev *pdev) !err ? "recovered" : "Failed"); } +static void mlx5_pci_reset_prepare(struct pci_dev *pdev) +{ + struct mlx5_core_dev *dev = pci_get_drvdata(pdev); + struct devlink *devlink; + bool fw_reset_in_prog; + + if (!dev) + return; + + devlink = priv_to_devlink(dev); + + devl_lock(devlink); + fw_reset_in_prog = mlx5_fw_reset_in_progress(dev); + if (!fw_reset_in_prog) + dev->pci_reset_in_progress = true; + devl_unlock(devlink); + + if (fw_reset_in_prog) { + mlx5_core_info(dev, + "%s: fw reset in progress, leaving drain to fw_reset flow\n", + __func__); + return; + } + + mlx5_core_info(dev, "%s: resetting, device state = %d, pci_status = %d, intf_state = 0x%lx\n", + __func__, dev->state, dev->pci_status, dev->intf_state); + + mlx5_enter_error_state(dev, true); + mlx5_drain_health_wq(dev); + mlx5_unload_one(dev, false); + mlx5_pci_disable_device(dev); +} + +static void mlx5_pci_reset_done(struct pci_dev *pdev) +{ + struct mlx5_core_dev *dev = pci_get_drvdata(pdev); + struct devlink *devlink; + int err; + + if (!dev) + return; + + devlink = priv_to_devlink(dev); + + if (!dev->pci_reset_in_progress) { + mlx5_core_info(dev, + "%s: reset owned by fw_reset flow, leaving reload to it\n", + __func__); + return; + } + + err = mlx5_pci_enable_device(dev); + if (err) { + mlx5_core_err(dev, "%s: mlx5_pci_enable_device failed, err = %d\n", + __func__, err); + goto clear_pci_reset_flag; + } + + pci_set_master(pdev); + err = wait_vital(pdev); + if (err) { + mlx5_core_err(dev, "%s: wait_vital failed, err = %d\n", + __func__, err); + goto disable; + } + + err = mlx5_load_one(dev, true); + if (err) { + mlx5_core_err(dev, "%s: mlx5_load_one failed, err = %d\n", + __func__, err); + goto disable; + } + + mlx5_core_info(dev, "%s: reset done, device state = %d, pci_status = %d\n", + __func__, dev->state, dev->pci_status); + goto clear_pci_reset_flag; + +disable: + mlx5_pci_disable_device(dev); +clear_pci_reset_flag: + devl_lock(devlink); + dev->pci_reset_in_progress = false; + devl_unlock(devlink); +} + static const struct pci_error_handlers mlx5_err_handler = { .error_detected = mlx5_pci_err_detected, .slot_reset = mlx5_pci_slot_reset, - .resume = mlx5_pci_resume + .resume = mlx5_pci_resume, + .reset_prepare = mlx5_pci_reset_prepare, + .reset_done = mlx5_pci_reset_done, }; static int mlx5_try_fast_unload(struct mlx5_core_dev *dev)
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index abb99371ff13..7b9f3cd23e0b 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h@@ -746,6 +746,7 @@ struct mlx5_core_dev { /* sync pci state */ struct mutex pci_status_mutex; enum mlx5_pci_status pci_status; + bool pci_reset_in_progress; u8 rev_id; char board_id[MLX5_BOARD_ID_LEN]; struct mlx5_cmd cmd;
base-commit: d8674294aefef02266c4d47ad10131f1bffbe534 -- 2.44.0