[net PATCH 12/13] fbnic: Wait for the outbound flush to complete on mailbox teardown
flat view
WARM2d
From: Alexander Duyck <hidden>
Date: 2026-10-28 21:37:31
Subsystem:
meta ethernet drivers, networking drivers, the rest · Maintainers:
Alexander Duyck, Jakub Kicinski, Andrew Lunn, "David S. Miller", Eric Dumazet, Paolo Abeni, Linus Torvalds
From: Alexander Duyck <alexanderduyck@fb.com> The fbnic_mbx_reset_desc_ring function drops bus mastering and arms the outbound read and write flush in a single register write, but nothing waits for that flush to finish. The fbnic_mbx_clean function returns as soon as the rings are torn down, so the caller is free to let the device reset while the outbound path still owns read requests it has not retired. A completion landing after that point, or a tag handed back out while the old one is still live, has nowhere valid to go. The PUL link reset status register reports when the flush has drained in both directions, so poll it before returning. The flush only has to retire what is already in flight, so it settles well inside a microsecond; the 1ms cap is there so a wedged device cannot spin. Every caller of the fbnic_mbx_clean function runs with the datapath already down, which matters because the flush covers all outstanding reads, not just the mailbox ones. A device that has lost its link reads back as all ones, so both done bits appear set and the poll ends on the first pass. Fixes: 3b12f00ddd08 ("fbnic: Gate AXI read/write enabling on FW mailbox") Signed-off-by: Alexander Duyck <alexanderduyck@fb.com> --- drivers/net/ethernet/meta/fbnic/fbnic_csr.h | 4 ++++ drivers/net/ethernet/meta/fbnic/fbnic_fw.c | 16 ++++++++++++++++ 2 files changed, 20 insertions(+)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
index cbb25d11638c..d7583243e5cd 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h@@ -955,6 +955,10 @@ enum { /* PUL User Registers */ #define FBNIC_CSR_START_PUL_USER 0x31000 /* CSR section delimiter */ +#define FBNIC_PUL_USER_IP_LINK_RST_STATUS \ + 0x31030 /* 0xc40c0 */ +#define FBNIC_PUL_USER_IP_LINK_RST_STATUS_AW_FLUSH_DONE CSR_BIT(7) +#define FBNIC_PUL_USER_IP_LINK_RST_STATUS_AR_FLUSH_DONE CSR_BIT(8) #define FBNIC_PUL_OB_TLP_HDR_AW_CFG 0x3103d /* 0xc40f4 */ #define FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH_MODE CSR_BIT(20) #define FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH CSR_BIT(19)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
index 09d03274be08..32a18cb8990c 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c@@ -7,6 +7,7 @@ #include <linux/dev_printk.h> #include <linux/dma-mapping.h> #include <linux/gfp.h> +#include <linux/iopoll.h> #include <linux/types.h> #include "fbnic.h"
@@ -187,10 +188,25 @@ static void fbnic_mbx_clean_desc_ring(struct fbnic_dev *fbd, int mbx_idx) void fbnic_mbx_clean(struct fbnic_dev *fbd) { + u32 flush_done = FBNIC_PUL_USER_IP_LINK_RST_STATUS_AR_FLUSH_DONE | + FBNIC_PUL_USER_IP_LINK_RST_STATUS_AW_FLUSH_DONE; + u32 val; int i; for (i = 0; i < FBNIC_IPC_MBX_INDICES; i++) fbnic_mbx_clean_desc_ring(fbd, i); + + /* Arming the flush above only starts it. Wait for the outbound path + * to retire the requests it still owns so none of them can complete + * after the reset that follows. A lost link reads back as all ones, + * which ends the poll on the first pass. + */ + if (read_poll_timeout_atomic(fbnic_rd32, val, + (val & flush_done) == flush_done, 2, 1000, + false, fbd, + FBNIC_PUL_USER_IP_LINK_RST_STATUS)) + dev_warn(fbd->dev, "Outbound flush did not complete: %08x\n", + val); } #define FBNIC_MBX_MAX_PAGE_SIZE FIELD_MAX(FBNIC_IPC_MBX_DESC_LEN_MASK)