[next-next PATCH v2 2/2] fbnic: Remove BMC routing rules when the BMC channel is disabled
flat view
COOLING5d
From: Alexander Duyck <hidden>
Date: 2026-09-28 16:12:17
Subsystem:
meta ethernet drivers, networking drivers, the rest · Maintainers:
Alexander Duyck, Jakub Kicinski, Andrew Lunn, "David S. Miller", Eric Dumazet, Paolo Abeni, Linus Torvalds
From: Alexander Duyck <alexanderduyck@fb.com> While a BMC is present the driver programs MACDA entries for the BMC addresses, action rules steering matching traffic to it, the DEST_BMC copy bit on the multicast/broadcast RSS actions, and a host-unicast copy of the RSS actions. All of these are only torn down when the host interface goes down. A BMC can disable its NC-SI channel while the host interface stays up. The filters are then left in place, so the host keeps steering traffic to a BMC that is no longer there, and the driver logs "Found BMC MAC address w/ BMC not present" at the next interface down. Detect that bmc_present dropped while BMC tagged rules are still programmed in fbnic_bmc_rpc_check(), and remove the BMC MAC entries, the action rule and the host-unicast RSS entries. That needs three supporting changes: 1. Action rules must be marked for deletion in any live state rather than only VALID, as a missed deletion leaks the rule for good. The same applies to the BMC all-multi rule, which ifdown leaves in UPDATE. 2. The host-unicast entries have to go because fbnic_rss_reinit() only programs them while a BMC is present and would otherwise leave them stale and valid in hardware. 3. Recompute the RSS actions for both directions; without that a disable and re-enable cycle leaves the BMC without its multicast/broadcast copies until the interface bounces. Take the instance lock over the whole of fbnic_bmc_rpc_check() while here. fbnic_fw_xmit_rpc_macda_sync() walks the entire MACDA shadow and the teardown above rewrites it, but the service task they both run from holds only RTNL. RTNL used to cover that, back when ndo_set_rx_mode() was called inline from dev_set_rx_mode(). fbnic is ops-locked, so netif_rx_mode_run() now calls ndo_set_rx_mode_async() under the instance lock alone, having dropped netif_addr_lock and handed over a snapshot of the address lists. The ethtool NFC paths are in the same position. The instance lock is the one lock every writer of the shadow holds. The flags are tested once without it first so the lock stays off the common path, where the service task has nothing to do. Signed-off-by: Alexander Duyck <alexanderduyck@fb.com> --- drivers/net/ethernet/meta/fbnic/fbnic_fw.h | 4 + drivers/net/ethernet/meta/fbnic/fbnic_rpc.c | 97 ++++++++++++++++++++++++++- 2 files changed, 99 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
index e542b17bdac7..f47669129c9d 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h@@ -55,6 +55,10 @@ enum { FBNIC_FW_CAP_F_BMC_MACDA_SYNC, }; +#define FBNIC_FW_CAP_BMC_PENDING \ + (BIT(FBNIC_FW_CAP_F_BMC_TCAM_REINIT) | \ + BIT(FBNIC_FW_CAP_F_BMC_MACDA_SYNC)) + struct fbnic_fw_cap { unsigned long state; struct {
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c
index 51fd0564d33f..939d56585b26 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_rpc.c@@ -143,9 +143,11 @@ void fbnic_bmc_rpc_all_multi_config(struct fbnic_dev *fbd, /* If we are not enabling the rule just delete it. We will fall * back to the RSS rules that support the multicast addresses. + * Delete in any live state, not just VALID, as ifdown demotes it to + * UPDATE and a miss there leaves it to be rewritten to hardware. */ if (!fbnic_bmc_all_multi(fbd) || enable_host) { - if (act_tcam->state == FBNIC_TCAM_S_VALID) + if (act_tcam->state != FBNIC_TCAM_S_DISABLED) act_tcam->state = FBNIC_TCAM_S_DELETE; return; }
@@ -234,10 +236,91 @@ void fbnic_bmc_rpc_init(struct fbnic_dev *fbd) act_tcam->state = FBNIC_TCAM_S_UPDATE; } +/** + * fbnic_bmc_rules_present - is the BMC currently programmed into the filters? + * @fbd: Pointer to fbnic device struct + * + * The BMC tag is only set while a BMC is present, so it doubles as the state. + * + * Return: true if any MACDA entry carries the BMC tag, false otherwise. + */ +static bool fbnic_bmc_rules_present(struct fbnic_dev *fbd) +{ + int idx; + + for (idx = ARRAY_SIZE(fbd->mac_addr); idx--;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[idx]; + + if (mac_addr->state == FBNIC_TCAM_S_DISABLED) + continue; + + if (test_bit(FBNIC_MAC_ADDR_T_BMC, mac_addr->act_tcam)) + return true; + } + + return false; +} + +/** + * fbnic_bmc_rpc_disable - remove the BMC MAC and action rules + * @fbd: Pointer to fbnic device struct + * + * Undo fbnic_bmc_rpc_init() when the BMC drops its NC-SI channel while the + * host interface stays up. Only marks shadow state; the caller's + * __fbnic_set_rx_mode() pushes it to hardware, so ordering here is moot. + */ +static void fbnic_bmc_rpc_disable(struct fbnic_dev *fbd) +{ + struct fbnic_act_tcam *act_tcam; + int idx; + + /* Drop the BMC's claim; entries the host also uses survive */ + for (idx = ARRAY_SIZE(fbd->mac_addr); idx--;) { + struct fbnic_mac_addr *mac_addr = &fbd->mac_addr[idx]; + + if (mac_addr->state == FBNIC_TCAM_S_DISABLED) + continue; + + __fbnic_xc_unsync(mac_addr, FBNIC_MAC_ADDR_T_BMC); + } + + /* Delete in any live state, not just VALID: init leaves it UPDATE and + * ifdown demotes VALID to UPDATE. A miss leaks the rule for good, as + * fbnic_bmc_rules_present() keys off the MACDA tag just cleared above. + */ + act_tcam = &fbd->act_tcam[FBNIC_RPC_ACT_TBL_BMC_OFFSET]; + if (act_tcam->state != FBNIC_TCAM_S_DISABLED) + act_tcam->state = FBNIC_TCAM_S_DELETE; + + /* fbnic_rss_reinit() only programs the host-unicast entries while a BMC + * is present, so the reinit after this would leave them stale and valid + * in hardware. Delete them to converge on the no-BMC layout. + */ + for (idx = 0; idx < FBNIC_RSS_EN_NUM_UNICAST; idx++) { + act_tcam = &fbd->act_tcam[FBNIC_RPC_ACT_TBL_RSS_OFFSET + idx]; + if (act_tcam->state != FBNIC_TCAM_S_DISABLED) + act_tcam->state = FBNIC_TCAM_S_DELETE; + } +} + void fbnic_bmc_rpc_check(struct fbnic_dev *fbd) { + struct fbnic_net *fbn = netdev_priv(fbd->netdev); int err; + /* Nothing to do unless the firmware raised one of the flags. Test + * before taking the lock to keep it off the common path; one raised + * after this is picked up on the next pass. + */ + if (!(READ_ONCE(fbd->fw_cap.state) & FBNIC_FW_CAP_BMC_PENDING)) + return; + + /* The rx mode work and the ethtool paths rewrite the MACDA and action + * TCAM shadows under the instance lock, and this runs from the service + * task under RTNL only, so the two do not exclude each other. + */ + netdev_lock(fbd->netdev); + /* Consume the flag before the state it advertises. The ordering * implied by test_and_clear_bit() pairs with the barrier in * fbnic_fw_parse_bmc_cap(), and claiming it atomically means a set
@@ -245,7 +328,15 @@ void fbnic_bmc_rpc_check(struct fbnic_dev *fbd) */ if (test_and_clear_bit(FBNIC_FW_CAP_F_BMC_TCAM_REINIT, &fbd->fw_cap.state)) { - fbnic_bmc_rpc_init(fbd); + if (!fbnic_bmc_present(fbd) && fbnic_bmc_rules_present(fbd)) + fbnic_bmc_rpc_disable(fbd); + else + fbnic_bmc_rpc_init(fbd); + + /* Neither path touches the RSS actions, which carry the mc/bc + * copy to the BMC, so recompute them alongside. + */ + fbnic_rss_reinit(fbd, fbn); netif_addr_lock_bh(fbd->netdev); __fbnic_set_rx_mode(fbd, &fbd->netdev->uc, &fbd->netdev->mc); netif_addr_unlock_bh(fbd->netdev);
@@ -258,6 +349,8 @@ void fbnic_bmc_rpc_check(struct fbnic_dev *fbd) dev_warn(fbd->dev, "Writing MACDA table to FW failed, err: %d\n", err); } + + netdev_unlock(fbd->netdev); } #define FBNIC_ACT1_INIT(_l4, _udp, _ip, _v6) \