From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:22:16
From: Vladimir Oltean <vladimir.oltean@nxp.com>
The initial goal of this series was to have better support for
standalone ports mode and multiple bridges on the DSA drivers like
ocelot/felix and sja1105. Proper support for standalone mode requires
disabling address learning, which in turn requires interaction with the
switchdev notifier, which is actually where most of the patches are.
I also noticed that most of the drivers are actually talking either to
firmware or SPI/MDIO connected devices from the brport flags switchdev
attribute handler, so it makes sense to actually make it sleepable
instead of atomic.
Vladimir Oltean (11):
net: switchdev: propagate extack to port attributes
net: bridge: offload all port flags at once in br_setport
net: bridge: don't print in br_switchdev_set_port_flag
net: dsa: configure proper brport flags when ports leave the bridge
net: squash switchdev attributes PRE_BRIDGE_FLAGS and BRIDGE_FLAGS
net: dsa: kill .port_egress_floods overengineering
net: prep switchdev drivers for concurrent
SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS
net: bridge: put SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS on the blocking
call chain
net: mscc: ocelot: use separate flooding PGID for broadcast
net: mscc: ocelot: offload bridge port flags to device
net: dsa: sja1105: offload bridge port flags to device
drivers/net/dsa/b53/b53_common.c | 20 +-
drivers/net/dsa/mv88e6xxx/chip.c | 21 +-
drivers/net/dsa/ocelot/felix.c | 10 +
drivers/net/dsa/sja1105/sja1105.h | 2 +
drivers/net/dsa/sja1105/sja1105_main.c | 212 +++++++++++++++++-
drivers/net/dsa/sja1105/sja1105_spi.c | 6 +
.../marvell/prestera/prestera_switchdev.c | 54 +++--
.../mellanox/mlxsw/spectrum_switchdev.c | 90 ++++----
drivers/net/ethernet/mscc/ocelot.c | 72 +++++-
drivers/net/ethernet/mscc/ocelot_net.c | 7 +-
drivers/net/ethernet/rocker/rocker.h | 2 +-
drivers/net/ethernet/rocker/rocker_main.c | 24 +-
drivers/net/ethernet/rocker/rocker_ofdpa.c | 26 ++-
drivers/net/ethernet/ti/cpsw_switchdev.c | 35 ++-
drivers/staging/fsl-dpaa2/ethsw/ethsw.c | 43 ++--
include/net/dsa.h | 7 +-
include/net/switchdev.h | 14 +-
include/soc/mscc/ocelot.h | 18 +-
net/bridge/br_netlink.c | 162 ++++++-------
net/bridge/br_private.h | 6 +-
net/bridge/br_switchdev.c | 33 ++-
net/bridge/br_sysfs_if.c | 21 +-
net/dsa/dsa_priv.h | 8 +-
net/dsa/port.c | 76 ++++---
net/dsa/slave.c | 10 +-
net/switchdev/switchdev.c | 11 +-
26 files changed, 654 insertions(+), 336 deletions(-)
--
2.25.1
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:24:24
From: Vladimir Oltean <vladimir.oltean@nxp.com>
If for example this command:
ip link set swp0 type bridge_slave flood off mcast_flood off learning off
succeeded at configuring BR_FLOOD and BR_MCAST_FLOOD but not at
BR_LEARNING, there would be no attempt to revert the partial state in
any way. Arguably, if the user changes more than one flag through the
same netlink command, this one _should_ be all or nothing, which means
it should be passed through switchdev as all or nothing.
We also move the br->lock handling inside br_setport in anticipation of
a future patch which will temporarily drop the lock around
br_switchdev_set_port_flag, since we would like that switchdev
notification to be emitted in blocking context.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
Don't attempt to drop br->lock around br_switchdev_set_port_flag now,
move that part to a later patch.
Changes in v2:
Patch is new.
Changes in v2:
Patch is new.
net/bridge/br_netlink.c | 145 ++++++++++++++------------------------
net/bridge/br_switchdev.c | 7 +-
2 files changed, 58 insertions(+), 94 deletions(-)
@@ -853,103 +853,76 @@ static int br_set_port_state(struct net_bridge_port *p, u8 state)}/* Set/clear or port flags based on attribute */-staticintbr_set_port_flag(structnet_bridge_port*p,structnlattr*tb[],-intattrtype,unsignedlongmask)+staticvoidbr_set_port_flag(structnet_bridge_port*p,structnlattr*tb[],+intattrtype,unsignedlongmask){-unsignedlongflags;-interr;-if(!tb[attrtype])-return0;+return;if(nla_get_u8(tb[attrtype]))-flags=p->flags|mask;+p->flags|=mask;else-flags=p->flags&~mask;--err=br_switchdev_set_port_flag(p,flags,mask);-if(err)-returnerr;--p->flags=flags;-return0;+p->flags&=~mask;}/* Process bridge protocol info on port */staticintbr_setport(structnet_bridge_port*p,structnlattr*tb[]){-unsignedlongold_flags=p->flags;-boolbr_vlan_tunnel_old=false;+unsignedlongold_flags,changed_mask;+boolbr_vlan_tunnel_old;interr;-err=br_set_port_flag(p,tb,IFLA_BRPORT_MODE,BR_HAIRPIN_MODE);-if(err)-returnerr;+spin_lock_bh(&p->br->lock);-err=br_set_port_flag(p,tb,IFLA_BRPORT_GUARD,BR_BPDU_GUARD);-if(err)-returnerr;+old_flags=p->flags;+br_vlan_tunnel_old=(old_flags&BR_VLAN_TUNNEL)?true:false;-err=br_set_port_flag(p,tb,IFLA_BRPORT_FAST_LEAVE,BR_MULTICAST_FAST_LEAVE);-if(err)-returnerr;+br_set_port_flag(p,tb,IFLA_BRPORT_MODE,BR_HAIRPIN_MODE);+br_set_port_flag(p,tb,IFLA_BRPORT_GUARD,BR_BPDU_GUARD);+br_set_port_flag(p,tb,IFLA_BRPORT_FAST_LEAVE,+BR_MULTICAST_FAST_LEAVE);+br_set_port_flag(p,tb,IFLA_BRPORT_PROTECT,BR_ROOT_BLOCK);+br_set_port_flag(p,tb,IFLA_BRPORT_LEARNING,BR_LEARNING);+br_set_port_flag(p,tb,IFLA_BRPORT_UNICAST_FLOOD,BR_FLOOD);+br_set_port_flag(p,tb,IFLA_BRPORT_MCAST_FLOOD,BR_MCAST_FLOOD);+br_set_port_flag(p,tb,IFLA_BRPORT_MCAST_TO_UCAST,+BR_MULTICAST_TO_UNICAST);+br_set_port_flag(p,tb,IFLA_BRPORT_BCAST_FLOOD,BR_BCAST_FLOOD);+br_set_port_flag(p,tb,IFLA_BRPORT_PROXYARP,BR_PROXYARP);+br_set_port_flag(p,tb,IFLA_BRPORT_PROXYARP_WIFI,BR_PROXYARP_WIFI);+br_set_port_flag(p,tb,IFLA_BRPORT_VLAN_TUNNEL,BR_VLAN_TUNNEL);+br_set_port_flag(p,tb,IFLA_BRPORT_NEIGH_SUPPRESS,BR_NEIGH_SUPPRESS);+br_set_port_flag(p,tb,IFLA_BRPORT_ISOLATED,BR_ISOLATED);-err=br_set_port_flag(p,tb,IFLA_BRPORT_PROTECT,BR_ROOT_BLOCK);-if(err)-returnerr;+changed_mask=old_flags^p->flags;-err=br_set_port_flag(p,tb,IFLA_BRPORT_LEARNING,BR_LEARNING);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_UNICAST_FLOOD,BR_FLOOD);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_MCAST_FLOOD,BR_MCAST_FLOOD);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_MCAST_TO_UCAST,BR_MULTICAST_TO_UNICAST);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_BCAST_FLOOD,BR_BCAST_FLOOD);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_PROXYARP,BR_PROXYARP);-if(err)-returnerr;--err=br_set_port_flag(p,tb,IFLA_BRPORT_PROXYARP_WIFI,BR_PROXYARP_WIFI);-if(err)-returnerr;--br_vlan_tunnel_old=(p->flags&BR_VLAN_TUNNEL)?true:false;-err=br_set_port_flag(p,tb,IFLA_BRPORT_VLAN_TUNNEL,BR_VLAN_TUNNEL);-if(err)-returnerr;+err=br_switchdev_set_port_flag(p,p->flags,changed_mask);+if(err){+p->flags=old_flags;+gotoout;+}if(br_vlan_tunnel_old&&!(p->flags&BR_VLAN_TUNNEL))nbp_vlan_tunnel_info_flush(p);+br_port_flags_change(p,changed_mask);+if(tb[IFLA_BRPORT_COST]){err=br_stp_set_path_cost(p,nla_get_u32(tb[IFLA_BRPORT_COST]));if(err)-returnerr;+gotoout;}if(tb[IFLA_BRPORT_PRIORITY]){err=br_stp_set_port_priority(p,nla_get_u16(tb[IFLA_BRPORT_PRIORITY]));if(err)-returnerr;+gotoout;}if(tb[IFLA_BRPORT_STATE]){err=br_set_port_state(p,nla_get_u8(tb[IFLA_BRPORT_STATE]));if(err)-returnerr;+gotoout;}if(tb[IFLA_BRPORT_FLUSH])
@@ -999,17 +965,21 @@ static int br_setport(struct net_bridge_port *p, struct nlattr *tb[])if(backup_ifindex){backup_dev=__dev_get_by_index(dev_net(p->dev),backup_ifindex);-if(!backup_dev)-return-ENOENT;+if(!backup_dev){+err=-ENOENT;+gotoout;+}}err=nbp_backup_change(p,backup_dev);if(err)-returnerr;+gotoout;}-br_port_flags_change(p,old_flags^p->flags);-return0;+out:+spin_unlock_bh(&p->br->lock);++returnerr;}/* Change state and parameters on port. */
@@ -1045,9 +1015,7 @@ int br_setlink(struct net_device *dev, struct nlmsghdr *nlh, u16 flags,if(err)returnerr;-spin_lock_bh(&p->br->lock);err=br_setport(p,tb);-spin_unlock_bh(&p->br->lock);}else{/* Binary compatibility with old RSTP */if(nla_len(protinfo)<sizeof(u8))
@@ -1134,17 +1102,10 @@ static int br_port_slave_changelink(struct net_device *brdev,structnlattr*data[],structnetlink_ext_ack*extack){-structnet_bridge*br=netdev_priv(brdev);-intret;-if(!data)return0;-spin_lock_bh(&br->lock);-ret=br_setport(br_port_get_rtnl(dev),data);-spin_unlock_bh(&br->lock);--returnret;+returnbr_setport(br_port_get_rtnl(dev),data);}staticintbr_port_fill_slave_info(structsk_buff*skb,
@@ -65,16 +65,19 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,structswitchdev_attrattr={.orig_dev=p->dev,.id=SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS,-.u.brport_flags=mask,};structswitchdev_notifier_port_attr_infoinfo={.attr=&attr,};interr;-if(mask&~BR_PORT_FLAGS_HW_OFFLOAD)+flags&=BR_PORT_FLAGS_HW_OFFLOAD;+mask&=BR_PORT_FLAGS_HW_OFFLOAD;+if(!mask)return0;+attr.u.brport_flags=mask;+/* We run from atomic context here */err=call_switchdev_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,&info.info,NULL);
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:25:03
From: Vladimir Oltean <vladimir.oltean@nxp.com>
For a DSA switch port operating in standalone mode, address learning
doesn't make much sense since that is a bridge function. In fact,
address learning even breaks setups such as this one:
+---------------------------------------------+
| |
| +-------------------+ |
| | br0 | send receive |
| +--------+-+--------+ +--------+ +--------+ |
| | | | | | | | | |
| | swp0 | | swp1 | | swp2 | | swp3 | |
| | | | | | | | | |
+-+--------+-+--------+-+--------+-+--------+-+
| ^ | ^
| | | |
| +-----------+ |
| |
+--------------------------------+
because if the switch has a single FDB (can offload a single bridge)
then source address learning on swp3 can "steal" the source MAC address
of swp2 from br0's FDB, because learning frames coming from swp2 will be
done twice: first on the swp1 ingress port, second on the swp3 ingress
port. So the hardware FDB will become out of sync with the software
bridge, and when swp2 tries to send one more packet towards swp1, the
ASIC will attempt to short-circuit the forwarding path and send it
directly to swp3 (since that's the last port it learned that address on),
which it obviously can't, because swp3 operates in standalone mode.
So DSA drivers operating in standalone mode should still configure a
list of bridge port flags even when they are standalone. Currently DSA
attempts to call dsa_port_bridge_flags with 0, which disables egress
flooding of unknown unicast and multicast, something which doesn't make
much sense. For the switches that implement .port_egress_floods - b53
and mv88e6xxx, it probably doesn't matter too much either, since they
can possibly inject traffic from the CPU into a standalone port,
regardless of MAC DA, even if egress flooding is turned off for that
port, but certainly not all DSA switches can do that - sja1105, for
example, can't. So it makes sense to use a better common default there,
such as "flood everything".
It should also be noted that what DSA calls "dsa_port_bridge_flags()"
is a degenerate name for just calling .port_egress_floods(), since
nothing else is implemented - not learning, in particular. But disabling
address learning, something that this driver is also coding up for, will
be supported by individual drivers once .port_egress_floods is replaced
with a more generic .port_bridge_flags.
Previous attempts to code up this logic have been in the common bridge
layer, but as pointed out by Ido Schimmel, there are corner cases that
are missed when doing that:
https://patchwork.kernel.org/project/netdevbpf/patch/20210209151936.97382-5-olteanv@gmail.com/
So, at least for now, let's leave DSA in charge of setting port flags
before and after the bridge join and leave.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
Patch is new, logically it was moved from the bridge layer to the DSA
layer.
net/dsa/port.c | 45 ++++++++++++++++++++++++++++++++++++++-------
1 file changed, 38 insertions(+), 7 deletions(-)
@@ -132,10 +153,10 @@ int dsa_port_bridge_join(struct dsa_port *dp, struct net_device *br)};interr;-/* Set the flooding mode before joining the port in the switch */-err=dsa_port_bridge_flags(dp,BR_FLOOD|BR_MCAST_FLOOD);-if(err)-returnerr;+/* Notify the port driver to set its configurable flags in a way that+*matchestheinitialsettingsofabridgeport.+*/+dsa_port_change_brport_flags(dp,true);/* Here the interface is already bridged. Reflect the current*configurationsothatdriverscanprogramtheirchipsaccordingly.
@@ -146,7 +167,7 @@ int dsa_port_bridge_join(struct dsa_port *dp, struct net_device *br)/* The bridging is rolled back on error */if(err){-dsa_port_bridge_flags(dp,0);+dsa_port_change_brport_flags(dp,false);dp->bridge_dev=NULL;}
@@ -172,8 +193,18 @@ void dsa_port_bridge_leave(struct dsa_port *dp, struct net_device *br)if(err)pr_err("DSA: failed to notify DSA_NOTIFIER_BRIDGE_LEAVE\n");-/* Port is leaving the bridge, disable flooding */-dsa_port_bridge_flags(dp,0);+/* Configure the port for standalone mode (no address learning,+*floodeverything).+*ThebridgeonlyemitsSWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGSevents+*whentheuserrequestsitthroughnetlinkorsysfs,butnot+*automaticallyatportjoinorleave,soweneedtohandleresetting+*thebrportflagsourselves.Butweevenpreferitthatway,because+*otherwise,somesetupsmightnevergetthenotificationtheyneed,+*forexample,whenaportleavesaLAGthatoffloadsthebridge,+*itbecomesstandalone,butasfarasthebridgeisconcerned,no+*porteverleft.+*/+dsa_port_change_brport_flags(dp,false);/* Port left the bridge, put in BR_STATE_DISABLED by the bridge layer,*soallowittobeinBR_STATE_FORWARDINGtobekeptfunctional
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:26:05
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Currently br_switchdev_set_port_flag has two options for error handling
and neither is good:
- The driver returns -EOPNOTSUPP in PRE_BRIDGE_FLAGS if it doesn't
support offloading that flag, and this gets silently ignored and
converted to an errno of 0. Nobody does this.
- The driver returns some other error code, like -EINVAL, in
PRE_BRIDGE_FLAGS, and br_switchdev_set_port_flag shouts loudly.
The problem is that we'd like to offload some port flags during bridge
join and leave, but also not have the bridge shout at us if those fail.
But on the other hand we'd like the user to know that we can't offload
something when they set that through netlink. And since we can't have
the driver return -EOPNOTSUPP or -EINVAL depending on whether it's
called by the user or internally by the bridge, let's just add an extack
argument to br_switchdev_set_port_flag and propagate it to its callers.
Then, when we need offloading to really fail silently, this can simply
be passed a NULL argument.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
- Deal with the br_switchdev_set_port_flag call from sysfs too.
Changes in v2:
- br_set_port_flag now returns void, so no extack there.
- don't overwrite extack in br_switchdev_set_port_flag if already
populated.
net/bridge/br_netlink.c | 9 +++++----
net/bridge/br_private.h | 6 ++++--
net/bridge/br_switchdev.c | 13 +++++++------
net/bridge/br_sysfs_if.c | 7 +++++--
4 files changed, 21 insertions(+), 14 deletions(-)
@@ -866,7 +866,8 @@ static void br_set_port_flag(struct net_bridge_port *p, struct nlattr *tb[],}/* Process bridge protocol info on port */-staticintbr_setport(structnet_bridge_port*p,structnlattr*tb[])+staticintbr_setport(structnet_bridge_port*p,structnlattr*tb[],+structnetlink_ext_ack*extack){unsignedlongold_flags,changed_mask;boolbr_vlan_tunnel_old;
@@ -80,14 +81,15 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,/* We run from atomic context here */err=call_switchdev_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,-&info.info,NULL);+&info.info,extack);err=notifier_to_errno(err);if(err==-EOPNOTSUPP)return0;if(err){-br_warn(p->br,"bridge flag offload is not supported %u(%s)\n",-(unsignedint)p->port_no,p->dev->name);+if(extack&&!extack->_msg)+NL_SET_ERR_MSG_MOD(extack,+"bridge flag offload is not supported");return-EOPNOTSUPP;}
@@ -97,8 +99,7 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,err=switchdev_port_attr_set(p->dev,&attr);if(err){-br_warn(p->br,"error setting offload flag on port %u(%s)\n",-(unsignedint)p->port_no,p->dev->name);+NL_SET_ERR_MSG_MOD(extack,"error setting offload flag on port");returnerr;}
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:26:40
From: Vladimir Oltean <vladimir.oltean@nxp.com>
There does not appear to be any strong reason why
br_switchdev_set_port_flag issues a separate notification for checking
the supported brport flags rather than just attempting to apply them and
propagating the error if that fails.
However, there is a reason why this switchdev API is counterproductive
for a driver writer, and that is because although br_switchdev_set_port_flag
gets passed a "flags" and a "mask", those are passed piecemeal to the
driver, so while the PRE_BRIDGE_FLAGS listener knows what changed
because it has the "mask", the BRIDGE_FLAGS listener doesn't, because it
only has the final value. This means that "edge detection" needs to be
done by each individual BRIDGE_FLAGS listener by XOR-ing the old and the
new flags, which in turn means that copying the flags into a driver
private variable is strictly necessary.
This can be solved by passing the "flags" and the "mask" together into
a single switchdev attribute, and it also reduces some boilerplate in
the drivers that offload this.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
- Reworked mlxsw to check mask before performing any change.
- Also adapted the newly introduced dsa_port_change_brport_flags to the
new API.
Changes in v2:
- Renamed "val" to "flags".
- Reworked drivers to check mask before performing any change.
.../marvell/prestera/prestera_switchdev.c | 29 +++++----
.../mellanox/mlxsw/spectrum_switchdev.c | 59 +++++++++----------
drivers/net/ethernet/rocker/rocker_main.c | 24 ++------
drivers/net/ethernet/ti/cpsw_switchdev.c | 32 ++++------
drivers/staging/fsl-dpaa2/ethsw/ethsw.c | 43 +++++++-------
include/net/switchdev.h | 8 ++-
net/bridge/br_switchdev.c | 15 +----
net/dsa/dsa_priv.h | 4 +-
net/dsa/port.c | 43 ++++++--------
net/dsa/slave.c | 3 -
10 files changed, 109 insertions(+), 151 deletions(-)
@@ -908,31 +908,32 @@ static int dpaa2_switch_port_attr_stp_state_set(struct net_device *netdev,returndpaa2_switch_port_set_stp_state(port_priv,state);}-staticintdpaa2_switch_port_attr_br_flags_pre_set(structnet_device*netdev,-unsignedlongflags)-{-if(flags&~(BR_LEARNING|BR_FLOOD))-return-EINVAL;--return0;-}--staticintdpaa2_switch_port_attr_br_flags_set(structnet_device*netdev,-unsignedlongflags)+staticint+dpaa2_switch_port_attr_br_flags_set(structnet_device*netdev,+structswitchdev_brport_flagsflags){structethsw_port_priv*port_priv=netdev_priv(netdev);interr=0;-/* Learning is enabled per switch */-err=dpaa2_switch_set_learning(port_priv->ethsw_data,-!!(flags&BR_LEARNING));-if(err)-gotoexit;+if(flags.mask&~(BR_LEARNING|BR_FLOOD))+return-EINVAL;++if(flags.mask&BR_LEARNING){+/* Learning is enabled per switch */+err=dpaa2_switch_set_learning(port_priv->ethsw_data,+!!(flags.val&BR_LEARNING));+if(err)+returnerr;+}-err=dpaa2_switch_port_set_flood(port_priv,!!(flags&BR_FLOOD));+if(flags.mask&BR_FLOOD){+err=dpaa2_switch_port_set_flood(port_priv,+!!(flags.val&BR_FLOOD));+if(err)+returnerr;+}-exit:-returnerr;+return0;}staticintdpaa2_switch_port_attr_set(structnet_device*netdev,
@@ -945,10 +946,6 @@ static int dpaa2_switch_port_attr_set(struct net_device *netdev,err=dpaa2_switch_port_attr_stp_state_set(netdev,attr->u.stp_state);break;-caseSWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS:-err=dpaa2_switch_port_attr_br_flags_pre_set(netdev,-attr->u.brport_flags);-break;caseSWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS:err=dpaa2_switch_port_attr_br_flags_set(netdev,attr->u.brport_flags);
@@ -65,7 +65,7 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,{structswitchdev_attrattr={.orig_dev=p->dev,-.id=SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS,+.id=SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS,};structswitchdev_notifier_port_attr_infoinfo={.attr=&attr,
@@ -77,7 +77,8 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,if(!mask)return0;-attr.u.brport_flags=mask;+attr.u.brport_flags.val=flags;+attr.u.brport_flags.mask=mask;/* We run from atomic context here */err=call_switchdev_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,
@@ -93,16 +94,6 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,return-EOPNOTSUPP;}-attr.id=SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS;-attr.flags=SWITCHDEV_F_DEFER;-attr.u.brport_flags=flags;--err=switchdev_port_attr_set(p->dev,&attr);-if(err){-NL_SET_ERR_MSG_MOD(extack,"error setting offload flag on port");-returnerr;-}-return0;}
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:27:07
From: Vladimir Oltean <vladimir.oltean@nxp.com>
The bridge offloads the port flags through a single bit mask using
switchdev, which among others, contains learning and flooding settings.
The commit 57652796aa97 ("net: dsa: add support for bridge flags")
missed one crucial aspect of the SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS API
when designing the API one level lower, towards the drivers.
This is that the bitmask of passed brport flags never has more than one
bit set at a time. On the other hand, the prototype passed to the driver
is .port_egress_floods(int port, bool unicast, bool multicast), which
configures two flags at a time.
DSA currently checks if .port_egress_floods is implemented, and if it
is, reports both BR_FLOOD and BR_MCAST_FLOOD as supported. So the driver
has no choice if it wants to inform the bridge that, for example, it
can't configure unicast flooding independently of multicast flooding -
the DSA mid layer is standing in the way. Or the other way around: a new
driver wants to start configuring BR_BCAST_FLOOD separately, but what do
we do with the rest, which only support unicast and multicast flooding?
Do we report broadcast flooding configuration as supported for those
too, and silently do nothing?
Secondly, currently DSA deems the driver too dumb to deserve knowing that
a SWITCHDEV_ATTR_ID_BRIDGE_MROUTER attribute was offloaded, because it
just calls .port_egress_floods for the CPU port. When we'll add support
for the plain SWITCHDEV_ATTR_ID_PORT_MROUTER, that will become a real
problem because the flood settings will need to be held statefully in
the DSA middle layer, otherwise changing the mrouter port attribute will
impact the flooding attribute. And that's _assuming_ that the underlying
hardware doesn't have anything else to do when a multicast router
attaches to a port than flood unknown traffic to it. If it does, there
will need to be a dedicated .port_set_mrouter anyway.
Lastly, we have DSA drivers that have a backlink into a pure switchdev
driver (felix -> ocelot). It seems reasonable that the other switchdev
drivers should not have to suffer from the oddities of DSA overengineering,
so keeping DSA a pass-through layer makes more sense there.
To simplify the brport flags situation we just delete .port_egress_floods
and we introduce a simple .port_bridge_flags which is passed to the
driver. Also, the logic from dsa_port_mrouter is removed and a
.port_set_mrouter is created.
Functionally speaking, we simply move the calls to .port_egress_floods
one step lower, in the two drivers that implement it: mv88e6xxx and b53,
so things should work just as before.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
- Pass extack through the newly introduce dsa_port_change_brport_flags.
Changes in v2:
- Reordered with previous patch such that we don't need to introduce
.port_pre_bridge_flags
- Pass extack to drivers.
drivers/net/dsa/b53/b53_common.c | 20 +++++++++++++++++++-
drivers/net/dsa/mv88e6xxx/chip.c | 21 ++++++++++++++++++++-
include/net/dsa.h | 7 +++++--
net/dsa/dsa_priv.h | 6 ++++--
net/dsa/port.c | 20 +++++++++-----------
net/dsa/slave.c | 4 ++--
6 files changed, 59 insertions(+), 19 deletions(-)
@@ -1948,6 +1948,23 @@ int b53_br_egress_floods(struct dsa_switch *ds, int port,}EXPORT_SYMBOL(b53_br_egress_floods);+staticintb53_br_flags(structdsa_switch*ds,intport,+structswitchdev_brport_flagsflags,+structnetlink_ext_ack*extack)+{+if(flags.mask&~(BR_FLOOD|BR_MCAST_FLOOD))+return-EINVAL;++returnb53_br_egress_floods(ds,port,flags.val&BR_FLOOD,+flags.val&BR_MCAST_FLOOD);+}++staticintb53_set_mrouter(structdsa_switch*ds,intport,boolmrouter,+structnetlink_ext_ack*extack)+{+returnb53_br_egress_floods(ds,port,true,mrouter);+}+staticboolb53_possible_cpu_port(structdsa_switch*ds,intport){/* Broadcom switches will accept enabling Broadcom tags on the
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:27:41
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Since we would like br_switchdev_set_port_flag to not use an atomic
notifier, it should be called from outside spinlock context.
We can temporarily drop br->lock, but that creates some concurrency
complications (example below is given for sysfs):
- There might be an "echo 1 > multicast_flood" simultaneous with an
"echo 0 > multicast_flood". The result of this is nondeterministic
either way, so I'm not too concerned as long as the result is
consistent (no other flags have changed).
- There might be an "echo 1 > multicast_flood" simultaneous with an
"echo 0 > learning". My expectation is that none of the two writes are
"eaten", and the final flags contain BR_MCAST_FLOOD=1 and BR_LEARNING=0
regardless of the order of execution. That is actually possible if, on
the commit path, we don't do a trivial "p->flags = flags" which might
overwrite bits outside of our mask, but instead we just change the
flags corresponding to our mask.
Now that br_switchdev_set_port_flag is never called from under br->lock,
it runs in sleepable context.
All switchdev drivers handle SWITCHDEV_PORT_ATTR_SET as both blocking
and atomic, so no changes are needed on that front.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
- Drop the br->lock around br_switchdev_set_port_flag in this patch, for
both sysfs and netlink.
- Only set/restore the masked bits in p->flags to avoid concurrency
issues.
Changes in v2:
Patch is new.
net/bridge/br_netlink.c | 10 +++++++---
net/bridge/br_switchdev.c | 5 ++---
net/bridge/br_sysfs_if.c | 22 ++++++++++++++--------
3 files changed, 23 insertions(+), 14 deletions(-)
@@ -79,9 +79,8 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,attr.u.brport_flags.val=flags&mask;attr.u.brport_flags.mask=mask;-/* We run from atomic context here */-err=call_switchdev_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,-&info.info,extack);+err=call_switchdev_blocking_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,+&info.info,extack);err=notifier_to_errno(err);if(err==-EOPNOTSUPP)return0;
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:29:00
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Because the bridge will start offloading SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS
while not serialized by any lock such as the br->lock spinlock, existing
drivers that treat that attribute and cache the brport flags might no
longer work correctly.
The issue is that the brport flags are a single unsigned long bitmask,
and the bridge only guarantees the validity of the changed bits, not the
full state. So when offloading two concurrent switchdev attributes, such
as one for BR_LEARNING and another for BR_FLOOD, it might happen that
the flags having BR_FLOOD are written into the cached value, and this in
turn disables the BR_LEARNING bit which was set previously.
We can fix this across the board by keeping individual boolean variables
for each brport flag. Note that mlxsw and prestera were setting the
BR_LEARNING_SYNC flag too, but that appears to be just dead code, so I
removed it.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
Patch is new.
.../marvell/prestera/prestera_switchdev.c | 32 ++++++++++------
.../mellanox/mlxsw/spectrum_switchdev.c | 38 ++++++++++++-------
drivers/net/ethernet/rocker/rocker.h | 2 +-
drivers/net/ethernet/rocker/rocker_main.c | 2 +-
drivers/net/ethernet/rocker/rocker_ofdpa.c | 26 +++++++------
net/bridge/br_switchdev.c | 3 +-
6 files changed, 62 insertions(+), 41 deletions(-)
@@ -72,12 +72,11 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,};interr;-flags&=BR_PORT_FLAGS_HW_OFFLOAD;mask&=BR_PORT_FLAGS_HW_OFFLOAD;if(!mask)return0;-attr.u.brport_flags.val=flags;+attr.u.brport_flags.val=flags&mask;attr.u.brport_flags.mask=mask;/* We run from atomic context here */
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:29:09
From: Vladimir Oltean <vladimir.oltean@nxp.com>
We should not be unconditionally enabling address learning, since doing
that is actively detrimential when a port is standalone and not offloading
a bridge. Namely, if a port in the switch is standalone and others are
offloading the bridge, then we could enter a situation where we learn an
address towards the standalone port, but the bridged ports could not
forward the packet there, because the CPU is the only path between the
standalone and the bridged ports. The solution of course is to not
enable address learning unless the bridge asks for it.
We need to set up the initial port flags for no learning and flooding
everything, then the bridge takes over. The flood configuration was
already configured ok in ocelot_init, we just need to disable learning
in ocelot_init_port.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
---
Changes in v3:
None.
Changes in v2:
- Disable learning in ocelot_init_port.
- Keep a single bool ocelot_port->learn_ena instead of
ocelot_port->brport_flags.
- Stop touching the brport_flags from ocelot_port_bridge_leave (which
was a leftover).
drivers/net/dsa/ocelot/felix.c | 10 +++++
drivers/net/ethernet/mscc/ocelot.c | 59 +++++++++++++++++++++++++-
drivers/net/ethernet/mscc/ocelot_net.c | 4 ++
include/soc/mscc/ocelot.h | 3 ++
4 files changed, 75 insertions(+), 1 deletion(-)
@@ -1480,6 +1482,57 @@ int ocelot_get_max_mtu(struct ocelot *ocelot, int port)}EXPORT_SYMBOL(ocelot_get_max_mtu);+intocelot_port_bridge_flags(structocelot*ocelot,intport,+structswitchdev_brport_flagsflags)+{+structocelot_port*ocelot_port=ocelot->ports[port];++if(flags.mask&~(BR_LEARNING|BR_FLOOD|BR_MCAST_FLOOD|+BR_BCAST_FLOOD))+return-EINVAL;++if(flags.mask&BR_LEARNING){+u32val=0;++ocelot_port->learn_ena=!!(flags.val&BR_LEARNING);+if(ocelot_port->learn_ena)+val=ANA_PORT_PORT_CFG_LEARN_ENA;++ocelot_rmw_gix(ocelot,val,ANA_PORT_PORT_CFG_LEARN_ENA,+ANA_PORT_PORT_CFG,port);+}++if(flags.mask&BR_FLOOD){+u32val=0;++if(flags.val&BR_FLOOD)+val=BIT(port);++ocelot_rmw_rix(ocelot,val,BIT(port),ANA_PGID_PGID,PGID_UC);+}++if(flags.mask&BR_MCAST_FLOOD){+u32val=0;++if(flags.val&BR_MCAST_FLOOD)+val=BIT(port);++ocelot_rmw_rix(ocelot,val,BIT(port),ANA_PGID_PGID,PGID_MC);+}++if(flags.mask&BR_BCAST_FLOOD){+u32val=0;++if(flags.val&BR_BCAST_FLOOD)+val=BIT(port);++ocelot_rmw_rix(ocelot,val,BIT(port),ANA_PGID_PGID,PGID_BC);+}++return0;+}+EXPORT_SYMBOL(ocelot_port_bridge_flags);+voidocelot_init_port(structocelot*ocelot,intport){structocelot_port*ocelot_port=ocelot->ports[port];
@@ -1524,6 +1577,10 @@ void ocelot_init_port(struct ocelot *ocelot, int port)ANA_PORT_DROP_CFG_DROP_MC_SMAC_ENA,ANA_PORT_DROP_CFG,port);+/* Disable source address learning for standalone mode */+ocelot_rmw_gix(ocelot,0,ANA_PORT_PORT_CFG_LEARN_ENA,+ANA_PORT_PORT_CFG,port);+/* Set default VLAN and tag type to 8021Q. */ocelot_rmw_gix(ocelot,REW_PORT_VLAN_CFG_PORT_TPID(ETH_P_8021Q),REW_PORT_VLAN_CFG_PORT_TPID_M,
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:29:09
From: Vladimir Oltean <vladimir.oltean@nxp.com>
In preparation of offloading the bridge port flags which have
independent settings for unknown multicast and for broadcast, we should
also start reserving one destination Port Group ID for the flooding of
broadcast packets, to allow configuring it individually.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
None.
Changes in v2:
None.
drivers/net/ethernet/mscc/ocelot.c | 13 ++++++++-----
include/soc/mscc/ocelot.h | 15 ++++++++-------
2 files changed, 16 insertions(+), 12 deletions(-)
@@ -1683,15 +1683,18 @@ int ocelot_init(struct ocelot *ocelot)ocelot_write_rix(ocelot,0,ANA_PGID_PGID,PGID_SRC+port);}-/* Allow broadcast MAC frames. */for_each_nonreserved_multicast_dest_pgid(ocelot,i){u32val=ANA_PGID_PGID_PGID(GENMASK(ocelot->num_phys_ports-1,0));ocelot_write_rix(ocelot,val,ANA_PGID_PGID,i);}-ocelot_write_rix(ocelot,-ANA_PGID_PGID_PGID(GENMASK(ocelot->num_phys_ports,0)),-ANA_PGID_PGID,PGID_MC);+/* Allow broadcast and unknown L2 multicast to the CPU. */+ocelot_rmw_rix(ocelot,ANA_PGID_PGID_PGID(BIT(ocelot->num_phys_ports)),+ANA_PGID_PGID_PGID(BIT(ocelot->num_phys_ports)),+ANA_PGID_PGID,PGID_MC);+ocelot_rmw_rix(ocelot,ANA_PGID_PGID_PGID(BIT(ocelot->num_phys_ports)),+ANA_PGID_PGID_PGID(BIT(ocelot->num_phys_ports)),+ANA_PGID_PGID,PGID_BC);ocelot_write_rix(ocelot,0,ANA_PGID_PGID,PGID_MCIPV4);ocelot_write_rix(ocelot,0,ANA_PGID_PGID,PGID_MCIPV6);
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 09:29:09
From: Vladimir Oltean <vladimir.oltean@nxp.com>
The chip can configure unicast flooding, broadcast flooding and learning.
Learning is per port, while flooding is per {ingress, egress} port pair
and we need to configure the same value for all possible ingress ports
towards the requested one.
While multicast flooding is not officially supported, we can hack it by
using a feature of the second generation (P/Q/R/S) devices, which is that
FDB entries are maskable, and multicast addresses always have an odd
first octet. So by putting a match-all for 00:01:00:00:00:00 addr and
00:01:00:00:00:00 mask at the end of the FDB, we make sure that it is
always checked last, and does not take precedence in front of any other
MDB. So it behaves effectively as an unknown multicast entry.
For the first generation switches, this feature is not available, so
unknown multicast will always be treated the same as unknown unicast.
So the only thing we can do is request the user to offload the settings
for these 2 flags in tandem, i.e.
ip link set swp2 type bridge_slave flood off
Error: sja1105: This chip cannot configure multicast flooding independently of unicast.
ip link set swp2 type bridge_slave flood off mcast_flood off
ip link set swp2 type bridge_slave mcast_flood on
Error: sja1105: This chip cannot configure multicast flooding independently of unicast.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
None.
Changes in v2:
Patch is new.
drivers/net/dsa/sja1105/sja1105.h | 2 +
drivers/net/dsa/sja1105/sja1105_main.c | 212 +++++++++++++++++++++++--
drivers/net/dsa/sja1105/sja1105_spi.c | 6 +
3 files changed, 209 insertions(+), 11 deletions(-)
@@ -42,15 +44,10 @@ static voidsja1105_port_allow_traffic(structsja1105_l2_forwarding_entry*l2_fwd,intfrom,intto,boolallow){-if(allow){-l2_fwd[from].bc_domain|=BIT(to);+if(allow)l2_fwd[from].reach_port|=BIT(to);-l2_fwd[from].fl_domain|=BIT(to);-}else{-l2_fwd[from].bc_domain&=~BIT(to);+elsel2_fwd[from].reach_port&=~BIT(to);-l2_fwd[from].fl_domain&=~BIT(to);-}}/* Structure used to temporarily transport device tree
@@ -220,17 +217,43 @@ static int sja1105_init_mii_settings(struct sja1105_private *priv,staticintsja1105_init_static_fdb(structsja1105_private*priv){+structsja1105_l2_lookup_entry*l2_lookup;structsja1105_table*table;+intport;table=&priv->static_config.tables[BLK_IDX_L2_LOOKUP];-/* We only populate the FDB table through dynamic-*L2AddressLookupentries+/* We only populate the FDB table through dynamic L2 Address Lookup+*entries,exceptforaspecialentryattheendwhichisacatch-all+*forunknownmulticastandwillbeusedtocontrolfloodingdomain.*/if(table->entry_count){kfree(table->entries);table->entry_count=0;}++if(!priv->info->can_limit_mcast_flood)+return0;++table->entries=kcalloc(1,table->ops->unpacked_entry_size,+GFP_KERNEL);+if(!table->entries)+return-ENOMEM;++table->entry_count=1;+l2_lookup=table->entries;++/* All L2 multicast addresses have an odd first octet */+l2_lookup[0].macaddr=SJA1105_UNKNOWN_MULTICAST;+l2_lookup[0].mask_macaddr=SJA1105_UNKNOWN_MULTICAST;+l2_lookup[0].lockeds=true;+l2_lookup[0].index=SJA1105_MAX_L2_LOOKUP_COUNT-1;++/* Flood multicast to every port by default */+for(port=0;port<priv->ds->num_ports;port++)+if(!dsa_is_unused_port(priv->ds,port))+l2_lookup[0].destports|=BIT(port);+return0;}
@@ -390,6 +413,12 @@ static int sja1105_init_l2_forwarding(struct sja1105_private *priv)sja1105_port_allow_traffic(l2fwd,i,upstream,true);sja1105_port_allow_traffic(l2fwd,upstream,i,true);++l2fwd[i].bc_domain=BIT(upstream);+l2fwd[i].fl_domain=BIT(upstream);++l2fwd[upstream].bc_domain|=BIT(i);+l2fwd[upstream].fl_domain|=BIT(i);}/* Next 8 entries define VLAN PCP mapping from ingress to egress.*Createaone-to-onemapping.
@@ -1514,6 +1543,12 @@ static int sja1105_fdb_dump(struct dsa_switch *ds, int port,*/if(!(l2_lookup.destports&BIT(port)))continue;++/* We need to hide the FDB entry for unknown multicast */+if(l2_lookup.macaddr==SJA1105_UNKNOWN_MULTICAST&&+l2_lookup.mask_macaddr==SJA1105_UNKNOWN_MULTICAST)+continue;+u64_to_ether_addr(l2_lookup.macaddr,macaddr);/* We need to hide the dsa_8021q VLANs from the user. */
@@ -3239,6 +3274,160 @@ static void sja1105_port_policer_del(struct dsa_switch *ds, int port)sja1105_static_config_reload(priv,SJA1105_BEST_EFFORT_POLICING);}+staticintsja1105_port_set_learning(structsja1105_private*priv,intport,+boolenabled)+{+structsja1105_mac_config_entry*mac;+intrc;++mac=priv->static_config.tables[BLK_IDX_MAC_CONFIG].entries;++mac[port].dyn_learn=!!(priv->learn_ena&BIT(port));++rc=sja1105_dynamic_config_write(priv,BLK_IDX_MAC_CONFIG,port,+&mac[port],true);+if(rc)+returnrc;++if(enabled)+priv->learn_ena|=BIT(port);+else+priv->learn_ena&=~BIT(port);++return0;+}++/* Common function for unicast and broadcast flood configuration.+*Floodingisconfiguredbetweeneach{ingress,egress}portpair,andsince+*thebridge'ssemanticsarethoseof"egress flooding",itmeanswemust+*enablefloodingtowardsthisportfromallingressportsthatareinthe+*samebridge.Inpractice,wejustenablefloodingfromallpossibleingress+*portsregardlessofwhetherthey'reinthesamebridgeornot,sincethe+*reach_portconfigurationwillnotallowfloodedframestoleakacross+*bridgingdomainsanyway.+*/+staticintsja1105_port_ucast_bcast_flood(structsja1105_private*priv,intto,+structswitchdev_brport_flagsflags)+{+structsja1105_l2_forwarding_entry*l2_fwd;+intfrom,rc;++l2_fwd=priv->static_config.tables[BLK_IDX_L2_FORWARDING].entries;++for(from=0;from<priv->ds->num_ports;from++){+if(dsa_is_unused_port(priv->ds,from))+continue;+if(from==to)+continue;++/* Unicast */+if(flags.mask&BR_FLOOD){+if(flags.val&BR_FLOOD)+l2_fwd[from].fl_domain|=BIT(to);+else+l2_fwd[from].fl_domain&=~BIT(to);+}+/* Broadcast */+if(flags.mask&BR_BCAST_FLOOD){+if(flags.val&BR_BCAST_FLOOD)+l2_fwd[from].bc_domain|=BIT(to);+else+l2_fwd[from].bc_domain&=~BIT(to);+}++rc=sja1105_dynamic_config_write(priv,BLK_IDX_L2_FORWARDING,+from,&l2_fwd[from],true);+if(rc<0)+returnrc;+}++return0;+}++staticintsja1105_port_mcast_flood(structsja1105_private*priv,intto,+structswitchdev_brport_flagsflags,+structnetlink_ext_ack*extack)+{+structsja1105_l2_lookup_entry*l2_lookup;+structsja1105_table*table;+intmatch;++table=&priv->static_config.tables[BLK_IDX_L2_LOOKUP];+l2_lookup=table->entries;++for(match=0;match<table->entry_count;match++)+if(l2_lookup[match].macaddr==SJA1105_UNKNOWN_MULTICAST&&+l2_lookup[match].mask_macaddr==SJA1105_UNKNOWN_MULTICAST)+break;++if(match==table->entry_count){+NL_SET_ERR_MSG_MOD(extack,+"Could not find FDB entry for unknown multicast");+return-ENOSPC;+}++if(flags.val&BR_MCAST_FLOOD)+l2_lookup[match].destports|=BIT(to);+else+l2_lookup[match].destports&=~BIT(to);++returnsja1105_dynamic_config_write(priv,BLK_IDX_L2_LOOKUP,+l2_lookup[match].index,+&l2_lookup[match],+true);+}++staticintsja1105_port_bridge_flags(structdsa_switch*ds,intport,+structswitchdev_brport_flagsflags,+structnetlink_ext_ack*extack)+{+structsja1105_private*priv=ds->priv;+intrc;++if(flags.mask&~(BR_LEARNING|BR_FLOOD|BR_MCAST_FLOOD|+BR_BCAST_FLOOD))+return-EINVAL;++if(flags.mask&BR_LEARNING){+boollearn_ena=!!(flags.val&BR_LEARNING);++rc=sja1105_port_set_learning(priv,port,learn_ena);+if(rc)+returnrc;+}++if(flags.mask&(BR_FLOOD|BR_BCAST_FLOOD)){+rc=sja1105_port_ucast_bcast_flood(priv,port,flags);+if(rc)+returnrc;+}++if(flags.mask&(BR_FLOOD|BR_MCAST_FLOOD)&&+!priv->info->can_limit_mcast_flood){+boolmulticast=!!(flags.val&BR_MCAST_FLOOD);+boolunicast=!!(flags.val&BR_FLOOD);++if(unicast!=multicast){+NL_SET_ERR_MSG_MOD(extack,+"This chip cannot configure multicast flooding independently of unicast");+return-EINVAL;+}+}++/* For chips that can't offload BR_MCAST_FLOOD independently, there+*isnothingtodohere,weensuredtheconfigurationisinsyncby+*offloadingBR_FLOOD.+*/+if(flags.mask&BR_MCAST_FLOOD&&priv->info->can_limit_mcast_flood){+rc=sja1105_port_mcast_flood(priv,port,flags,+extack);+if(rc)+returnrc;+}++return0;+}+staticconststructdsa_switch_opssja1105_switch_ops={.get_tag_protocol=sja1105_get_tag_protocol,.setup=sja1105_setup,
On Wed, Feb 10, 2021 at 11:14:41AM +0200, Vladimir Oltean wrote:
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Because the bridge will start offloading SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS
while not serialized by any lock such as the br->lock spinlock, existing
drivers that treat that attribute and cache the brport flags might no
longer work correctly.
Can you explain the race? This notification is sent from sysfs/netlink
call path, both of which take rtnl.
From: Nikolay Aleksandrov <hidden> Date: 2021-02-10 10:28:23
On 10/02/2021 11:14, Vladimir Oltean wrote:
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Since we would like br_switchdev_set_port_flag to not use an atomic
notifier, it should be called from outside spinlock context.
We can temporarily drop br->lock, but that creates some concurrency
complications (example below is given for sysfs):
- There might be an "echo 1 > multicast_flood" simultaneous with an
"echo 0 > multicast_flood". The result of this is nondeterministic
either way, so I'm not too concerned as long as the result is
consistent (no other flags have changed).
- There might be an "echo 1 > multicast_flood" simultaneous with an
"echo 0 > learning". My expectation is that none of the two writes are
"eaten", and the final flags contain BR_MCAST_FLOOD=1 and BR_LEARNING=0
regardless of the order of execution. That is actually possible if, on
the commit path, we don't do a trivial "p->flags = flags" which might
overwrite bits outside of our mask, but instead we just change the
flags corresponding to our mask.
Not sure I follow here, how do we get any concurrency issues with sysfs or netlink
when both take rtnl before doing any changes ?
quoted hunk
Now that br_switchdev_set_port_flag is never called from under br->lock,
it runs in sleepable context.
All switchdev drivers handle SWITCHDEV_PORT_ATTR_SET as both blocking
and atomic, so no changes are needed on that front.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
Changes in v3:
- Drop the br->lock around br_switchdev_set_port_flag in this patch, for
both sysfs and netlink.
- Only set/restore the masked bits in p->flags to avoid concurrency
issues.
Changes in v2:
Patch is new.
net/bridge/br_netlink.c | 10 +++++++---
net/bridge/br_switchdev.c | 5 ++---
net/bridge/br_sysfs_if.c | 22 ++++++++++++++--------
3 files changed, 23 insertions(+), 14 deletions(-)
@@ -79,9 +79,8 @@ int br_switchdev_set_port_flag(struct net_bridge_port *p,attr.u.brport_flags.val=flags&mask;attr.u.brport_flags.mask=mask;-/* We run from atomic context here */-err=call_switchdev_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,-&info.info,extack);+err=call_switchdev_blocking_notifiers(SWITCHDEV_PORT_ATTR_SET,p->dev,+&info.info,extack);err=notifier_to_errno(err);if(err==-EOPNOTSUPP)return0;
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 10:36:12
On Wed, Feb 10, 2021 at 12:12:57PM +0200, Ido Schimmel wrote:
On Wed, Feb 10, 2021 at 11:14:41AM +0200, Vladimir Oltean wrote:
quoted
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Because the bridge will start offloading SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS
while not serialized by any lock such as the br->lock spinlock, existing
drivers that treat that attribute and cache the brport flags might no
longer work correctly.
Can you explain the race? This notification is sent from sysfs/netlink
call path, both of which take rtnl.
Replying here to both you and Nikolay: there isn't any race, sorry, I
missed the fact that brport_store takes the rtnl_mutex and by extension
I thought that RTM_SETLINK runs unlocked too, without really checking.
Well, at least that's good news, the implementation can be a lot more
straightforward then...
From: Nikolay Aleksandrov <hidden> Date: 2021-02-10 10:38:35
On 10/02/2021 11:14, Vladimir Oltean wrote:
From: Vladimir Oltean <vladimir.oltean@nxp.com>
The initial goal of this series was to have better support for
standalone ports mode and multiple bridges on the DSA drivers like
ocelot/felix and sja1105. Proper support for standalone mode requires
disabling address learning, which in turn requires interaction with the
switchdev notifier, which is actually where most of the patches are.
I also noticed that most of the drivers are actually talking either to
firmware or SPI/MDIO connected devices from the brport flags switchdev
attribute handler, so it makes sense to actually make it sleepable
instead of atomic.
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
Thanks,
Nik
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 10:49:54
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
From: Nikolay Aleksandrov <hidden> Date: 2021-02-10 10:58:26
On 10/02/2021 12:45, Vladimir Oltean wrote:
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 11:04:38
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
From: Nikolay Aleksandrov <hidden> Date: 2021-02-10 11:13:10
On 10/02/2021 13:01, Vladimir Oltean wrote:
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 12:04:30
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
From: Nikolay Aleksandrov <hidden> Date: 2021-02-10 12:16:54
On 10/02/2021 14:01, Vladimir Oltean wrote:
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
That's a bug that's gone unnoticed, surely not an argument to make error-prone changes.
It's not because of ugliness, rather for easier reasoning when people want to work with
that code, easier to maintain and later easier to verify when the lock gets removed.
We'll reduce the chance for new bugs by having code that can be understood easier,
especially for locking it's never a good idea to play games, we must try to avoid it
when we can.
On Wed, Feb 10, 2021 at 02:01:06PM +0200, Vladimir Oltean wrote:
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
Can you please point to the bug? I'm not following
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 12:33:55
On Wed, Feb 10, 2021 at 02:21:05PM +0200, Ido Schimmel wrote:
On Wed, Feb 10, 2021 at 02:01:06PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
Can you please point to the bug? I'm not following
For example, mlxsw eventually calls mlxsw_sp_fid_flood_set from the
SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS handling data path, and this
function allocates memory with GFP_KERNEL.
Another example is prestera which eventually calls prestera_fw_send_req
which takes a mutex_lock.
Yet another example are mv88e6xxx and b53 which use MDIO and SPI
from their .port_egress_floods implementation, buses which have
might_sleep() in them.
On Wed, Feb 10, 2021 at 02:29:36PM +0200, Vladimir Oltean wrote:
On Wed, Feb 10, 2021 at 02:21:05PM +0200, Ido Schimmel wrote:
quoted
On Wed, Feb 10, 2021 at 02:01:06PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
Can you please point to the bug? I'm not following
For example, mlxsw eventually calls mlxsw_sp_fid_flood_set from the
SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS handling data path, and this
function allocates memory with GFP_KERNEL.
Another example is prestera which eventually calls prestera_fw_send_req
which takes a mutex_lock.
Yet another example are mv88e6xxx and b53 which use MDIO and SPI
from their .port_egress_floods implementation, buses which have
might_sleep() in them.
And check how SWITCHDEV_F_DEFER is used.
We can squash SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS and
SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS into one blocking notification
by reducing the scope of the bridge lock like Nik suggested. Currently
it's just blindly taken around br_setport().
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-02-10 12:56:38
On Wed, Feb 10, 2021 at 02:38:23PM +0200, Ido Schimmel wrote:
On Wed, Feb 10, 2021 at 02:29:36PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 02:21:05PM +0200, Ido Schimmel wrote:
quoted
On Wed, Feb 10, 2021 at 02:01:06PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
Can you please point to the bug? I'm not following
For example, mlxsw eventually calls mlxsw_sp_fid_flood_set from the
SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS handling data path, and this
function allocates memory with GFP_KERNEL.
Another example is prestera which eventually calls prestera_fw_send_req
which takes a mutex_lock.
Yet another example are mv88e6xxx and b53 which use MDIO and SPI
from their .port_egress_floods implementation, buses which have
might_sleep() in them.
And check how SWITCHDEV_F_DEFER is used.
We can squash SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS and
SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS into one blocking notification
by reducing the scope of the bridge lock like Nik suggested. Currently
it's just blindly taken around br_setport().
Okay, so the deferred attr_set propagates just a possible ENOMEM from
the deferred work enqueue, not the actual failure if that occurred.
I can leave alone the piece that sends two notifications for now, but I
would still need to deliver the full struct switchdev_brport_flags with
both the flags and the mask to both the PRE_BRIDGE_FLAGS and the
BRIDGE_FLAGS, because I need to deliver an extack from the sja1105 driver
that BR_FLOOD should always have the same value as BR_MCAST_FLOOD.
On Wed, Feb 10, 2021 at 02:55:01PM +0200, Vladimir Oltean wrote:
On Wed, Feb 10, 2021 at 02:38:23PM +0200, Ido Schimmel wrote:
quoted
On Wed, Feb 10, 2021 at 02:29:36PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 02:21:05PM +0200, Ido Schimmel wrote:
quoted
On Wed, Feb 10, 2021 at 02:01:06PM +0200, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 01:05:57PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 13:01, Vladimir Oltean wrote:
quoted
On Wed, Feb 10, 2021 at 12:52:33PM +0200, Nikolay Aleksandrov wrote:
quoted
On 10/02/2021 12:45, Vladimir Oltean wrote:
quoted
Hi Nikolay,
On Wed, Feb 10, 2021 at 12:31:43PM +0200, Nikolay Aleksandrov wrote:
quoted
Hi Vladimir,
Let's take a step back for a moment and discuss the bridge unlock/lock sequences
that come with this set. I'd really like to avoid those as they're a recipe
for future problems. The only good way to achieve that currently is to keep
the PRE_FLAGS call and do that in unsleepable context but move the FLAGS call
after the flags have been changed (if they have changed obviously). That would
make the code read much easier since we'll have all our lock/unlock sequences
in the same code blocks and won't play games to get sleepable context.
Please let's think and work in that direction, rather than having:
+ spin_lock_bh(&p->br->lock);
+ if (err) {
+ netdev_err(p->dev, "%s\n", extack._msg);
+ return err;
}
+
which immediately looks like a bug even though after some code checking we can
verify it's ok. WDYT?
I plan to get rid of most of the br->lock since it's been abused for a very long
time because it's essentially STP lock, but people have started using it for other
things and I plan to fix that when I get more time.
This won't make the sysfs codepath any nicer, will it?
Currently we'll have to live with a hack that checks if the flags have changed. I agree
it won't be pretty, but we won't have to unlock and lock again in the middle of the
called function and we'll have all our locking in the same place, easier to verify and
later easier to remove. Once I get rid of most of the br->lock usage we can revisit
the drop of PRE_FLAGS if it's a problem. The alternative is to change the flags, then
send the switchdev notification outside of the lock and revert the flags if it doesn't
go through which doesn't sound much better.
I'm open to any other suggestions, but definitely would like to avoid playing locking games.
Even if it means casing out flag setting from all other store_ functions for sysfs.
Yes, this can work but will need a bit more changes because of br_port_flags_change().
Then the netlink side can be modeled in a similar way.
What I just don't understand is how others can get away with doing
sleepable work in atomic context but I can't make the notifier blocking
by dropping a spinlock which isn't needed there, because it looks ugly :D
Can you please point to the bug? I'm not following
For example, mlxsw eventually calls mlxsw_sp_fid_flood_set from the
SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS handling data path, and this
function allocates memory with GFP_KERNEL.
Another example is prestera which eventually calls prestera_fw_send_req
which takes a mutex_lock.
Yet another example are mv88e6xxx and b53 which use MDIO and SPI
from their .port_egress_floods implementation, buses which have
might_sleep() in them.
And check how SWITCHDEV_F_DEFER is used.
We can squash SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS and
SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS into one blocking notification
by reducing the scope of the bridge lock like Nik suggested. Currently
it's just blindly taken around br_setport().
Okay, so the deferred attr_set propagates just a possible ENOMEM from
the deferred work enqueue, not the actual failure if that occurred.
I can leave alone the piece that sends two notifications for now, but I
would still need to deliver the full struct switchdev_brport_flags with
both the flags and the mask to both the PRE_BRIDGE_FLAGS and the
BRIDGE_FLAGS, because I need to deliver an extack from the sja1105 driver
that BR_FLOOD should always have the same value as BR_MCAST_FLOOD.
From: David Miller <davem@davemloft.net> Date: 2021-02-10 23:35:13
From: Vladimir Oltean <olteanv@gmail.com>
Date: Wed, 10 Feb 2021 11:14:41 +0200
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Because the bridge will start offloading SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS
while not serialized by any lock such as the br->lock spinlock, existing
drivers that treat that attribute and cache the brport flags might no
longer work correctly.
The issue is that the brport flags are a single unsigned long bitmask,
and the bridge only guarantees the validity of the changed bits, not the
full state. So when offloading two concurrent switchdev attributes, such
as one for BR_LEARNING and another for BR_FLOOD, it might happen that
the flags having BR_FLOOD are written into the cached value, and this in
turn disables the BR_LEARNING bit which was set previously.
We can fix this across the board by keeping individual boolean variables
for each brport flag. Note that mlxsw and prestera were setting the
BR_LEARNING_SYNC flag too, but that appears to be just dead code, so I
removed it.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
This needs updating because, as discussed, there is no race.
From: Vladimir Oltean <vladimir.oltean@nxp.com>
When a struct switchdev_attr is notified through switchdev, there is no
way to report informational messages, unlike for struct switchdev_obj.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
From: Vladimir Oltean <vladimir.oltean@nxp.com>
For a DSA switch port operating in standalone mode, address learning
doesn't make much sense since that is a bridge function. In fact,
address learning even breaks setups such as this one:
+---------------------------------------------+
| |
| +-------------------+ |
| | br0 | send receive |
| +--------+-+--------+ +--------+ +--------+ |
| | | | | | | | | |
| | swp0 | | swp1 | | swp2 | | swp3 | |
| | | | | | | | | |
+-+--------+-+--------+-+--------+-+--------+-+
| ^ | ^
| | | |
| +-----------+ |
| |
+--------------------------------+
because if the switch has a single FDB (can offload a single bridge)
then source address learning on swp3 can "steal" the source MAC address
of swp2 from br0's FDB, because learning frames coming from swp2 will be
done twice: first on the swp1 ingress port, second on the swp3 ingress
port. So the hardware FDB will become out of sync with the software
bridge, and when swp2 tries to send one more packet towards swp1, the
ASIC will attempt to short-circuit the forwarding path and send it
directly to swp3 (since that's the last port it learned that address on),
which it obviously can't, because swp3 operates in standalone mode.
So DSA drivers operating in standalone mode should still configure a
list of bridge port flags even when they are standalone. Currently DSA
attempts to call dsa_port_bridge_flags with 0, which disables egress
flooding of unknown unicast and multicast, something which doesn't make
much sense. For the switches that implement .port_egress_floods - b53
and mv88e6xxx, it probably doesn't matter too much either, since they
can possibly inject traffic from the CPU into a standalone port,
regardless of MAC DA, even if egress flooding is turned off for that
port, but certainly not all DSA switches can do that - sja1105, for
example, can't. So it makes sense to use a better common default there,
such as "flood everything".
It should also be noted that what DSA calls "dsa_port_bridge_flags()"
is a degenerate name for just calling .port_egress_floods(), since
nothing else is implemented - not learning, in particular. But disabling
address learning, something that this driver is also coding up for, will
be supported by individual drivers once .port_egress_floods is replaced
with a more generic .port_bridge_flags.
Previous attempts to code up this logic have been in the common bridge
layer, but as pointed out by Ido Schimmel, there are corner cases that
are missed when doing that:
https://patchwork.kernel.org/project/netdevbpf/patch/20210209151936.97382-5-olteanv@gmail.com/
So, at least for now, let's leave DSA in charge of setting port flags
before and after the bridge join and leave.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
From: Vladimir Oltean <vladimir.oltean@nxp.com>
The bridge offloads the port flags through a single bit mask using
switchdev, which among others, contains learning and flooding settings.
The commit 57652796aa97 ("net: dsa: add support for bridge flags")
missed one crucial aspect of the SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS API
when designing the API one level lower, towards the drivers.
This is that the bitmask of passed brport flags never has more than one
bit set at a time. On the other hand, the prototype passed to the driver
is .port_egress_floods(int port, bool unicast, bool multicast), which
configures two flags at a time.
DSA currently checks if .port_egress_floods is implemented, and if it
is, reports both BR_FLOOD and BR_MCAST_FLOOD as supported. So the driver
has no choice if it wants to inform the bridge that, for example, it
can't configure unicast flooding independently of multicast flooding -
the DSA mid layer is standing in the way. Or the other way around: a new
driver wants to start configuring BR_BCAST_FLOOD separately, but what do
we do with the rest, which only support unicast and multicast flooding?
Do we report broadcast flooding configuration as supported for those
too, and silently do nothing?
Secondly, currently DSA deems the driver too dumb to deserve knowing that
a SWITCHDEV_ATTR_ID_BRIDGE_MROUTER attribute was offloaded, because it
just calls .port_egress_floods for the CPU port. When we'll add support
for the plain SWITCHDEV_ATTR_ID_PORT_MROUTER, that will become a real
problem because the flood settings will need to be held statefully in
the DSA middle layer, otherwise changing the mrouter port attribute will
impact the flooding attribute. And that's _assuming_ that the underlying
hardware doesn't have anything else to do when a multicast router
attaches to a port than flood unknown traffic to it. If it does, there
will need to be a dedicated .port_set_mrouter anyway.
Lastly, we have DSA drivers that have a backlink into a pure switchdev
driver (felix -> ocelot). It seems reasonable that the other switchdev
drivers should not have to suffer from the oddities of DSA overengineering,
so keeping DSA a pass-through layer makes more sense there.
To simplify the brport flags situation we just delete .port_egress_floods
and we introduce a simple .port_bridge_flags which is passed to the
driver. Also, the logic from dsa_port_mrouter is removed and a
.port_set_mrouter is created.
Functionally speaking, we simply move the calls to .port_egress_floods
one step lower, in the two drivers that implement it: mv88e6xxx and b53,
so things should work just as before.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
From: Vladimir Oltean <vladimir.oltean@nxp.com>
In preparation of offloading the bridge port flags which have
independent settings for unknown multicast and for broadcast, we should
also start reserving one destination Port Group ID for the flooding of
broadcast packets, to allow configuring it individually.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
From: Vladimir Oltean <vladimir.oltean@nxp.com>
We should not be unconditionally enabling address learning, since doing
that is actively detrimential when a port is standalone and not offloading
a bridge. Namely, if a port in the switch is standalone and others are
offloading the bridge, then we could enter a situation where we learn an
address towards the standalone port, but the bridged ports could not
forward the packet there, because the CPU is the only path between the
standalone and the bridged ports. The solution of course is to not
enable address learning unless the bridge asks for it.
We need to set up the initial port flags for no learning and flooding
everything, then the bridge takes over. The flood configuration was
already configured ok in ocelot_init, we just need to disable learning
in ocelot_init_port.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Alexandre Belloni <alexandre.belloni@bootlin.com>