From: Russell King - ARM Linux <hidden> Date: 2015-09-24 19:18:17
Hi,
The third version of this series fixes the build error which David
identified, and drops the broken changes for the Cavium Thunger BGX
ethernet driver as this driver requires some complex changes to
resolve the leakage - and this is best done by people who can test
the driver.
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"
I _think_ I've been able to build-test all the drivers touched by
that patch to some degree now, though several of them needed the
Kconfig hacked to allow it (not all had || COMPILE_TEST clause on
their dependencies.)
Previous cover letters below:
This is the second version of the series, with the comments David had
on the first patch fixed up. Original series description with updated
diffstat below.
While looking at the DSA code, I noticed we have a
of_find_net_device_by_node(), and it looks like users of that are
similarly buggy - it looks like net/dsa/dsa.c is the only user. Fix
that too.
Hi,
While looking at the phy code, I identified a number of weaknesses
where refcounting on device structures was being leaked, where
modules could be removed while in-use, and where the fixed-phy could
end up having unintended consequences caused by incorrect calls to
fixed_phy_update_state().
This patch series resolves those issues, some of which were discovered
with testing on an Armada 388 board. Not all patches are fully tested,
particularly the one which touches several network drivers.
When resolving the struct device refcounting problems, several different
solutions were considered before settling on the implementation here -
one of the considerations was to avoid touching many network drivers.
The solution here is:
phy_attach*() - takes a refcount
phy_detach*() - drops the phy_attach refcount
Provided drivers always attach and detach their phys, which they should
already be doing, this should change nothing, even if they leak a refcount.
of_phy_find_device() and of_* functions which use that take
a refcount. Arrange for this refcount to be dropped once
the phy is attached.
This is the reason why the previous change is important - we can't drop
this refcount taken by of_phy_find_device() until something else holds
a reference on the device. This resolves the leaked refcount caused by
using of_phy_connect() or of_phy_attach().
Even without the above changes, these drivers are leaking by calling
of_phy_find_device(). These drivers are addressed by adding the
appropriate release of that refcount.
The mdiobus code also suffered from the same kind of leak, but thankfully
this only happened in one place - the mdio-mux code.
I also found that the try_module_get() in the phy layer code was utterly
useless: phydev->dev.driver was guaranteed to always be NULL, so
try_module_get() was always being called with a NULL argument. I proved
this with my SFP code, which declares its own MDIO bus - the module use
count was never incremented irrespective of how I set the MDIO bus up.
This allowed the MDIO bus code to be removed from the kernel while there
were still PHYs attached to it.
One other bug was discovered: while using in-band-status with mvneta, it
was found that if a real phy is attached with in-band-status enabled,
and another ethernet interface is using the fixed-phy infrastructure, the
interface using the fixed-phy infrastructure is configured according to
the other interface using the in-band-status - which is caused by the
fixed-phy code not verifying that the phy_device passed in is actually
a fixed-phy device, rather than a real MDIO phy.
Lastly, having mdio_bus reversing phy_device_register() internals seems
like a layering violation - it's trivial to move that code to the phy
device layer.
drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++----
drivers/net/ethernet/freescale/gianfar.c | 6 ++-
drivers/net/ethernet/freescale/ucc_geth.c | 8 +++-
drivers/net/ethernet/marvell/mvneta.c | 2 +
drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 +
drivers/net/phy/fixed_phy.c | 2 +-
drivers/net/phy/mdio-mux.c | 19 +++++---
drivers/net/phy/mdio_bus.c | 24 ++++++----
drivers/net/phy/phy_device.c | 62 ++++++++++++++++++++------
drivers/of/of_mdio.c | 27 +++++++++--
include/linux/phy.h | 6 ++-
net/core/net-sysfs.c | 9 ++++
net/dsa/dsa.c | 41 ++++++++++++++---
13 files changed, 181 insertions(+), 51 deletions(-)
--
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Russell King <hidden> Date: 2015-09-24 19:18:29
of_mdio_find_bus() leaks a struct device refcount, caused by using
class_find_device() and not realising that the device reference has
its refcount incremented:
* Note, you will need to drop the reference with put_device() after use.
...
while ((dev = class_dev_iter_next(&iter))) {
if (match(dev, data)) {
get_device(dev);
break;
}
Update the comment, and arrange for the phy code to drop this refcount
when disposing of a reference to it.
Signed-off-by: Russell King <redacted>
---
drivers/net/phy/mdio-mux.c | 19 +++++++++++++------
drivers/net/phy/mdio_bus.c | 4 +++-
2 files changed, 16 insertions(+), 7 deletions(-)
@@ -113,18 +113,18 @@ int mdio_mux_init(struct device *dev,if(!parent_bus_node)return-ENODEV;-parent_bus=of_mdio_find_bus(parent_bus_node);-if(parent_bus==NULL){-ret_val=-EPROBE_DEFER;-gotoerr_parent_bus;-}-pb=devm_kzalloc(dev,sizeof(*pb),GFP_KERNEL);if(pb==NULL){ret_val=-ENOMEM;gotoerr_parent_bus;}+parent_bus=of_mdio_find_bus(parent_bus_node);+if(parent_bus==NULL){+ret_val=-EPROBE_DEFER;+gotoerr_parent_bus;+}+pb->switch_data=data;pb->switch_fn=switch_fn;pb->current_child=-1;
@@ -173,6 +173,10 @@ int mdio_mux_init(struct device *dev,dev_info(dev,"Version "DRV_VERSION"\n");return0;}++/* balance the reference of_mdio_find_bus() took */+put_device(&pb->mii_bus->dev);+err_parent_bus:of_node_put(parent_bus_node);returnret_val;
@@ -189,6 +193,9 @@ void mdio_mux_uninit(void *mux_handle)mdiobus_free(cb->mii_bus);cb=cb->next;}++/* balance the reference of_mdio_find_bus() in mdio_mux_init() took */+put_device(&pb->mii_bus->dev);}EXPORT_SYMBOL_GPL(mdio_mux_uninit);
From: Russell King <hidden> Date: 2015-09-24 19:18:34
Current users of of_mdio_find_bus() leak a struct device refcount, as
they fail to clean up the reference obtained inside class_find_device().
Fix the DSA code to properly refcount the returned MDIO bus by:
1. taking a reference on the struct device whenever we assign it to
pd->chip[x].host_dev.
2. dropping the reference when we overwrite the existing reference.
3. dropping the reference when we free the data structure.
4. dropping the initial reference we obtained after setting up the
platform data structure, or on failure.
In step 2 above, where we obtain a new MDIO bus, there is no need to
take a reference on it as we would only have to drop it immediately
after assignment again, iow:
put_device(cd->host_dev); /* drop original assignment ref */
cd->host_dev = get_device(&mdio_bus_switch->dev); /* get our ref */
put_device(&mdio_bus_switch->dev); /* drop of_mdio_find_bus ref */
Signed-off-by: Russell King <redacted>
---
net/dsa/dsa.c | 38 +++++++++++++++++++++++++++++++-------
1 file changed, 31 insertions(+), 7 deletions(-)
@@ -634,6 +634,10 @@ static void dsa_of_free_platform_data(struct dsa_platform_data *pd)port_index++;}kfree(pd->chip[i].rtable);++/* Drop our reference to the MDIO bus device */+if(pd->chip[i].host_dev)+put_device(pd->chip[i].host_dev);}kfree(pd->chip);}
@@ -661,16 +665,22 @@ static int dsa_of_probe(struct device *dev)return-EPROBE_DEFER;ethernet=of_parse_phandle(np,"dsa,ethernet",0);-if(!ethernet)-return-EINVAL;+if(!ethernet){+ret=-EINVAL;+gotoout_put_mdio;+}ethernet_dev=of_find_net_device_by_node(ethernet);-if(!ethernet_dev)-return-EPROBE_DEFER;+if(!ethernet_dev){+ret=-EPROBE_DEFER;+gotoout_put_mdio;+}pd=kzalloc(sizeof(*pd),GFP_KERNEL);-if(!pd)-return-ENOMEM;+if(!pd){+ret=-ENOMEM;+gotoout_put_mdio;+}dev->platform_data=pd;pd->of_netdev=ethernet_dev;
@@ -691,7 +701,9 @@ static int dsa_of_probe(struct device *dev)cd=&pd->chip[chip_index];cd->of_node=child;-cd->host_dev=&mdio_bus->dev;++/* When assigning the host device, increment its refcount */+cd->host_dev=get_device(&mdio_bus->dev);sw_addr=of_get_property(child,"reg",NULL);if(!sw_addr)
@@ -711,6 +723,12 @@ static int dsa_of_probe(struct device *dev)ret=-EPROBE_DEFER;gotoout_free_chip;}++/* Drop the mdio_bus device ref, replacing the host+*devicewiththemdio_bus_switchdevice,keeping+*therefcountfromof_mdio_find_bus()above.+*/+put_device(cd->host_dev);cd->host_dev=&mdio_bus_switch->dev;}
@@ -744,6 +762,10 @@ static int dsa_of_probe(struct device *dev)}}+/* The individual chips hold their own refcount on the mdio bus,+*sodropours*/+put_device(&mdio_bus->dev);+return0;out_free_chip:
@@ -751,6 +773,8 @@ static int dsa_of_probe(struct device *dev)out_free:kfree(pd);dev->platform_data=NULL;+out_put_mdio:+put_device(&mdio_bus->dev);returnret;}
From: Russell King <hidden> Date: 2015-09-24 19:18:39
Re-implement the mdiobus module refcounting to ensure that we actually
ensure that the mdiobus module code does not go away while we might call
into it.
The old scheme using bus->dev.driver was buggy, because bus->dev is a
class device which never has a struct device_driver associated with it,
and hence the associated code trying to obtain a refcount did nothing
useful.
Instead, take the approach that other subsystems do: pass the module
when calling mdiobus_register(), and record that in the mii_bus struct.
When we need to increment the module use count in the phy code, use
this stored pointer. When the phy is deteched, drop the module
refcount, remembering that the phy device might go away at that point.
This doesn't stop the mii_bus going away while there are in-use phys -
it merely stops the underlying code vanishing.
Signed-off-by: Russell King <redacted>
---
drivers/net/phy/mdio_bus.c | 5 +++--
drivers/net/phy/phy_device.c | 32 ++++++++++++++++++--------------
include/linux/phy.h | 5 ++++-
3 files changed, 25 insertions(+), 17 deletions(-)
@@ -582,10 +582,15 @@ EXPORT_SYMBOL(phy_init_hw);intphy_attach_direct(structnet_device*dev,structphy_device*phydev,u32flags,phy_interface_tinterface){+structmii_bus*bus=phydev->bus;structdevice*d=&phydev->dev;-structmodule*bus_module;interr;+if(!try_module_get(bus->owner)){+dev_err(&dev->dev,"failed to get the bus module\n");+return-EIO;+}+/* Assume that if there is no driver, that it doesn't*exist,andweshouldusethegenphydriver.*/
@@ -600,20 +605,13 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev,err=device_bind_driver(d);if(err)-returnerr;+gotoerror;}if(phydev->attached_dev){dev_err(&dev->dev,"PHY already attached\n");-return-EBUSY;-}--/* Increment the bus module reference count */-bus_module=phydev->bus->dev.driver?-phydev->bus->dev.driver->owner:NULL;-if(!try_module_get(bus_module)){-dev_err(&dev->dev,"failed to get the bus module\n");-return-EIO;+err=-EBUSY;+gotoerror;}phydev->attached_dev=dev;
From: Russell King <hidden> Date: 2015-09-24 19:18:42
bus_find_device() is defined as:
* This is similar to the bus_for_each_dev() function above, but it
* returns a reference to a device that is 'found' for later use, as
* determined by the @match callback.
and it does indeed return a reference-counted pointer to the device:
while ((dev = next_device(&i)))
if (match(dev, data) && get_device(dev))
^^^^^^^^^^^^^^^
break;
klist_iter_exit(&i);
return dev;
What that means is that when we're done with the struct device, we must
drop that reference. Neither of_phy_connect() nor of_phy_attach() did
this when phy_connect_direct() or phy_attach_direct() failed.
With our previous patch, phy_connect_direct() and phy_attach_direct()
take a new refcount on the phy device when successful, so we can drop
our local reference immediatley after these functions, whether or not
they succeeded.
Signed-off-by: Russell King <redacted>
---
drivers/of/of_mdio.c | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
@@ -225,13 +228,19 @@ struct phy_device *of_phy_connect(struct net_device *dev,phy_interface_tiface){structphy_device*phy=of_phy_find_device(phy_np);+intret;if(!phy)returnNULL;phy->dev_flags=flags;-returnphy_connect_direct(dev,phy,hndlr,iface)?NULL:phy;+ret=phy_connect_direct(dev,phy,hndlr,iface);++/* refcount is held by phy_connect_direct() on success */+put_device(&phy->dev);++returnret?NULL:phy;}EXPORT_SYMBOL(of_phy_connect);
@@ -241,17 +250,27 @@ EXPORT_SYMBOL(of_phy_connect);*@phy_np:NodepointerforthePHY*@flags:flagstopasstothePHY*@iface:PHYdatainterfacetype+*+*Ifsuccessful,returnsapointertothephy_devicewiththeembedded+*structdevicerefcountincrementedbyone,orNULLonfailure.The+*refcountmustbedroppedbycallingphy_disconnect()orphy_detach().*/structphy_device*of_phy_attach(structnet_device*dev,structdevice_node*phy_np,u32flags,phy_interface_tiface){structphy_device*phy=of_phy_find_device(phy_np);+intret;if(!phy)returnNULL;-returnphy_attach_direct(dev,phy,flags,iface)?NULL:phy;+ret=phy_attach_direct(dev,phy,flags,iface);++/* refcount is held by phy_attach_direct() on success */+put_device(&phy->dev);++returnret?NULL:phy;}EXPORT_SYMBOL(of_phy_attach);
From: Russell King <hidden> Date: 2015-09-24 19:18:43
Take a refcount on the phy struct device when the phy device is attached
to a network device, and drop it after it's detached. This ensures that
a refcount is held on the phy device while the device is being used by
a network device, thereby preventing the phy_device from being
unexpectedly kfree()'d by phy_device_release().
Signed-off-by: Russell King <redacted>
---
drivers/net/phy/phy_device.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
@@ -591,6 +592,8 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev,return-EIO;}+get_device(d);+/* Assume that if there is no driver, that it doesn't*exist,andweshouldusethegenphydriver.*/
From: Russell King <hidden> Date: 2015-09-24 19:18:48
of_phy_find_device() increments the phy struct device refcount, which
we need to properly balance. Add code to network drivers using this
function to ensure that the struct device refcount is correctly
balanced.
For xgene, looking back in the history, we should be able to use
of_phy_connect() with a zero flags argument for the DT case as this is
how the driver used to operate prior to de7b5b3d790a ("net: eth: xgene:
change APM X-Gene SoC platform ethernet to support ACPI").
This leaves the Cavium Thunder BGX unfixed; fixing this driver is a
complicated task, one which the maintainers need to be involved with.
Signed-off-by: Russell King <redacted>
---
drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++++++++++++--------
drivers/net/ethernet/freescale/gianfar.c | 3 +++
drivers/net/ethernet/freescale/ucc_geth.c | 8 +++++++-
drivers/net/ethernet/marvell/mvneta.c | 2 ++
drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 ++
5 files changed, 30 insertions(+), 9 deletions(-)
@@ -689,16 +689,24 @@ static int xgene_enet_phy_connect(struct net_device *ndev)netdev_dbg(ndev,"No phy-handle found in DT\n");return-ENODEV;}-pdata->phy_dev=of_phy_find_device(phy_np);-}-phy_dev=pdata->phy_dev;+phy_dev=of_phy_connect(ndev,phy_np,&xgene_enet_adjust_link,+0,pdata->phy_mode);+if(!phy_dev){+netdev_err(ndev,"Could not connect to PHY\n");+return-ENODEV;+}++pdata->phy_dev=phy_dev;+}else{+phy_dev=pdata->phy_dev;-if(!phy_dev||-phy_connect_direct(ndev,phy_dev,&xgene_enet_adjust_link,-pdata->phy_mode)){-netdev_err(ndev,"Could not connect to PHY\n");-return-ENODEV;+if(!phy_dev||+phy_connect_direct(ndev,phy_dev,&xgene_enet_adjust_link,+pdata->phy_mode)){+netdev_err(ndev,"Could not connect to PHY\n");+return-ENODEV;+}}pdata->phy_speed=SPEED_UNKNOWN;
@@ -1702,6 +1702,7 @@ static void gfar_configure_serdes(struct net_device *dev)tbiphy=of_phy_find_device(priv->tbi_node);if(!tbiphy){dev_err(&dev->dev,"error: Could not get TBI device\n");+put_device(&tbiphy->dev);return;}
@@ -828,6 +828,8 @@ static int xemaclite_mdio_setup(struct net_local *lp, struct device *dev)if(!phydev)dev_info(dev,"MDIO of the phy is not registered yet\n");+else+put_device(&phydev->dev);return0;}
--
2.1.0
_______________________________________________
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
From: Russell King <hidden> Date: 2015-09-24 19:19:03
Validate that the phy_device passed into fixed_phy_update_state() is a
fixed-phy device before walking the list of phys for a fixed phy at the
same address.
Signed-off-by: Russell King <redacted>
---
drivers/net/phy/fixed_phy.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Russell King <hidden> Date: 2015-09-24 19:19:12
Add a phy_device_remove() function to complement phy_device_register(),
which undoes the effects of phy_device_register() by removing the phy
device from visibility, but not freeing it.
This allows these details to be moved out of the mdio bus code into
the phy code where this action belongs.
Signed-off-by: Russell King <redacted>
---
drivers/net/ethernet/freescale/gianfar.c | 5 +++--
drivers/net/phy/mdio_bus.c | 15 ++++++++++-----
drivers/net/phy/phy_device.c | 18 ++++++++++++++++++
include/linux/phy.h | 1 +
4 files changed, 32 insertions(+), 7 deletions(-)
@@ -1702,7 +1702,6 @@ static void gfar_configure_serdes(struct net_device *dev)tbiphy=of_phy_find_device(priv->tbi_node);if(!tbiphy){dev_err(&dev->dev,"error: Could not get TBI device\n");-put_device(&tbiphy->dev);return;}
@@ -1711,8 +1710,10 @@ static void gfar_configure_serdes(struct net_device *dev)*everythingforus?Resettingittakesthelinkdownandrequires*severalsecondsforittocomeback.*/-if(phy_read(tbiphy,MII_BMSR)&BMSR_LSTATUS)+if(phy_read(tbiphy,MII_BMSR)&BMSR_LSTATUS){+put_device(&tbiphy->dev);return;+}/* Single clk mode, mii mode off(for serdes communication) */phy_write(tbiphy,MII_TBICON,TBICON_CLK_SELECT);
From: Russell King <hidden> Date: 2015-09-24 19:19:15
of_find_net_device_by_node() uses class_find_device() internally to
lookup the corresponding network device. class_find_device() returns
a reference to the embedded struct device, with its refcount
incremented.
Add a comment to the definition in net/core/net-sysfs.c indicating the
need to drop this refcount, and fix the DSA code to drop this refcount
when the OF-generated platform data is cleaned up and freed. Also
arrange for the ref to be dropped when handling errors.
Signed-off-by: Russell King <redacted>
---
net/core/net-sysfs.c | 9 +++++++++
net/dsa/dsa.c | 5 ++++-
2 files changed, 13 insertions(+), 1 deletion(-)
From: Andrew Lunn <andrew@lunn.ch> Date: 2015-09-24 22:05:56
...
While looking at the DSA code, I noticed we have a
of_find_net_device_by_node(), and it looks like users of that are
similarly buggy - it looks like net/dsa/dsa.c is the only user. Fix
that too.
...
The mdiobus code also suffered from the same kind of leak, but thankfully
this only happened in one place - the mdio-mux code.
Hi Russell
I tested both of these with my board. It is a Freescale Vybrid, using
the FEC ethernet driver, and i have three switches attached, using
mdio-mux to give three mdio busses.
No obvious regressions, my board boots, the switches are all present
and correct. I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <andrew@lunn.ch>
Thanks
Andrew
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <andrew@lunn.ch>
So, is this a regression?
Please don't provide a "Tested-by: " tag is you encounter a new
problem which could have been introduced by the changes in question.
That _REALLY_ screws everything up for me.
Thank.
From: Russell King - ARM Linux <hidden> Date: 2015-09-24 22:16:09
On Thu, Sep 24, 2015 at 11:57:31PM +0200, Andrew Lunn wrote:
Hi Russell
I tested both of these with my board. It is a Freescale Vybrid, using
the FEC ethernet driver, and i have three switches attached, using
mdio-mux to give three mdio busses.
No obvious regressions, my board boots, the switches are all present
and correct. I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <andrew@lunn.ch>
Thanks for testing. Please could you confirm whether the same behaviour
is observed without the patches, just to make absolutely sure that isn't
a regression.
However, I think you are correct - I'm unable to locate where in the
DSA code:
- dst->master_dev's dev_hold() is undone (hence a reference left)
- dst is freed - dsa_probe() allocates it using kzalloc(), but
dsa_remove() and it's children don't free this structure.
There's no notifier which detects whether the underlying device has
gone away - it registers a netdev notifier (dsa_slave_netdevice_event)
but this only deals with slave devices, not the master device.
Thanks.
--
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <redacted>
So, is this a regression?
Sorry, worded that badly. Since DSA is still active, it should not be
possible to unload the FEC driver. DSA should have a reference to it,
and mdio-mux also should have a reference to the mdio bus of the FEC
driver.
As Russell requested, i will re-test without his patches, just to make
sure.
Andrew
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <redacted>
So, is this a regression?
Sorry, worded that badly. Since DSA is still active, it should not be
possible to unload the FEC driver. DSA should have a reference to it,
and mdio-mux also should have a reference to the mdio bus of the FEC
driver.
As Russell requested, i will re-test without his patches, just to make
sure.
Something needs to hold into the underlying device refcount of a DSA
blob so that an unload can't even be attempted in that state.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Andrew Lunn <andrew@lunn.ch> Date: 2015-09-24 22:58:52
Thanks for testing. Please could you confirm whether the same behaviour
is observed without the patches, just to make absolutely sure that isn't
a regression.
So i tested this now.
I have two FEC interfaces. One i my main access interface, and the
second is used by DSA to access switches. With your patches, the
module Used by count is equal to the number of interfaces which are
up.
Without your patches, the count is always 0.
When i try to remove the fec module, without your patches, but DSA
still using the interface, i get the same
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
as with your patch. So this is not a regression.
Andrew
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module
build, DSA has issues with that.
Tested-by: Andrew Lunn <andrew@lunn.ch>
So, is this a regression?
Sorry, worded that badly. Since DSA is still active, it should not be
possible to unload the FEC driver. DSA should have a reference to it,
and mdio-mux also should have a reference to the mdio bus of the FEC
driver.
As Russell requested, i will re-test without his patches, just to make
sure.
Something needs to hold into the underlying device refcount of a DSA
blob so that an unload can't even be attempted in that state.
Holding a reference on a struct device does _not_ stop the device
being unbound or the module driving it being removed. It merely
stops the struct device from being freed before all references have
been dropped.
Devices are always free to be unbound from their bound drivers
irrespective of the struct device refcount. Even taking a reference
on the module (via try_module_get()) does not stop this.
--
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
From: Russell King - ARM Linux <hidden> Date: 2015-09-24 23:33:27
On Fri, Sep 25, 2015 at 12:50:33AM +0200, Andrew Lunn wrote:
quoted
Thanks for testing. Please could you confirm whether the same behaviour
is observed without the patches, just to make absolutely sure that isn't
a regression.
So i tested this now.
I have two FEC interfaces. One i my main access interface, and the
second is used by DSA to access switches. With your patches, the
module Used by count is equal to the number of interfaces which are
up.
Without your patches, the count is always 0.
That will be as a result of the MDIO bus module refcounting patch -
"phy: fix mdiobus module safety". The code prior to that patch was
totally useless and ineffectual - it might as well not even have
been present, because it would never have any effect. bus_module
would always be NULL in phy_attach_direct().
While my change makes the code start to work as originally intended,
it's still unsafe: there's nothing to stop you manually unbinding the
driver providing the MDIO bus from the struct device. The driver
will then free the resources it claimed in its probe function, which
may include the MMIO mapping for the MDIO bus accessor functions.
If the accessors are then called, despite keeping the mdio bus, phy,
etc data structures properly refcounted, the kernel will oops when
the (many) MDIO bus drivers hit the free'd MMIO mapping. This is,
unfortunately, just another pre-existing bug in this code.
To stop that, we need some way to say "this MDIO bus has been removed,
prevent further access" and that needs to be done in a race free way.
Right now, that doesn't exist.
--
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
On 24/09/15 12:17, Russell King - ARM Linux wrote:
Hi,
The third version of this series fixes the build error which David
identified, and drops the broken changes for the Cavium Thunger BGX
ethernet driver as this driver requires some complex changes to
resolve the leakage - and this is best done by people who can test
the driver.
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"
I _think_ I've been able to build-test all the drivers touched by
that patch to some degree now, though several of them needed the
Kconfig hacked to allow it (not all had || COMPILE_TEST clause on
their dependencies.)
Previous cover letters below:
This is the second version of the series, with the comments David had
on the first patch fixed up. Original series description with updated
diffstat below.
While looking at the DSA code, I noticed we have a
of_find_net_device_by_node(), and it looks like users of that are
similarly buggy - it looks like net/dsa/dsa.c is the only user. Fix
that too.
Hi,
While looking at the phy code, I identified a number of weaknesses
where refcounting on device structures was being leaked, where
modules could be removed while in-use, and where the fixed-phy could
end up having unintended consequences caused by incorrect calls to
fixed_phy_update_state().
This patch series resolves those issues, some of which were discovered
with testing on an Armada 388 board. Not all patches are fully tested,
particularly the one which touches several network drivers.
When resolving the struct device refcounting problems, several different
solutions were considered before settling on the implementation here -
one of the considerations was to avoid touching many network drivers.
The solution here is:
phy_attach*() - takes a refcount
phy_detach*() - drops the phy_attach refcount
Provided drivers always attach and detach their phys, which they should
already be doing, this should change nothing, even if they leak a refcount.
of_phy_find_device() and of_* functions which use that take
a refcount. Arrange for this refcount to be dropped once
the phy is attached.
This is the reason why the previous change is important - we can't drop
this refcount taken by of_phy_find_device() until something else holds
a reference on the device. This resolves the leaked refcount caused by
using of_phy_connect() or of_phy_attach().
Even without the above changes, these drivers are leaking by calling
of_phy_find_device(). These drivers are addressed by adding the
appropriate release of that refcount.
The mdiobus code also suffered from the same kind of leak, but thankfully
this only happened in one place - the mdio-mux code.
I also found that the try_module_get() in the phy layer code was utterly
useless: phydev->dev.driver was guaranteed to always be NULL, so
try_module_get() was always being called with a NULL argument. I proved
this with my SFP code, which declares its own MDIO bus - the module use
count was never incremented irrespective of how I set the MDIO bus up.
This allowed the MDIO bus code to be removed from the kernel while there
were still PHYs attached to it.
One other bug was discovered: while using in-band-status with mvneta, it
was found that if a real phy is attached with in-band-status enabled,
and another ethernet interface is using the fixed-phy infrastructure, the
interface using the fixed-phy infrastructure is configured according to
the other interface using the in-band-status - which is caused by the
fixed-phy code not verifying that the phy_device passed in is actually
a fixed-phy device, rather than a real MDIO phy.
Lastly, having mdio_bus reversing phy_device_register() internals seems
like a layering violation - it's trivial to move that code to the phy
device layer.
drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++----
drivers/net/ethernet/freescale/gianfar.c | 6 ++-
drivers/net/ethernet/freescale/ucc_geth.c | 8 +++-
drivers/net/ethernet/marvell/mvneta.c | 2 +
drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 +
drivers/net/phy/fixed_phy.c | 2 +-
drivers/net/phy/mdio-mux.c | 19 +++++---
drivers/net/phy/mdio_bus.c | 24 ++++++----
drivers/net/phy/phy_device.c | 62 ++++++++++++++++++++------
drivers/of/of_mdio.c | 27 +++++++++--
include/linux/phy.h | 6 ++-
net/core/net-sysfs.c | 9 ++++
net/dsa/dsa.c | 41 ++++++++++++++---
13 files changed, 181 insertions(+), 51 deletions(-)
On 24/09/15 12:17, Russell King - ARM Linux wrote:
quoted
Hi,
The third version of this series fixes the build error which David
identified, and drops the broken changes for the Cavium Thunger BGX
ethernet driver as this driver requires some complex changes to
resolve the leakage - and this is best done by people who can test
the driver.
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"
I _think_ I've been able to build-test all the drivers touched by
that patch to some degree now, though several of them needed the
Kconfig hacked to allow it (not all had || COMPILE_TEST clause on
their dependencies.)
Series applied, thanks.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html