Re: [PATCH] net: stmmac: fix MAC not working when system resume back with WoL enabled

On Wed, Sep 01, 2021 at 10:25:15AM +0000, Joakim Zhang wrote:

Hi Vladimir,

quoted

-----Original Message-----
From: Vladimir Oltean <olteanv@gmail.com>
Sent: 2021年9月1日 17:22
To: Joakim Zhang <redacted>
Cc: peppe.cavallaro@st.com; alexandre.torgue@foss.st.com;
joabreu@synopsys.com; davem@davemloft.net; kuba@kernel.org;
mcoquelin.stm32@gmail.com; linux@armlinux.org.uk;
netdev@vger.kernel.org; andrew@lunn.ch; f.fainelli@gmail.com;
hkallweit1@gmail.com; dl-linux-imx [off-list ref]
Subject: Re: [PATCH] net: stmmac: fix MAC not working when system resume
back with WoL enabled

On Wed, Sep 01, 2021 at 05:02:28PM +0800, Joakim Zhang wrote:

quoted

We can reproduce this issue with below steps:
1) enable WoL on the host
2) host system suspended
3) remote client send out wakeup packets We can see that host system
resume back, but can't work, such as ping failed.

After a bit digging, this issue is introduced by the commit
46f69ded988d
("net: stmmac: Use resolved link config in mac_link_up()"), which use
the finalised link parameters in mac_link_up() rather than the
parameters in mac_config().

There are two scenarios for MAC suspend/resume:

1) MAC suspend with WoL disabled, stmmac_suspend() call
phylink_mac_change() to notify phylink machine that a change in MAC
state, then .mac_link_down callback would be invoked. Further, it will
call phylink_stop() to stop the phylink instance. When MAC resume
back, firstly phylink_start() is called to start the phylink instance,
then call phylink_mac_change() which will finally trigger phylink
machine to invoke .mac_config and .mac_link_up callback. All is fine
since configuration in these two callbacks will be initialized.

2) MAC suspend with WoL enabled, phylink_mac_change() will put link
down, but there is no phylink_stop() to stop the phylink instance, so
it will link up again, that means .mac_config and .mac_link_up would
be invoked before system suspended. After system resume back, it will
do DMA initialization and SW reset which let MAC lost the hardware
setting (i.e MAC_Configuration register(offset 0x0) is reset). Since
link is up before system suspended, so .mac_link_up would not be
invoked after system resume back, lead to there is no chance to
initialize the configuration in .mac_link_up callback, as a result,
MAC can't work any longer.

Have you tried putting phylink_stop in .suspend, and phylink_start in .resume?

Yes, I tried, but the system can't be wakeup with remote packets.
Please see the code change.

That makes it a PHY driver issue then, I guess?
At least some PHY drivers avoid suspending when WoL is active, like lan88xx_suspend.
Even the phy_suspend function takes wol.wolopts into consideration
before proceeding to call the driver. What PHY driver is it?

quoted

Do you know exactly why it used to work prior to this patch?

Yes, since it configures the MAC_CTRL_REG register in .mac_config callback,
it will be called when system resume back with WoL enabled.
https://elixir.bootlin.com/linux/v5.4.143/source/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c#L852

If configure the MAC_CTRL_REG register in .mac_link_up callback, when system resume back with WoL active,
.mac_link_up would not be called, so MAC can't work any longer.
https://elixir.bootlin.com/linux/v5.14-rc7/source/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c#L1044

Ok, so it worked because phylink_mac_change triggers a phylink resolve,
and that function calls phylink_mac_config if the link is up (which it is),
but phylink_link_up only if the link state actually changed (which it did not).
So you are saying that the momentary link flap induced by phylink_mac_change(false),
which set pl->mac_link_dropped = true, all consumed itself _before_ the actual
suspend, and therefore does not help after the resume. Interesting behavior.
Bad assumption in the stmmac driver, if the intention was for the link
state change to be induced to phylink after the resume?