Re: FEC MDIO timeout and polled IO
From: Francesco Dolcini <hidden>
Date: 2022-03-25 15:35:27
Hello Andrew On Fri, Mar 25, 2022 at 04:17:13PM +0100, Andrew Lunn wrote:
On Fri, Mar 25, 2022 at 03:08:08PM +0100, Francesco Dolcini wrote:quoted
Hello Andrew and all, I was recently debugging an issue in the FEC driver, about 2% of the time the driver is failing with "MDIO read timeout" at boot on a 5.4 kernel. This issue is not new and from time to time appear again, it seems that the previous interrupt based mechanism is somehow easy to break. I backported your patch f166f890c8f0 (net: ethernet: fec: Replace interrupt driven MDIO with polled IO, 2020-05-02) to kernel 5.4 and it seems that it fixes the issue (I was able to do 470 power cycles, while before it was failing after a couple of hundreds cycles best case). Shouldn't this patch be backported to kernel 5.4?Hi Francesco This patch was purely a performance boost, it was not a bug fix in any way. That change also caused a lot of pain. There are at least two different implementations of the MDIO bus in the FEC, and they behaviour slightly differently. So what worked for me with the Vybrid broke some other platforms. It took an NXP software engineer talking to there hardware guys to figure out how to do this correctly. Which is why you will see a complicated patch history. I personally would not recommend a back port, unless you can test the back port on a wide range of SoC with the FEC.
I can test quite a few of i.MX SoC, but there is more than that using this driver. I do not see a reason to push for such a change if you do not feel like being a good idea.
If you are getting timeouts, i would suggest you look at whatever else is happening in the system during boot. Are interrupts getting disabled for too long? Is something blocking the running of the completion?
I tried to do some debugging, but it was incredibly painful given that the issue manifest itself only after a couple of hundreds boots. I also tried the very simple workaround to double the timeout but it didn't work out. Bad enough the issue started to appear after updating to a more recent 5.4 kernel patch version.
Or just update to v5.15.
I will probably just keep your patch in our tree till we are able to migrate to a newer kernel, it seems to work pretty well (and yes, I took also this [0]). Thanks a lot, Francesco! [0] 0f0011824921 (net: fec: fix MDIO probing for some FEC hardware blocks, 2020-10-28)