On 12/19/2019 12:21 AM, Russell King - ARM Linux admin wrote:

On Wed, Dec 18, 2019 at 08:15:59PM +0000, Alexandru Marginean wrote:

quoted

On 12/18/2019 6:22 PM, Russell King - ARM Linux admin wrote:

quoted

On Wed, Dec 18, 2019 at 03:00:41PM +0000, Alexandru Marginean wrote:

quoted

On 12/18/2019 2:29 PM, Russell King - ARM Linux admin wrote:

quoted

On Wed, Dec 18, 2019 at 03:21:02PM +0200, Vladimir Oltean wrote:

quoted

- The at803x.c driver explicitly checks for the ACK from the MAC PCS,
and prints "SGMII link is not ok" otherwise, and refuses to bring the
link up. This hurts us in 4.19 because I think the check is a bit
misplaced in the .aneg_done callback. To be precise, what we observe
is that this function is not called by the state machine a second,
third time etc to recheck if the AN has completed in the meantime. In
current net-next, as far as I could figure out, at803x_aneg_done is
dead code. What is ironic about the commit f62265b53ef3 ("at803x:
double check SGMII side autoneg") that introduced this function is
that it's for the gianfar driver (Freescale eTSEC), a MAC that has
never supported reprogramming itself based on the in-band config word.
In fact, if you look at gfar_configure_serdes, it even configures its
register 0x4 with an advertisement for 1000Base-X, not SGMII (0x4001).
So I really wonder if there is any real purpose to this check in
at803x_aneg_done, and if not, I would respectfully remove it.

Please check whether at803x will pass data if the SGMII config exchange
has not completed - I'm aware of some PHYs that, although link comes up
on the copper side, if AN does not complete on the SGMII side, they
will not pass data, even if the MAC side is forced up.

I don't see any configuration bits in the 8031 that suggest the SGMII
config exchange can be bypassed.

quoted

- The vsc8514 PHY driver configures SerDes AN in U-Boot, but not in
Linux. So we observe that if we disable PHY configuration in U-Boot,
in-band AN breaks in Linux. We are actually wondering how we should
fix this: from what you wrote above, it seems ok to hardcode SGMII AN
in the PHY driver, and just ignore it in the PCS if managed =
"in-band-status" is not set with PHYLINK. But as you said, in the
general case maybe not all PHYs work until they haven't received the
ACK from the MAC PCS, which makes this insufficient as a general
solution.

But the 2 cases above illustrate the lack of consistency among PHY
drivers w.r.t. in-band aneg.

Indeed - it's something of a mine field at the moment, because we aren't
quite sure whether "SGMII" means that the PHY requires in-band AN or
doesn't provide it. For the Broadcom case I mentioned, when it's used on
a SFP, I've had to add a quirk to phylink to work around it.

The problem is, it's not a case that the MAC can demand that the PHY
provides in-band config - some PHYs are incapable of doing so. Whatever
solution we come up with needs to be a "negotiation" between the PHY
driver and the MAC driver for it to work well in the known scenarios -
like the case with the Broadcom PHY on a SFP that can be plugged into
any SFP supporting network interface...

Some sort of capability negotiation does seem to be the proper solution.
We can have a new capabilities field in phydev for system interface
capabilities and match that with MAC capabilities, configuration, factor
in the quirks.  The result would tell if a solution is possible,
especially with quirky PHYs, and if PHY drivers need to enable AN.

Until we have that in place, any recommended approach for PHY drivers,
is it acceptable to hardcode system side AN on as a short term fix?
I've just tested VSC8514 and it doesn't allow traffic through if SI AN
is enabled but does not complete.  We do use it with AN on on NXP
systems, and it only works because U-Boot sets things up that way, but
relying on U-Boot isn't great.
Aquantia PHYs we use also require AN to complete if enabled.  For them
Linux depends on U-Boot or on PHY firmware to enable AN.  I don't know
if anyone out there uses these PHYs with AN off.  Would a patch that
hardcodes AN on for any of these PHYs be acceptable?

I'm not sure why you're talking about hard-coding anything. As I've
already mentioned, phylink allows you to specify today whether you
want to use in-band AN or not, provided the MAC implements it as is
done with mvneta and mvpp2.

I was asking about PHY drivers, not MAC, in the meantime I noticed
phy_device carries a pointer to phylink.  I assume it's OK if PHY
drivers check link_an_mode and set up PHY system interface based on it.

Definitely not on several counts:

1. it's an opaque structure to everything but phylink itself (you
    may notice that it's defined in phylink.c, and that is why - it
    is unsafe to go poking about in it.)  Lessons should've been
    learnt from phylib allowing MAC drivers to go poking about in
    phylib internal state (which I hear phylib maintainers regret
    allowing.)  This is an intentional design choice on my part to
    ensure that we do not fall into the same trap.

2. phylink really needs to know what the PHY properties are before
    the driver goes poking about in phylink - the current phylink
    state at a particular point in time may not be the operating state
    ultimately chosen for the PHY.

3. phylib may not be used with phylink, and we need phylib to
    continue to work in phylink-less systems.

(2) is the killer, because it's useless trying to look at the phylink
state when the PHY is "attached" (and hence config_init is called.)

My bad, I just assumed without checking that phylink is more like phylib 
than it actually is.  (3) is probably less of a problem, phy drivers 
could check if phylink pointers is NULL, but it doesn't matter anyway, 
since the phylink structure is opaque.

To see why we need PHY property information earlier, look at
phylink.c in net-next, specifically:

1. phylink_bringup_phy(), where we have a special case for Clause 45
    PHYs that switch their interface modes.

2. phylink_phy_no_inband() and phylink_sfp_connect_phy() which
    special-cases the Broadcom PHY that provides no in-band AN.
    That is particularly relevant to my point (2) above.

3. phylink_sfp_connect_phy() needing to know the capabilities of
    the PHY before the PHY driver has been attached, so that we can
    select a particular interface mode that is suitable for the PHY
    to co-operate with the MAC.

This is all fair and I'm not arguing against a proper solution that 
involves a capability exchange between PHY and MAC.

Until we have that in place, or if phylib is used, PHY drivers should 
probably apply a recommended configuration in config_init based on 
information available to them at that time.  For phy-mode = "sgmii" or 
"qsgmii" I think PHY drivers should enable AN on system side by default 
if that's an option in the PHY hardware.  Currently some seem to do it, 
other don't.  In that context vsc8514 config_init could hardcode 
enabling of AN, phy-mode wouldn't even matter as the PHY only supports 
qsgmii.

This kind of change does not address the overall problem but at least 
makes PHYs more uniform and predictable.

In the proper solution phylink could later determine what the PHY 
configuration should be based on MAC capabilities and configuration and 
re-apply it using a new op, potentially turning off system side AN in 
vsc8514 case.  Does this look reasonable?

Thanks!
Alex