On December 14, 2015 11:26:21 PM PST, Dustin Byford [off-list ref] wrote:

On Mon Dec 07 17:35, Russell King - ARM Linux wrote:

quoted

Hi,

Hello.

quoted

SFP modules are hot-pluggable ethernet transceivers; they can be
detected at runtime and accordingly configured.  There are a range of
modules offering many different features.

Some SFP modules have PHYs conventional integrated into them, others
drive a laser diode from the Serdes bus.  Some have monitoring,

others

quoted

do not.

Some SFP modules want to use SGMII over the Serdes link, others want
to use 1000base-X over the Serdes link.

This makes it non-trivial to support with the existing code

structure.

quoted

Not wanting to write something specific to the mvneta driver, I

decided

quoted

to have a go at coming up with something more generic.

My initial attempts were to provide a PHY driver, but I found that
phylib's state machine got in the way, and it was hard to support two
chained PHYs.  Conversely, having a fixed DT specified setup (via
the fixed phy infrastructure) would allow some SFP modules to work,

but

quoted

not others.  The same is true of the "managed" in-band status (which
is SGMII.)

The result is that I came up with phylink - an infrastructure layer
which sits between the network driver and any attached PHY, and a
SFP module layer detects the SFP module, and configures phylink
accordingly.

Overall, this supports:

* switching the serdes mode at the NIC driver
* controlling autonegotiation and autoneg results
* allowing PHYs to be hotplugged
* allowing SFP modules to be hotplugged with proper link indication
* fixed-mode links without involving phylib
* flow control
* EEE support
* reading SFP module EEPROMs

Overall, phylink supports several link modes, with dynamic switching
possible between these:
* A true fixed link mode, where the parameters are set by DT.
* PHY mode, where we read the negotiation results from the PHY

registers

quoted

  and pass them to the NIC driver.
* SGMII mode, where the in-band status indicates the speed, duplex

and

quoted

  flow control settings of the link partner.
* 1000base-X mode, where the in-band status indicates only duplex and
  flow control settings (different, incompatible bit layout from

SGMII.)

I've been working on some similar code to handle interactions with a
wide range of SFF modules, 1G to 100G, on Linux network switches for
some time.  For practical reasons a lot of that was in userspace but
I've been planning and recently working on an SFF kernel driver that
does some of what's done in this series.  I think the model you're
proposing is right on, and since you're further along in implementation
I'd like to help round out support for the other SFF modules if I can.
Then make this work on the network ASICs I have access to.

Any concrete plans for QSFP or the new 25G modules?

quoted

Ethtool support is included, as well as emulation of the MII

registers

quoted

for situations where a PHY is not attached, giving compatible

emulation

quoted

of existing user interfaces where required.

The patches here include modification of mvneta (against 4.4-rc1, so
probably won't apply to current development tips.)  It basically
hooks into the places where the phylib would hook into.

DT wise, the changes needed to support SFP look like this (example
taken from Clearfog):

 			ethernet@34000 {
+				managed = "in-band-status";
 				phy-mode = "sgmii";
 				status = "okay";
-
-				fixed-link {
-					speed = <1000>;
-					full-duplex;
-				};
 			};
...
+	sfp: sfp {
+		compatible = "sff,sfp";
+		i2c-bus = <&i2c1>;
+		los-gpio = <&expander0 12 GPIO_ACTIVE_HIGH>;
+		moddef0-gpio = <&expander0 15 GPIO_ACTIVE_LOW>;
+		sfp,ethernet = <&eth2>;

Using &eth2 is unambiguous in the this case because there's only one
serdes and one mac involved.  To specify the mac/serdes/cage
associations at the same level of detail as the gpios it might be nice
(at least for some devices) to point to a serdes node (or 4 in the case
of QSFP) instead of &eth2.  Any thoughts on that?

Using a phandle here allows for quite a lot of flexibility on how you want to associate a given SFP to its data plane partner. I do not think we need to get more strict than that strictly mandate an actual Ethernet controller node. These Marvell adapters typically have one or more " ports", each of them being backed by a netdev. The same could be true with a switch properly modeled.

Switch ASICs, and I imagine at least some NICs, are really flexible in
terms of how serdes are wired to a cage.  Both in the sense that the
board designer gets to pick which wires route to the cage based on
physical constraints and the user gets to pick which serdes or group of
serdes compose the ethernet device.  For example, using a breakout
cable
to get 4xSFP out of a QSFP or the other way around.

Perhaps the simple case (sfp,ethernet -> &eth2) can remain simple, but
I'd be interested in any thoughts you have on introducing a serdes
layer here.

I think adding such a layer would make it easier to 1) make serdes to
cage mappings part of the platform description (DT or ACPI) and 2)
allow
automatic reconfiguration of the mac based on the SFF module.  For
example, if a user plugs in a QSFP->4xSFP breakout cable why not
automatically create four netdevs instead of one?

Would this be something you expect to happen dynamically? Not that this does not seem reasonable but would these netdevs serve a different purpose than being control endpoints, or would they become real logical netdevs with separate data planes at the MAC they would be linked to?

quoted

+		tx-disable-gpio = <&expander0 14 GPIO_ACTIVE_HIGH>;
+		tx-fault-gpio = <&expander0 13 GPIO_ACTIVE_HIGH>;
+	};

These DT changes are omitted from this patch set as the baseline DT
file is not in mainline yet (has been submitted.)

Cool.  Do you have a link to the DT patches?


In short, I think this is awesome, and I'd like to help where I can.
I'll start by having a look at the rest of the series.  I'd like to
apply it and see if I can make it work on one of my systems.

Thanks,

	--Dustin

-- 
Florian