On 8/21/23 15:58, Vladimir Oltean wrote:

On Mon, Aug 21, 2023 at 02:46:53PM -0400, Sean Anderson wrote:

quoted

After further review, it seems the reason 28g can get away without this
is because there's a one-to-one mapping between protocol controllers and
lanes. Unfortunately, that regularity is not present for 10g.

--Sean

There are some things I saw in your phy-fsl-lynx-10g.c driver and device
tree bindings that I don't understand (the concept of lane groups)

Each lane group corresponds to a phy device (struct phy). For protocols
like PCIe or XAUI which can use multiple lanes, this lets the driver
coordinate configuring all lanes at once in the correct order.

and
I'm not sure if they're related with what you're saying here, so if you
could elaborate a bit more (ideally with an example) on the one-to-one
mapping and the specific problems it causes, it would be great.

For e.g. the LS2088A, SerDes1 lanes H-A use SG1-8 and XFI1-8. SerDes2
lanes A-H use SG9-16 and XFI9-16. Each lane has its own controller, and
the mapping is 1-to-1. In the PCCRs, each controller uses the same
value, and is mapped in a regular way. So you can go directly from the
lane number to the right value to mask in the PCCR, with a very simple
translation scheme.

For e.g. the LS1046A, SerDes1 lane D uses XFI.9 (aka XFIA) and lane C
uses XFI.10 (aka XFIB). This is the opposite of how SerDes1 lanes A-D
use SGMII.9, .10, .5, and .6 (aka SGMIIA-D).

For e.g. the T4240, SerDes1 lanes A-H use sg1.5, .6, .10, .9, .1, .2,
.3, .4 (aka SGMII E, F, H, G, A, B, C, D).

For e.g. the B4860, SerDes lanes A uses sgmii1 or sgmii5 and B uses
sgmii2 or sgmii6. The MAC selected is determined based on the value
programmed into PCCR2.

While I appreciate that your hardware engineers did a better job for
28g, many 10g serdes arbitrarily map lanes to protocol controllers.
I think the mapping is too irregular to tame, and it is better to say
"if you want this configuration, program this value".

I may be off with my understanding of the regularity you are talking about,
but the LX2160 (and Lynx 28G block) also has multi-lane protocols like 40G,
100G, assuming that's what you are talking about. I haven't started yet
working on those for the mtip_backplane driver, but I'm not currently
seeing a problem with the architecture where a phy_device represents a
single lane that's part of a multi-lane port, and not an entire group.

Resetting one lane in a group will reset the rest, which could confuse
the driver. Additionally, treating the lanes as one phy lets us set the
reset direction and first lane bits correctly.

In my imagination, there are 2 cases:
- all 4 lanes are managed by the single dpaa2-mac consumer (which has 4
  phandles, and iterates over them with a "for" loop)
- each of the 4 lanes is managed by the respective backplane AN/LT core,
  and thus, there's one phandle to each lane

By doing the grouping in the driver, we also simplify the consumer
implementation. The MAC can always use a single phy, without worrying
about the actual number of lanes. This matches the hardware, since the
MAC is going to talk XGMII (or whatever) to the protocol controller
anyway.

I think it will be a lot easier to add multi-lane support with this
method because it gives the driver more information about what's going
on. The driver can control the whole configuration/reset process and the
timing.

I sketched some dt-bindings for the second case here, so I guess it must
be the first scenario that's somehow problematic?
https://cas5-0-urlprotect.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fpatchwork.kernel.org%2fproject%2fnetdevbpf%2fpatch%2f20230817150644.3605105%2d9%2dvladimir.oltean%40nxp.com%2f&umid=9e644233-009e-4197-a266-5d9a85eb1148&auth=d807158c60b7d2502abde8a2fc01f40662980862-cc1d5330d84af8fa40745b165a44849db50f8a67

Yes, no issues with the second case.

--Sean

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel