Re: [EXT] Re: [RFC, net-next] net: qos: introduce a frer action to implement 802.1CB

On Fri, Oct 01, 2021 at 08:55:24PM +0300, Vladimir Oltean wrote:

On Fri, Oct 01, 2021 at 10:27:12AM -0700, Vinicius Costa Gomes wrote:

quoted

Xiaoliang Yang [off-list ref] writes:

quoted

Hi Vinicius,

On Sep 29, 2021 at 6:35:59 +0000, Vinicius Costa Gomes wrote:

quoted

quoted

This patch introduce a frer action to implement frame replication and
elimination for reliability, which is defined in IEEE P802.1CB.

An action seems, to me, a bit too limiting/fine grained for a frame replication
and elimination feature.

At least I want to hear the reasons that the current hsr/prp support cannot be
extended to support one more tag format/protocol.

And the current name for the spec is IEEE 802.1CB-2017.

802.1CB can be set on bridge ports, and need to use bridge forward
Function as a relay system. It only works on identified streams,
unrecognized flows still need to pass through the bridged network
normally.

This ("only on identified streams") is the strongest argument so far to
have FRER also as an action, in adition to the current hsr netdevice
approach.

quoted

But current hsr/prp seems only support two ports, and cannot use the
ports in bridge. It's hard to implement FRER functions on current HSR
driver.

That the hsr netdevice only support two ports, I think is more a bug
than a design issue. Which will need to get fixed at some point. 

What do you mean 'a bug'? HSR and PRP, as protocols, use _two_ ports,
see IEC 62439-3, that's where the "D" (doubly attached node) in DANH and
DANP comes from. There's no TANH/TANH for "triply attached node".
It doesn't scale.

quoted

Speaking of functions, one thing that might be interesting is trying to
see if it makes sense to make part of the current hsr functionality a
"library" so it can be used by tc-frer as well. (less duplication of
bugs).

You mean tc-frer should inherit from the get-go the plethora of bugs
from the unmaintained hsr driver? :)

That would be good for hsr, which is in a pretty poor state, but the
design of the 802.1CB spec isn't really in its favor sadly.

quoted

quoted

You can see chapter "D.2 Example 2: Various stack positions" in IEEE 802.1CB-2017,
Protocol stack for relay system is like follows:

             Stream Transfer Function
                |             |
  				|    	Sequence generation
                |       	Sequence encode/decode
  Stream identification		Active Stream identification
				|			  |
  			    |		Internal LAN---- Relay system forwarding
				|						|		|
				MAC						MAC		MAC

Use port actions to easily implement FRER tag add/delete, split, and
recover functions.

Current HSR/PRP driver can be used for port HSR/PRP set, and tc-frer
Action to be used for stream RTAG/HSR/PRP set and recover.

I am still reading the spec and trying to imagine how things would fit
together:
  - for which use cases tc-frer would be useful;
  - for which use cases the hsr netdevice would be useful;
  - would it make sense to have them in the same system?

You could use FRER in networks where normally you'd use HSR (aka rings).
In fact the 802.1CB demonstration I have, which uses the NXP tsntool
program with the downstream genetlink tsn interface, does exactly that:
https://github.com/vladimiroltean/tsn-scripts

Basically FRER is IEEE's take on redundancy protocols and more like a
generalization of HSR/PRP, the big changes are:
- not limited to two (or any number of) ports
- more than one type of stream/flow identification function: can look at
  source/destination MAC, source/destination IP, VLAN, and most
  importantly, there can be passive stream identification functions (don't
  modify the packet) and active stream identification functions (do
  modify the packet).

Please note that we've already started modeling IEEE 802.1CB stream
identification functions as tc flower filters, since those map nicely on top.
We use these for PSFP (former 802.1Qci) tc-police and tc-gate actions
(yes, tc-police is single-bucket and color-unaware, that needs to be improved).

Basically IEEE 802.1CB is a huge toolbox, the spec gives you the tools
but it doesn't tell you how to use them, that's why the stream
identification functions are so generic and decoupled from the
redundancy protocol itself.

In both HSR and PRP, sequence numbers are kept per source MAC address,
that is absolutely baken into the standard.

But think about this. When the sequence number is kept per source
station, frames sent from node A to multiple destinations (nodes B and C)
will be part of the same stream. So nodes B and C will see
discontinuities in the sequence numbers when node A talks to them.

The opposite is true as well. When sequence numbers are kept per
destination MAC address, then frames sent from multiple talkers (nodes A
and B) to the same destination (node C) will be interpreted as part of
the same stream by the listener. So there will be jumps in sequence
numbers seen by C when A and B are simultaneously transmitting to it.

Which type of stream identification you need depends on the traffic you
need to support, and the topology.

So again, IEEE 802.1CB doesn't tell you what to do, but it gives you the
tools. You can do source MAC based stream identification, and you can
emulate HSR, or you can do something that encompasses both source node
information as well as destination node information.

It's one whole degree of freedom more flexible, plain and simple.
And the topologies are not limited to:
- the rings that HSR supports
- the disjoint IP networks that PRP supports
but are rather generic graphs.

I fully expect there to be hardware out there already that can convert
between the HSR/PRP frame format on one set of ports to 802.1CB frame
format on another set of ports. Maybe that's something that some thought
needs to be put into.

And Xiaoliang, can you PLEASE remove the following email addresses from
further submissions you make:
andre.guedes@linux.intel.com
vishal@chelsio.com
ivan.khoronzhuk@linaro.org
m-karicheri2@ti.com
Arvid.Brodin@xdin.com

You also copied some of them on all 6 submissions for the PSFP offload series.
It gets really annoying to get email bounces from these addresses.
I've removed them from this email thread. Thanks.