Hi Andre,

On Thu, 9 Jan 2020 at 03:08, Andre Guedes [off-list ref] wrote:

Hi,

quoted

quoted

quoted

quoted

1. add support in taprio to be configured without any schedule in the
"full offload" mode. In practice, allowing taprio to work somewhat
similar to (mqprio + frame-preemption), changes in the code should de
fairly small;

+

And if follow mqprio settings logic then preemption also can be enabled
immediately while configuring taprio first time, and similarly new ADMIN
can't change it and can be set w/o preemption option afterwards.

So that following is correct:

OPER
$ tc qdisc add dev IFACE parent root handle 100 taprio \
       base-time 10000000 \
       num_tc 3 \
       map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
       queues 1@0 1@1 2@2 \
       preemption 0 1 1 1
       flags 1

then
ADMIN
$ tc qdisc add dev IFACE parent root handle 100 taprio \
       base-time 12000000 \
       num_tc 3 \
       map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
       queues 1@0 1@1 2@2 \
       preemption 0 1 1 1
       sched-entry S 01 300000 \
       sched-entry S 02 300000 \
       flags 1

then
ADMIN
$ tc qdisc add dev IFACE parent root handle 100 taprio \
       base-time 13000000 \
       sched-entry S 01 300000 \
       sched-entry S 02 300000 \
       flags 1

BUT:

1) The question is only should it be in this way? I mean preemption to be
enabled immediately? Also should include other parameters like
fragment size.

We can decide what things are allowed/useful here. For example, it might
make sense to allow "preemption" to be changed. We can extend taprio to
support changing the fragment size, if that makes sense.

quoted

2) What if I want to use frame preemption with another "transmission selection
algorithm"? Say another one "time sensitive" - CBS? How is it going to be
stacked?

I am not seeing any (conceptual*) problems when plugging a cbs (for
example) qdisc into one of taprio children. Or, are you talking about a
more general problem?

* here I am considering that support for taprio without an schedule is
  added.

quoted

In this case ethtool looks better, allowing this "MAC level" feature, to be
configured separately.

My only issue with using ethtool is that then we would have two
different interfaces for "complementary" features. And it would make
things even harder to configure and debug. The fact that one talks about
traffic classes and the other transmission queues doesn't make me more
comfortable as well.

On the other hand, as there isn't a way to implement frame preemption in
software, I agree that it makes it kind of awkward to have it in the tc
subsystem.

Absolutely. I think frame pre-emption feature flag, per queue express/
pre-empt state, frag size, timers (hold/release) to be configured
independently (perhaps through ethtool) and then taprio should check
this with the lower device and then allow supporting additional Gate
operations such as Hold/release if supported by underlying device.

What do you think? Why to abuse tc for this?

After reading all this great discussion and revisiting the 802.1Q and 802.3br
specs, I'm now leaning towards to not coupling Frame Preemption support under
taprio qdisc. Besides what have been discussed, Annex S.2 from 802.1Q-2018
foresees FP without EST so it makes me feel like we should keep them separate.

Regarding the FP configuration knobs, the following seems reasonable to me:
    * Enable/disable FP feature
    * Preemptable queue mapping
    * Fragment size multiplier

I'm not sure about the knob 'timers (hold/release)' described in the quotes
above. I couldn't find a match in the specs. If it refers to 'holdAdvance' and
'releaseAdvance' parameters described in 802.1Q-2018, I believe they are not
configurable. Do we know any hardware where they are configurable?

On NXP LS1028A, HOLD_ADVANCE is configurable on both ENETC and the
Felix switch (its default value is 127 bytes). Same as Synopsys, it is
a global setting and not per queue or per GCL entry.
RELEASE_ADVANCE is not configurable.
Regardless, I am not sure if there is any value in configuring this
knob. Remember that the minimum guard band size still needs to be
twice as large as the minimum Ethernet frame size.

As for the main topic of tc-taprio vs ethtool for configuring frame
preemption, I think ethtool is the more natural place for configuring
the traffic class to pMAC/eMAC mapping, global enable bit, and
fragment size, while tc-taprio is the more natural place for
configuring hold/release on individual GCL entries. The tc-taprio
offload can check the netdev support of the ethtool feature, just as
it checks right now the support for PTP clock for the full offload
feature.

Introducing tc-taprio with a null schedule is not really natural, and
frame preemption is a hardware-only feature that cannot be emulated,
so it is odd to enable it through tc.

Regards,

Andre

Regards,
-Vladimir