Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Is it okay to add such options to the ethtool ?
or will it be better to add a generic option to take pkt data offset
and number of bytes ?
Something like
ethtool -n <dev> rx-flow-hash tcp4 sdfn off <offset in the pkt> num
<number of bytes/bits>
Any comments, please.
Thanks,
Sunil.
From: Jakub Kicinski <kuba@kernel.org> Date: 2021-03-05 23:08:11
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Is it okay to add such options to the ethtool ?
or will it be better to add a generic option to take pkt data offset
and number of bytes ?
Something like
ethtool -n <dev> rx-flow-hash tcp4 sdfn off <offset in the pkt> num
<number of bytes/bits>
Any comments, please.
From: Andrew Lunn <andrew@lunn.ch> Date: 2021-03-06 00:19:24
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
Take a look at net/core/flow_dissect.c:__skb_flow_dissect()
This fits with the core ideas of the network stack and offloads. Hide
the fact an offload is being used, it should just look like a normal
interface.
Andrew
On Sat, Mar 6, 2021 at 4:37 AM Jakub Kicinski [off-list ref] wrote:
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
No, there is no driver for this tag in the kernel.
I have attached the tag format.
There are multiple DSA tag formats and representing them ethtool_flow
union would be difficult.
Hence wondering if it would be okay to add a more flexible way ie
offset and num_bytes from the start of packet.
Thanks,
Sunil.
On Sat, Mar 6, 2021 at 5:47 AM Andrew Lunn [off-list ref] wrote:
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
quoted
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
I understand your point.
The requirement to add DSA tag into RSS hashing is coming from one of
our customer.
Take a look at net/core/flow_dissect.c:__skb_flow_dissect()
This fits with the core ideas of the network stack and offloads. Hide
the fact an offload is being used, it should just look like a normal
interface.
Andrew
Yes, the pkt will look like a normal packet itself.
In our case HW strips the DSA tag from the packet and forwards it to a VF.
Thanks,
Sunil.
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-03-06 12:55:08
On Sat, Mar 06, 2021 at 05:38:14PM +0530, Sunil Kovvuri wrote:
quoted
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
No, there is no driver for this tag in the kernel.
I have attached the tag format.
There are multiple DSA tag formats and representing them ethtool_flow
union would be difficult.
Hence wondering if it would be okay to add a more flexible way ie
offset and num_bytes from the start of packet.
How sure are you that the tag format you've shared is not identical to
the one parsed by net/dsa/tag_dsa.c?
From: Vladimir Oltean <olteanv@gmail.com> Date: 2021-03-06 13:19:16
On Sat, Mar 06, 2021 at 06:04:14PM +0530, Sunil Kovvuri wrote:
On Sat, Mar 6, 2021 at 5:47 AM Andrew Lunn [off-list ref] wrote:
quoted
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
quoted
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
I understand your point.
The requirement to add DSA tag into RSS hashing is coming from one of
our customer.
quoted
Take a look at net/core/flow_dissect.c:__skb_flow_dissect()
This fits with the core ideas of the network stack and offloads. Hide
the fact an offload is being used, it should just look like a normal
interface.
Andrew
Yes, the pkt will look like a normal packet itself.
In our case HW strips the DSA tag from the packet and forwards it to a VF.
DSA-aware SR-IOV on master, very interesting. I expect that the reverse
is true as well: on xmit, the VF will automatically insert a FWD tag
into the packet too, which encodes a 'virtual' source port and switch id.
What Marvell controller is this? How is the SR-IOV implemented in the
kernel, is it modeled as switchdev, with host representors for the VFs?
Does it support VEPA mode or VEB too? If it supports VEB, does it learn
(more like 'steal') addresses from the DSA switch's FDB? Does it also
push addresses into the DSA switch's FDB?
To your question: why stop at hashing? What about flow steering?
What does your hardware actually support? It is obvious that it has deep
parsing of Marvell DSA tags specifically, so the generic 'masked u32 key'
matching may not be the best way to model this. Can your DSA master even
perform masked matching on generic u32 keys, or are you planning to just
look for the particular pattern of a Marvell DSA tag in the ethtool
rxnfc callbacks, and reject anything that doesn't match?
From: Andrew Lunn <andrew@lunn.ch> Date: 2021-03-06 14:33:15
On Sat, Mar 06, 2021 at 06:04:14PM +0530, Sunil Kovvuri wrote:
On Sat, Mar 6, 2021 at 5:47 AM Andrew Lunn [off-list ref] wrote:
quoted
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
quoted
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
I understand your point.
The requirement to add DSA tag into RSS hashing is coming from one of
our customer.
So what is the customer requirement? Why does the customer want to do
this? Please explain the real use case.
quoted
Take a look at net/core/flow_dissect.c:__skb_flow_dissect()
This fits with the core ideas of the network stack and offloads. Hide
the fact an offload is being used, it should just look like a normal
interface.
Andrew
Yes, the pkt will look like a normal packet itself.
In our case HW strips the DSA tag from the packet and forwards it to a VF.
So this flow hash will be used only inside the hardware? There is no
software equivalent of it? But the DSA header represents the ingress
port. So why cannot you identify the traffic based on netdev? Isn't
the -n dev equivalent to the DSA header?
Andrew
On Sat, Mar 06, 2021 at 14:54, Vladimir Oltean [off-list ref] wrote:
On Sat, Mar 06, 2021 at 05:38:14PM +0530, Sunil Kovvuri wrote:
quoted
quoted
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
No, there is no driver for this tag in the kernel.
I have attached the tag format.
There are multiple DSA tag formats and representing them ethtool_flow
union would be difficult.
Hence wondering if it would be okay to add a more flexible way ie
offset and num_bytes from the start of packet.
How sure are you that the tag format you've shared is not identical to
the one parsed by net/dsa/tag_dsa.c?
That is indeed the format parsed by tag_dsa.c. Based on the layout in
the image, I am pretty sure that it is from the functional spec. of the
Amethyst (6393X). So while the format is supported, that specific device
is not. Hopefully that will change soon:
https://lore.kernel.org/netdev/cover.1610071984.git.pavana.sharma@digi.com/
As for the NIC: Marvell has an EVK for the Amethyst, connected to a
CN9130 SoC. The ethernet controllers in those can parse DSA tags in
hardware, so I would put my money on that.
The upstream driver (mvpp2) does not seem to support it though, AFAIK.
On Sat, Mar 6, 2021 at 8:02 PM Andrew Lunn [off-list ref] wrote:
On Sat, Mar 06, 2021 at 06:04:14PM +0530, Sunil Kovvuri wrote:
quoted
On Sat, Mar 6, 2021 at 5:47 AM Andrew Lunn [off-list ref] wrote:
quoted
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
quoted
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
I understand your point.
The requirement to add DSA tag into RSS hashing is coming from one of
our customer.
So what is the customer requirement? Why does the customer want to do
this? Please explain the real use case.
Very sorry, after further discussions got to know from customer that
this is not needed.
They were observing some other issue and thought this would solve it.
Thanks,
Sunil.
On Sun, Mar 7, 2021 at 2:46 AM Tobias Waldekranz [off-list ref] wrote:
On Sat, Mar 06, 2021 at 14:54, Vladimir Oltean [off-list ref] wrote:
quoted
On Sat, Mar 06, 2021 at 05:38:14PM +0530, Sunil Kovvuri wrote:
quoted
quoted
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
No, there is no driver for this tag in the kernel.
I have attached the tag format.
There are multiple DSA tag formats and representing them ethtool_flow
union would be difficult.
Hence wondering if it would be okay to add a more flexible way ie
offset and num_bytes from the start of packet.
How sure are you that the tag format you've shared is not identical to
the one parsed by net/dsa/tag_dsa.c?
That is indeed the format parsed by tag_dsa.c. Based on the layout in
the image, I am pretty sure that it is from the functional spec. of the
Amethyst (6393X). So while the format is supported, that specific device
is not. Hopefully that will change soon:
https://lore.kernel.org/netdev/cover.1610071984.git.pavana.sharma@digi.com/
Thanks for the info.
As for the NIC: Marvell has an EVK for the Amethyst, connected to a
CN9130 SoC. The ethernet controllers in those can parse DSA tags in
hardware, so I would put my money on that.
The upstream driver (mvpp2) does not seem to support it though, AFAIK.
This is not for the mvpp2 driver, but for RVU driver
drivers/net/ethernet/marvell/octeontx2.
Thanks,
Sunil.
On Sat, Mar 6, 2021 at 6:48 PM Vladimir Oltean [off-list ref] wrote:
On Sat, Mar 06, 2021 at 06:04:14PM +0530, Sunil Kovvuri wrote:
quoted
On Sat, Mar 6, 2021 at 5:47 AM Andrew Lunn [off-list ref] wrote:
quoted
On Fri, Mar 05, 2021 at 03:07:02PM -0800, Jakub Kicinski wrote:
quoted
On Fri, 5 Mar 2021 16:15:51 +0530 Sunil Kovvuri wrote:
quoted
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Can you share the format of the DSA tag? Is there a driver for it
upstream? Do we need to represent it in union ethtool_flow_union?
Sorry, i missed the original question, there was no hint in the
subject line that DSA was involved.
Why do you want to include DSA tag in the hash? What normally happens
with DSA tag drivers is we detect the frame has been received from a
switch, and modify where the core flow dissect code looks in the frame
to skip over the DSA header and parse the IP header etc as normal.
I understand your point.
The requirement to add DSA tag into RSS hashing is coming from one of
our customer.
quoted
Take a look at net/core/flow_dissect.c:__skb_flow_dissect()
This fits with the core ideas of the network stack and offloads. Hide
the fact an offload is being used, it should just look like a normal
interface.
Andrew
Yes, the pkt will look like a normal packet itself.
In our case HW strips the DSA tag from the packet and forwards it to a VF.
DSA-aware SR-IOV on master, very interesting. I expect that the reverse
is true as well: on xmit, the VF will automatically insert a FWD tag
into the packet too, which encodes a 'virtual' source port and switch id.
Yes, HW inserts the tag on the transmit side automatically.
What Marvell controller is this? How is the SR-IOV implemented in the
kernel, is it modeled as switchdev, with host representors for the VFs?
This is Marvell OcteonTx2/CN10K RVU controller
drivers/net/ethernet/marvell/octeontx2.
And no the controller doesn't have a internal switch and hence
currently there is no switchdev support.
The switch I referred to is an external one whose CPU port is
connected to this controller.
Does it support VEPA mode or VEB too? If it supports VEB, does it learn
(more like 'steal') addresses from the DSA switch's FDB? Does it also
push addresses into the DSA switch's FDB?
To your question: why stop at hashing? What about flow steering?
What does your hardware actually support? It is obvious that it has deep
parsing of Marvell DSA tags specifically, so the generic 'masked u32 key'
matching may not be the best way to model this. Can your DSA master even
perform masked matching on generic u32 keys, or are you planning to just
look for the particular pattern of a Marvell DSA tag in the ethtool
rxnfc callbacks, and reject anything that doesn't match?
The parser in the HW is configurable and can parse any header with a mask.
Thanks,
Sunil.
On Fri, Mar 5, 2021 at 4:15 PM Sunil Kovvuri [off-list ref] wrote:
Hi,
We have a requirement where in we want RSS hashing to be done on packet fields
which are not currently supported by the ethtool.
Current options:
ehtool -n <dev> rx-flow-hash
tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r
Specifically our requirement is to calculate hash with DSA tag (which
is inserted by switch) plus the TCP/UDP 4-tuple as input.
Is it okay to add such options to the ethtool ?
or will it be better to add a generic option to take pkt data offset
and number of bytes ?
Something like
ethtool -n <dev> rx-flow-hash tcp4 sdfn off <offset in the pkt> num
<number of bytes/bits>
Any comments, please.
Thanks,
Sunil.
Apologies for wasting your time.
This hashing support is not needed anymore.
Thanks,
Sunil.
From: Andrew Lunn <andrew@lunn.ch> Date: 2021-03-08 13:43:07
This is Marvell OcteonTx2/CN10K RVU controller
drivers/net/ethernet/marvell/octeontx2.
And no the controller doesn't have a internal switch and hence
currently there is no switchdev support.
The switch I referred to is an external one whose CPU port is
connected to this controller.
Hi Sunil
That is a classic Linux DSA setup. I know it is no longer an issue,
but you might want to look at how switches are supported in Linux. I'm
sure there are people here who are interested in the setup, and having
the MAC do the DSA header work in hardware poses interesting questions
for the Linux DSA architecture. What would a tag driver in
net/dsa/tag_<FOO>.c look like, how do we tell the MAC about the switch
ports it needs to have slave interfaces for, etc.
Andrew