Hi all,
in our community mesh network we recently discovered that a TP-Link
Archer C2600 device is unable to receive IPv6 UDP packets with a zero
checksum when RX checksum offloading is enabled. The device uses
ipq806x-gmac-dwmac for its ethernet ports.
According to https://tools.ietf.org/html/rfc2460#section-8.1 this sounds
like correct behavior as it says a UDP checksum must not be zero for
IPv6 packets. But this definition was relaxed in
https://tools.ietf.org/html/rfc6935#section-5 to allow zero checksums in
tunneling protocols like VXLAN where we discovered the problem.
Can the behavior of the stmmac driver be changed to meet RFC6935 or
would it be possible to make the (RX) Checksum Offloading Engine
configurable via a device tree property to disable it in environments
were it causes problems?
Best regards,
Julian Labus
From: Andrew Lunn <andrew@lunn.ch> Date: 2021-04-05 13:27:35
On Mon, Apr 05, 2021 at 02:53:15PM +0200, Julian Labus wrote:
Hi all,
in our community mesh network we recently discovered that a TP-Link Archer
C2600 device is unable to receive IPv6 UDP packets with a zero checksum when
RX checksum offloading is enabled. The device uses ipq806x-gmac-dwmac for
its ethernet ports.
According to https://tools.ietf.org/html/rfc2460#section-8.1 this sounds
like correct behavior as it says a UDP checksum must not be zero for IPv6
packets. But this definition was relaxed in
https://tools.ietf.org/html/rfc6935#section-5 to allow zero checksums in
tunneling protocols like VXLAN where we discovered the problem.
Can the behavior of the stmmac driver be changed to meet RFC6935 or would it
be possible to make the (RX) Checksum Offloading Engine configurable via a
device tree property to disable it in environments were it causes problems?
Hi Julian
I don't know the stmmac driver at all...
Have you played around with ethtool -k/-K? Can use this to turn off
hardware checksums?
I doubt a DT property would be accepted. What you probably want to do
is react on the NETDEV notifiers for when an upper interface is
changed. If a VXLAN interface is added, turn off hardware checksums.
Andrew
On Mon, Apr 05, 2021 at 02:53:15PM +0200, Julian Labus wrote:
quoted
Hi all,
in our community mesh network we recently discovered that a TP-Link Archer
C2600 device is unable to receive IPv6 UDP packets with a zero checksum when
RX checksum offloading is enabled. The device uses ipq806x-gmac-dwmac for
its ethernet ports.
According to https://tools.ietf.org/html/rfc2460#section-8.1 this sounds
like correct behavior as it says a UDP checksum must not be zero for IPv6
packets. But this definition was relaxed in
https://tools.ietf.org/html/rfc6935#section-5 to allow zero checksums in
tunneling protocols like VXLAN where we discovered the problem.
Can the behavior of the stmmac driver be changed to meet RFC6935 or would it
be possible to make the (RX) Checksum Offloading Engine configurable via a
device tree property to disable it in environments were it causes problems?
Hi Julian
I don't know the stmmac driver at all...
Have you played around with ethtool -k/-K? Can use this to turn off
hardware checksums?
I doubt a DT property would be accepted. What you probably want to do
is react on the NETDEV notifiers for when an upper interface is
changed. If a VXLAN interface is added, turn off hardware checksums.
Andrew
Hi Andrew,
yes, disabling the offloading via "ethtool -K <ifname> rx off" works and
is used as a workaround in a startup script but the OpenWrt-based OS on
the mentioned device does not provide a reliable way to trigger ethtool
commands when network devices change.
Reacting to upper interface changes and disabling (rx) offloading per
interface sounds right when looking at RFC6935 again.
"As an alternative, certain protocols that use UDP as a tunnel
encapsulation MAY enable zero-checksum mode for a specific port
(or set of ports) for sending and/or receiving."
But was is still a bit strange to me is that it seems like the stmmac
driver behaves different than other ethernet drivers which do not drop
UDP packets with zero checksums when rx checksumming is enabled.
Julian
From: Andrew Lunn <andrew@lunn.ch> Date: 2021-04-05 16:42:56
But was is still a bit strange to me is that it seems like the stmmac driver
behaves different than other ethernet drivers which do not drop UDP packets
with zero checksums when rx checksumming is enabled.
To answer that, you need somebody with more knowledge of the stmmac
hardware. It is actually quite hard to do. It means you need to parse
more of the frame to determine if the frame contains a VXLAN
encapsulated frame. Probably the stmmac cannot do that. It sees the
checksum is wrong and drops the packet.
Have you looked at where it actually drops the packet?
Is it one of
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/norm_desc.c#L95
or
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/enh_desc.c#L87
It could be, you need to see if the checksum has fail, then check if
the checksum is actually zero, and then go deeper into the frame and
check if it is a vxlan frame. It could be the linux software checksum
code knows about this vxlan exception, so you can just run that before
deciding to drop the frame.
Andrew
No, I've not yet checked where the packet is actually dropped. This is
my first time debugging a network problem in the kernel and I have
little to no knowledge how this rx offloading works. But I don't think
it happens in norm_desc.c#L95 as I would expect the ipc_csum_error
counter to increase. But the only counters in "ethtool -S" which are
increasing are mmc_rx_udp_err and mmc_rx_udp_err_octets. I'll try to get
more information.
Julian
From: Jakub Kicinski <kuba@kernel.org> Date: 2021-04-05 23:03:44
On Mon, 5 Apr 2021 18:42:53 +0200 Andrew Lunn wrote:
quoted
But was is still a bit strange to me is that it seems like the stmmac driver
behaves different than other ethernet drivers which do not drop UDP packets
with zero checksums when rx checksumming is enabled.
To answer that, you need somebody with more knowledge of the stmmac
hardware.
+1 stmmac maintainers could you advise?
It is actually quite hard to do. It means you need to parse
more of the frame to determine if the frame contains a VXLAN
encapsulated frame. Probably the stmmac cannot do that. It sees the
checksum is wrong and drops the packet.
Have you looked at where it actually drops the packet?
Is it one of
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/norm_desc.c#L95
or
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/enh_desc.c#L87
It could be, you need to see if the checksum has fail, then check if
the checksum is actually zero, and then go deeper into the frame and
check if it is a vxlan frame. It could be the linux software checksum
code knows about this vxlan exception, so you can just run that before
deciding to drop the frame.
To be clear the expectation is that devices / drivers will only drop
packets on L2 / FCS errors. If L3 or L4 csum is incorrect the packet
should be passed up the stack and kernel will handle it how it sees fit.
Hi stmmac maintainers,
it's around 1 1/2 months without any response on this topic.
Could you please advice how to proceed with this problem?
Kind regards,
Julian
On 06.04.21 01:03, Jakub Kicinski wrote:
On Mon, 5 Apr 2021 18:42:53 +0200 Andrew Lunn wrote:
quoted
quoted
But was is still a bit strange to me is that it seems like the stmmac driver
behaves different than other ethernet drivers which do not drop UDP packets
with zero checksums when rx checksumming is enabled.
To answer that, you need somebody with more knowledge of the stmmac
hardware.
+1 stmmac maintainers could you advise?
quoted
It is actually quite hard to do. It means you need to parse
more of the frame to determine if the frame contains a VXLAN
encapsulated frame. Probably the stmmac cannot do that. It sees the
checksum is wrong and drops the packet.
Have you looked at where it actually drops the packet?
Is it one of
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/norm_desc.c#L95
or
https://elixir.bootlin.com/linux/latest/source/drivers/net/ethernet/stmicro/stmmac/enh_desc.c#L87
It could be, you need to see if the checksum has fail, then check if
the checksum is actually zero, and then go deeper into the frame and
check if it is a vxlan frame. It could be the linux software checksum
code knows about this vxlan exception, so you can just run that before
deciding to drop the frame.
To be clear the expectation is that devices / drivers will only drop
packets on L2 / FCS errors. If L3 or L4 csum is incorrect the packet
should be passed up the stack and kernel will handle it how it sees fit.