Hi,
I was pondering what it would take for a network device support
CHECKSUM_COMPLETE on its Rx path. And I simply failed to understand the
specifics when using the napi_gro_receive() interface.
Scenario 1 - TCP/IPv4 packet. This will eventually reach tcp4_gro_receive(), and from
there into skb_gro_checksum_validate().
Since this SKB is marked as COMPLETE, the pseudo TCP checksum will be computed
and skb_gro_checksum_validate_complete() be called.
Now, it seems to me as if the only way by which the flow won't actually try to
re-calculate the checksum over TCP header and payload is if NAPI_GRO_CB(skb)->csum
is actually the one's complement to the TCP pseudo header. But that sounds.... odd,
since that's the csum that should have been listed on the SKB - and it doesn't protect
any of the data, only the "metadata".
Scenario 2 - IP-GRE packet, with CSUM option on the GRE header.
This scenario will also eventually reach skb_gro_checksum_validate_complete(),
this time via gre_gro_receive().
This one is even odder, since in order for the flow not to re-calculate the checksum
The driver should have set the SKB's csum to 0xffffffff [as it's using the
null_compute_pseudo() which will return a pseudo_csum of 0].
From the comment in skbuff.h regarding CHECKSUM_COMPLETE it looks as if all
the HW needs to do is calculate the checksum over the entire data, starting at the
network header.
So, to summarize my questions -
1. What should a driver set as the SKBs csum value when passing CHECKSUM_COMPLETE?
2. Does the GRO flow for non-encapsulated packets work for packets marked as
CHECKSUM_COMPLETE? And by work, I obviously mean 'doesn't re-calculate checksums'.
3. Same as question 2, but for encapsulated packets.
Thanks,
Yuval
________________________________
This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.
From: Tom Herbert <hidden> Date: 2014-09-30 15:23:54
On Tue, Sep 30, 2014 at 5:51 AM, Yuval Mintz [off-list ref] wrote:
Hi,
I was pondering what it would take for a network device support
CHECKSUM_COMPLETE on its Rx path. And I simply failed to understand the
specifics when using the napi_gro_receive() interface.
Scenario 1 - TCP/IPv4 packet. This will eventually reach tcp4_gro_receive(), and from
there into skb_gro_checksum_validate().
Since this SKB is marked as COMPLETE, the pseudo TCP checksum will be computed
and skb_gro_checksum_validate_complete() be called.
Now, it seems to me as if the only way by which the flow won't actually try to
re-calculate the checksum over TCP header and payload is if NAPI_GRO_CB(skb)->csum
is actually the one's complement to the TCP pseudo header. But that sounds.... odd,
since that's the csum that should have been listed on the SKB - and it doesn't protect
any of the data, only the "metadata".
With CHECKSUM_COMPLETE, on entrance into GRO NAPI_GRO_CB(skb)->csum is
set to skb->csum. As the packet is parsed through various levels of
GRO the value is adjusted so that at any header level it is the
complete checksum starting from that header to the end of the packet.
So to validate a TCP checksum we just need to add the value of the
pseudo header checksum (a little computation) and compare to zero.
Scenario 2 - IP-GRE packet, with CSUM option on the GRE header.
This scenario will also eventually reach skb_gro_checksum_validate_complete(),
this time via gre_gro_receive().
This one is even odder, since in order for the flow not to re-calculate the checksum
The driver should have set the SKB's csum to 0xffffffff [as it's using the
null_compute_pseudo() which will return a pseudo_csum of 0].
From the comment in skbuff.h regarding CHECKSUM_COMPLETE it looks as if all
the HW needs to do is calculate the checksum over the entire data, starting at the
network header.
GRE is actually simpler since there is no pseudo header. If a GRE
checksum bit is set then skb->csum (NAPI_GRO_CB(skb)->csum) must be
zero for a valid checksum at the time GRE header is processed and we
have CHECKSUM_COMPLETE.
So, to summarize my questions -
1. What should a driver set as the SKBs csum value when passing CHECKSUM_COMPLETE?
This ones complete checksum of the Ethernet payload (start of IP
header to the end of the packet).
2. Does the GRO flow for non-encapsulated packets work for packets marked as
CHECKSUM_COMPLETE? And by work, I obviously mean 'doesn't re-calculate checksums'.
Yes.
3. Same as question 2, but for encapsulated packets.
Yes.
Hope this helps,
Tom
Thanks,
Yuval
________________________________
This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.
On Tue, Sep 30, 2014 at 7:59 AM, Tom Herbert [off-list ref] wrote:
quoted
So, to summarize my questions -
1. What should a driver set as the SKBs csum value when passing CHECKSUM_COMPLETE?
This ones complete checksum of the Ethernet payload (start of IP
header to the end of the packet).
I think it's confusing to describe CHECKSUM_COMPLETE this way.
It is such only because driver pulls eth header before passing skb to stack.
CHECKSUM_COMPLETE should cover the whole packet. If there are multiple
vlan headers in front of IP they should be part of csum, since not all
drivers may
have hw offloading for vlan. The most simplistic HW would compute csum over
the whole packet including eth header and driver will do
eth_type_trans+skb_postpull_rcsum
On Tue, Sep 30, 2014 at 7:59 AM, Tom Herbert [off-list ref] wrote:
quoted
quoted
So, to summarize my questions -
1. What should a driver set as the SKBs csum value when passing CHECKSUM_COMPLETE?
This ones complete checksum of the Ethernet payload (start of IP
header to the end of the packet).
I think it's confusing to describe CHECKSUM_COMPLETE this way.
It is such only because driver pulls eth header before passing skb to stack.
CHECKSUM_COMPLETE should cover the whole packet. If there are multiple
vlan headers in front of IP they should be part of csum, since not all
drivers may
have hw offloading for vlan. The most simplistic HW would compute csum over
the whole packet including eth header and driver will do
eth_type_trans+skb_postpull_rcsum
The sungem computes the checksum over the entire frame. Here are the relevant
parts from the programmer's manual:
RX Descriptor Fields
...
TCP Pseudo-Checksum - Contains the 16-bit TCP checksum calculated over
the entire frame, starting at an offset programmable in the RX
Configuration Register.
In the gengem driver this is the RXDMA_CFG_CSUMOFF field of the RXDMA_CFG
register.
The driver seems to set this to the size of the ethernet header, which has
the problems you mention.
Scenario 1 - TCP/IPv4 packet. This will eventually reach
tcp4_gro_receive(), and from there into skb_gro_checksum_validate().
Since this SKB is marked as COMPLETE, the pseudo TCP checksum will be
computed and skb_gro_checksum_validate_complete() be called.
Now, it seems to me as if the only way by which the flow won't
actually try to re-calculate the checksum over TCP header and payload
is if NAPI_GRO_CB(skb)->csum is actually the one's complement to the
TCP pseudo header. But that sounds.... odd, since that's the csum that
should have been listed on the SKB - and it doesn't protect any of the data,
only the "metadata".
quoted
With CHECKSUM_COMPLETE, on entrance into GRO NAPI_GRO_CB(skb)->csum
is set to skb->csum. As the packet is parsed through various levels of GRO the
value is adjusted so that at any header level it is the complete checksum starting
from that header to the end of the packet.
So to validate a TCP checksum we just need to add the value of the pseudo
header checksum (a little computation) and compare to zero.
...
quoted
1. What should a driver set as the SKBs csum value when passing CHECKSUM_COMPLETE?
This ones complete checksum of the Ethernet payload (start of IP header to the end of the packet).
Can you pinpoint me to what exactly in the code corrects NAPI_GRO_CB(skb)->csum
by negating the ipv4 part in the ones' complete checksum?
E.g., for ipv6 [in ipv6_gro_receive()] I can see that skb_gro_postpull_rcsum() is
called after pulling the ipv6 header [and before proceeding to the next protocol],
but I can't seem to find anything similar to this for ipv4 [in inet_gro_receive()].
________________________________
This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.
From: Eric Dumazet <hidden> Date: 2014-09-30 19:22:58
On Tue, 2014-09-30 at 19:05 +0000, Yuval Mintz wrote:
Can you pinpoint me to what exactly in the code corrects NAPI_GRO_CB(skb)->csum
by negating the ipv4 part in the ones' complete checksum?
E.g., for ipv6 [in ipv6_gro_receive()] I can see that skb_gro_postpull_rcsum() is
called after pulling the ipv6 header [and before proceeding to the next protocol],
but I can't seem to find anything similar to this for ipv4 [in inet_gro_receive()].
From: Eric Dumazet <hidden> Date: 2014-09-30 19:37:14
On Tue, 2014-09-30 at 12:22 -0700, Eric Dumazet wrote:
On Tue, 2014-09-30 at 19:05 +0000, Yuval Mintz wrote:
quoted
Can you pinpoint me to what exactly in the code corrects NAPI_GRO_CB(skb)->csum
by negating the ipv4 part in the ones' complete checksum?
E.g., for ipv6 [in ipv6_gro_receive()] I can see that skb_gro_postpull_rcsum() is
called after pulling the ipv6 header [and before proceeding to the next protocol],
but I can't seem to find anything similar to this for ipv4 [in inet_gro_receive()].
On Tue, 2014-09-30 at 12:22 -0700, Eric Dumazet wrote:
quoted
On Tue, 2014-09-30 at 19:05 +0000, Yuval Mintz wrote:
quoted
Can you pinpoint me to what exactly in the code corrects NAPI_GRO_CB(skb)->csum
by negating the ipv4 part in the ones' complete checksum?
E.g., for ipv6 [in ipv6_gro_receive()] I can see that skb_gro_postpull_rcsum() is
called after pulling the ipv6 header [and before proceeding to the next protocol],
but I can't seem to find anything similar to this for ipv4 [in inet_gro_receive()].
Can you pinpoint me to what exactly in the code corrects NAPI_GRO_CB(skb)->csum
by negating the ipv4 part in the ones' complete checksum?
E.g., for ipv6 [in ipv6_gro_receive()] I can see that skb_gro_postpull_rcsum() is
called after pulling the ipv6 header [and before proceeding to the next protocol],
but I can't seem to find anything similar to this for ipv4 [in inet_gro_receive()].
IPv4 header is supposed to have a 0 checksum ;)
Oops. Sorry, you're right.
But this is a bit strange.
I mean, if I go back to the IP-GRE case [with GRE checksum], the IP
csum negates the rest of the IP header and the GRE csum negates
everything else [GRE header, inner headers and payload].
So the mechanism might work as-is, but all the driver has to
do in order to claim that an IP-gre packet with checksum option is
CHECKSUM_COMPLETE is to write 0xffffffff on the skb's csum field.
I mean, if I go back to the IP-GRE case [with GRE checksum], the IP
csum negates the rest of the IP header and the GRE csum negates
everything else [GRE header, inner headers and payload].
So the mechanism might work as-is, but all the driver has to
do in order to claim that an IP-gre packet with checksum option is
CHECKSUM_COMPLETE is to write 0xffffffff on the skb's csum field.
Or looking back on the IPv4/TCP scenario, what exactly is the csum
now protecting?
Assume the HW passes the one's complement sum of the entire data,
IP header and onward to driver, which sets it in SKB->csum and marks
the SKB as CHECKSUM_COMPLETE.
In practice, SKB->csum should now equal to the one's complement of
the TCP pseudo header; But other than the length of data in the TCP
header and payload that pseudo-header doesn't include any
information about the payload itself.
If we're willing to accept the payload [and the correctness of the
header while we're at it] based solely on this, what guarantee do we
have that the TCP checksum is actually correct?
Seems like the only thing we've guaranteed is that nothing switched in
IP header since packet passed the HW - but we've already got that
covered by the IPv4 layer.
[I assume I'm missing something elementary here. Same as before ;-)]
From: Eric Dumazet <hidden> Date: 2014-10-01 05:12:05
From: Eric Dumazet <redacted>
Proper CHECKSUM_COMPLETE support needs to adjust skb->csum
when we remove one header. Its done using skb_gro_postpull_rcsum()
In the case of IPv4, we know that the adjustment is not really needed,
because the checksum over IPv4 header is 0. Lets add a comment to
ease code comprehension and avoid copy/paste errors.
Signed-off-by: Eric Dumazet <redacted>
---
net/ipv4/af_inet.c | 3 +++
1 file changed, 3 insertions(+)
From: Tom Herbert <hidden> Date: 2014-10-01 05:15:57
On Tue, Sep 30, 2014 at 1:32 PM, Yuval Mintz [off-list ref] wrote:
quoted
quoted
IPv4 header is supposed to have a 0 checksum ;)
quoted
Oops. Sorry, you're right.
quoted
But this is a bit strange.
quoted
I mean, if I go back to the IP-GRE case [with GRE checksum], the IP
csum negates the rest of the IP header and the GRE csum negates
everything else [GRE header, inner headers and payload].
So the mechanism might work as-is, but all the driver has to
do in order to claim that an IP-gre packet with checksum option is
CHECKSUM_COMPLETE is to write 0xffffffff on the skb's csum field.
Or looking back on the IPv4/TCP scenario, what exactly is the csum
now protecting?
Assume the HW passes the one's complement sum of the entire data,
IP header and onward to driver, which sets it in SKB->csum and marks
the SKB as CHECKSUM_COMPLETE.
In practice, SKB->csum should now equal to the one's complement of
the TCP pseudo header; But other than the length of data in the TCP
header and payload that pseudo-header doesn't include any
information about the payload itself.
But it does, as you pointed out the checksum calculation was performed
over the whole payload. The fact that the computed checksum equals the
one's complement of pseudo header is the indication that the checksum
is valid. If there were a bit corruption, the computed value would be
different (summing in pseudo header checksum would result in non-zero
value).
If we're willing to accept the payload [and the correctness of the
header while we're at it] based solely on this, what guarantee do we
have that the TCP checksum is actually correct?
Seems like the only thing we've guaranteed is that nothing switched in
IP header since packet passed the HW - but we've already got that
covered by the IPv4 layer.
[I assume I'm missing something elementary here. Same as before ;-)]
I think your assumption is correct ;-) You might want to take a look
at some of the RFCs that describe checksum verification with pseudo
header.
Or looking back on the IPv4/TCP scenario, what exactly is the csum now
protecting?
Assume the HW passes the one's complement sum of the entire data, IP
header and onward to driver, which sets it in SKB->csum and marks the
SKB as CHECKSUM_COMPLETE.
In practice, SKB->csum should now equal to the one's complement of the
TCP pseudo header; But other than the length of data in the TCP header
and payload that pseudo-header doesn't include any information about
the payload itself.
But it does, as you pointed out the checksum calculation was performed over
the whole payload. The fact that the computed checksum equals the one's
complement of pseudo header is the indication that the checksum is valid. If
there were a bit corruption, the computed value would be different (summing in
pseudo header checksum would result in non-zero value).
Thanks, I've already reached that conclusion myself
[even wrote an E-mail; but it got blocked for some reason].
From: David Miller <davem@davemloft.net> Date: 2014-10-01 05:26:00
Yuval, the base64 encoding of your emails is not correct and many
sites are rejecting your postings that are actually making it to the
list based upon this fact.
Please post pure ASCII text to this mailing list unless that's
unavoidable (a UTF character in someone's name during a patch
submission f.e.).
From: Eric Dumazet <redacted>
Proper CHECKSUM_COMPLETE support needs to adjust skb->csum
when we remove one header. Its done using skb_gro_postpull_rcsum()
In the case of IPv4, we know that the adjustment is not really needed,
because the checksum over IPv4 header is 0. Lets add a comment to
ease code comprehension and avoid copy/paste errors.
Signed-off-by: Eric Dumazet <redacted>