Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

7 messages, 4 authors, 2011-01-21 · open the first message on its own page

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Jesse Gross <hidden>
Date: 2011-01-18 20:06:55

On Tue, Jan 18, 2011 at 11:52 AM, Hans de Bruin [off-list ref] wrote:
On 01/16/2011 09:24 PM, Hans de Bruin wrote:
quoted
After last nights compile i lost the possibility to connect to ssh and
http over ipv6. The connection stops at syn_sent. connections to my
machine end in syn_recv. ping6 still works.
The bisect ended in:

0363466866d901fbc658f4e63dd61e7cc93dd0af is the first bad commit
commit 0363466866d901fbc658f4e63dd61e7cc93dd0af
Author: Jesse Gross [off-list ref]
Date:   Sun Jan 9 06:23:35 2011 +0000

   net offloading: Convert checksums to use centrally computed features.

   In order to compute the features for other offloads (primarily
   scatter/gather), we need to first check the ability of the NIC to
   offload the checksum for the packet.  Since we have already computed
   this, we can directly use the result instead of figuring it out
   again.

   Signed-off-by: Jesse Gross [off-list ref]
   Signed-off-by: David S. Miller [off-list ref]


ssh ::1  still works. And since dns still works I guess udp is not affected.
My nic is a:

09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752 Gigabit
Ethernet PCI Express (rev 02)
Are you using vlans?  If so, can you please test this patch?
http://patchwork.ozlabs.org/patch/79264/

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Hans de Bruin <hidden>
Date: 2011-01-18 21:41:10

On 01/18/2011 09:06 PM, Jesse Gross wrote:
On Tue, Jan 18, 2011 at 11:52 AM, Hans de Bruin[off-list ref]  wrote:
quoted
On 01/16/2011 09:24 PM, Hans de Bruin wrote:
quoted
After last nights compile i lost the possibility to connect to ssh and
http over ipv6. The connection stops at syn_sent. connections to my
machine end in syn_recv. ping6 still works.
The bisect ended in:

0363466866d901fbc658f4e63dd61e7cc93dd0af is the first bad commit
commit 0363466866d901fbc658f4e63dd61e7cc93dd0af
Author: Jesse Gross[off-list ref]
Date:   Sun Jan 9 06:23:35 2011 +0000

    net offloading: Convert checksums to use centrally computed features.

    In order to compute the features for other offloads (primarily
    scatter/gather), we need to first check the ability of the NIC to
    offload the checksum for the packet.  Since we have already computed
    this, we can directly use the result instead of figuring it out
    again.

    Signed-off-by: Jesse Gross[off-list ref]
    Signed-off-by: David S. Miller[off-list ref]


ssh ::1  still works. And since dns still works I guess udp is not affected.
My nic is a:

09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752 Gigabit
Ethernet PCI Express (rev 02)
Are you using vlans?  If so, can you please test this patch?
http://patchwork.ozlabs.org/patch/79264/
No I am not using vlans. The option is even not set in the config file. Except for the disk less bit is a straightforward setup:

  bash-4.1# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:1c:23:2d:73:87 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.6/16 brd 10.10.255.255 scope global eth0
    inet6 2001:610:76e:0:21c:23ff:fe2d:7387/64 scope global dynamic
       valid_lft 86399sec preferred_lft 14399sec
    inet6 fe80::21c:23ff:fe2d:7387/64 scope link
       valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop state DOWN
    link/sit 0.0.0.0 brd 0.0.0.0
bash-4.1# ip route show
10.10.0.0/16 dev eth0  proto kernel  scope link  src 10.10.0.6
default via 10.10.0.1 dev eth0
bash-4.1# ip -6 route show
2001:610:76e::/64 dev eth0  proto kernel  metric 256  expires 86404sec
fe80::/64 dev eth0  proto kernel  metric 256
ff00::/8 dev eth0  metric 256
default via fe80::230:18ff:feae:75d8 dev eth0  proto kernel  metric 1024  expires 29sec hoplimit 64
bash-4.1# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
bash-4.1# ip6tables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
bash-4.1# brctl show
bridge name     bridge id               STP enabled     interfaces
bash-4.1# cat /proc/cmdline
root=/dev/nfs nfsroot=10.10.0.2:/nfs/root/psion/,v3,tcp ro ip=::: ::dhcp BOOT_IMAGE=nightlybuild-psion

-- 
Hans

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Eric Dumazet <hidden>
Date: 2011-01-18 22:04:07

Le mardi 18 janvier 2011 à 22:42 +0100, Hans de Bruin a écrit :
On 01/18/2011 09:06 PM, Jesse Gross wrote:
quoted
On Tue, Jan 18, 2011 at 11:52 AM, Hans de Bruin[off-list ref]  wrote:
quoted
On 01/16/2011 09:24 PM, Hans de Bruin wrote:
quoted
After last nights compile i lost the possibility to connect to ssh and
http over ipv6. The connection stops at syn_sent. connections to my
machine end in syn_recv. ping6 still works.
The bisect ended in:

0363466866d901fbc658f4e63dd61e7cc93dd0af is the first bad commit
commit 0363466866d901fbc658f4e63dd61e7cc93dd0af
Author: Jesse Gross[off-list ref]
Date:   Sun Jan 9 06:23:35 2011 +0000

    net offloading: Convert checksums to use centrally computed features.

    In order to compute the features for other offloads (primarily
    scatter/gather), we need to first check the ability of the NIC to
    offload the checksum for the packet.  Since we have already computed
    this, we can directly use the result instead of figuring it out
    again.

    Signed-off-by: Jesse Gross[off-list ref]
    Signed-off-by: David S. Miller[off-list ref]


ssh ::1  still works. And since dns still works I guess udp is not affected.
My nic is a:

09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752 Gigabit
Ethernet PCI Express (rev 02)
Are you using vlans?  If so, can you please test this patch?
http://patchwork.ozlabs.org/patch/79264/
No I am not using vlans. The option is even not set in the config file. 
Except for the disk less bit is a straightforward setup:

   bash-4.1# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1/8 scope host lo
     inet6 ::1/128 scope host
        valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP 
qlen 1000
     link/ether 00:1c:23:2d:73:87 brd ff:ff:ff:ff:ff:ff
     inet 10.10.0.6/16 brd 10.10.255.255 scope global eth0
     inet6 2001:610:76e:0:21c:23ff:fe2d:7387/64 scope global dynamic
        valid_lft 86399sec preferred_lft 14399sec
     inet6 fe80::21c:23ff:fe2d:7387/64 scope link
        valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop state DOWN
     link/sit 0.0.0.0 brd 0.0.0.0
bash-4.1# ip route show
10.10.0.0/16 dev eth0  proto kernel  scope link  src 10.10.0.6
default via 10.10.0.1 dev eth0
bash-4.1# ip -6 route show
2001:610:76e::/64 dev eth0  proto kernel  metric 256  expires 86404sec
fe80::/64 dev eth0  proto kernel  metric 256
ff00::/8 dev eth0  metric 256
default via fe80::230:18ff:feae:75d8 dev eth0  proto kernel  metric 1024 
  expires 29sec hoplimit 64
bash-4.1# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
bash-4.1# ip6tables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
bash-4.1# brctl show
bridge name     bridge id               STP enabled     interfaces
bash-4.1# cat /proc/cmdline
root=/dev/nfs nfsroot=10.10.0.2:/nfs/root/psion/,v3,tcp ro ip=::: ::dhcp 
BOOT_IMAGE=nightlybuild-psion

You could try "tcpdump -i eth0 ip6 -v"

I guess you receive frames with bad checksums

You can ask other system to not offload tx checksums

ethtool -K eth0 tx off

Please give result of (on both machines)
lspci -v

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Hans de Bruin <hidden>
Date: 2011-01-21 19:46:58

On 01/18/2011 11:03 PM, Eric Dumazet wrote:
Le mardi 18 janvier 2011 à 22:42 +0100, Hans de Bruin a écrit :
quoted
On 01/18/2011 09:06 PM, Jesse Gross wrote:
...
You could try "tcpdump -i eth0 ip6 -v"

I guess you receive frames with bad checksums
While you where staring at the code, I was fooling around with tcpdump. And while the problem is fixed, I still have some questions:

Is there tool which shows whether a nic supports ipv6 checksum offload or not?

I have captured http traffic (wget http://bootes/) between psion (my git tree following laptop) and bootes (something running 2.6.33.7).
Attached is a capture with psion running 2.6.37 and one with this morning's git tree. Wat's with the 'chsum ... ( incorrect -> ' lines ?
ifconfig does not show errors on either of the machines.

-- 
Hans

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Eric Dumazet <hidden>
Date: 2011-01-21 19:55:50

Le vendredi 21 janvier 2011 à 20:47 +0100, Hans de Bruin a écrit :
On 01/18/2011 11:03 PM, Eric Dumazet wrote:
quoted
Le mardi 18 janvier 2011 à 22:42 +0100, Hans de Bruin a écrit :
quoted
On 01/18/2011 09:06 PM, Jesse Gross wrote:
...
quoted
You could try "tcpdump -i eth0 ip6 -v"

I guess you receive frames with bad checksums
While you where staring at the code, I was fooling around with tcpdump. 
And while the problem is fixed, I still have some questions:

Is there tool which shows whether a nic supports ipv6 checksum offload 
or not?

I have captured http traffic (wget http://bootes/) between psion (my git 
tree following laptop) and bootes (something running 2.6.33.7).
Attached is a capture with psion running 2.6.37 and one with this 
morning's git tree. Wat's with the 'chsum ... ( incorrect -> ' lines ?
ifconfig does not show errors on either of the machines.
tcpdump gets a copy of outgoing frames before NIC performs tx checksum
(if tx checksum handled by NIC), so it's normal to have "bad checksums"
on TX, unless you disable tx offloading (ethtool -K eth0 tx off)

I was referring to check with tcpdump incoming frames, because invalid
checksums in RX is sign that other peer sent wrong checksums


Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Hans de Bruin <hidden>
Date: 2011-01-21 20:37:49

On 01/21/2011 08:55 PM, Eric Dumazet wrote:
Le vendredi 21 janvier 2011 à 20:47 +0100, Hans de Bruin a écrit :
quoted
On 01/18/2011 11:03 PM, Eric Dumazet wrote:
quoted
Le mardi 18 janvier 2011 à 22:42 +0100, Hans de Bruin a écrit :
quoted
On 01/18/2011 09:06 PM, Jesse Gross wrote:
...
quoted
You could try "tcpdump -i eth0 ip6 -v"

I guess you receive frames with bad checksums
While you where staring at the code, I was fooling around with tcpdump.
And while the problem is fixed, I still have some questions:

Is there tool which shows whether a nic supports ipv6 checksum offload
or not?

I have captured http traffic (wget http://bootes/) between psion (my git
tree following laptop) and bootes (something running 2.6.33.7).
Attached is a capture with psion running 2.6.37 and one with this
morning's git tree. Wat's with the 'chsum ... ( incorrect ->  ' lines ?
ifconfig does not show errors on either of the machines.
tcpdump gets a copy of outgoing frames before NIC performs tx checksum
(if tx checksum handled by NIC), so it's normal to have "bad checksums"
on TX, unless you disable tx offloading (ethtool -K eth0 tx off)
That seem reasonable but: the bug was triggered because my nic could not offload checksumming, so what's tx=on if there's no support for it? I have turned tx off and my tcpdump still shows bad checksums on outgoing tcp/ip6 packets. I have tried 2.6.36: bad checksums, 2.6.35 and surprise: good checksums with tx=on.
I was referring to check with tcpdump incoming frames, because invalid
checksums in RX is sign that other peer sent wrong checksums
Ok, thats clear, the receiving site is apparently a more reliable checksummer than the sending site.

-- 
Hans

Re: [regression] 2.6.37+ commit 0363466866d9.... breaks tcp ipv6

From: Brandeburg, Jesse <hidden>
Date: 2011-01-21 22:15:50


On Fri, 21 Jan 2011, Hans de Bruin wrote:
quoted
tcpdump gets a copy of outgoing frames before NIC performs tx checksum
(if tx checksum handled by NIC), so it's normal to have "bad checksums"
on TX, unless you disable tx offloading (ethtool -K eth0 tx off)
That seem reasonable but: the bug was triggered because my nic could not 
offload checksumming, so what's tx=on if there's no support for it? I 
have turned tx off and my tcpdump still shows bad checksums on outgoing 
tcp/ip6 packets. I have tried 2.6.36: bad checksums, 2.6.35 and 
surprise: good checksums with tx=on.
maybe you're still seeing confusing output on tx due to GSO, try turning 
that off with ethtool -K ethX gso off

GSO still software-segments packets, and maybe the tap for tcpdump is 
before GSO operation.
quoted
I was referring to check with tcpdump incoming frames, because invalid
checksums in RX is sign that other peer sent wrong checksums
Ok, thats clear, the receiving site is apparently a more reliable 
checksummer than the sending site.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help