From: Ruslan N. Marchenko <hidden> Date: 2011-08-30 12:57:27
Hi guys,
I've faced with strange behaviour of 8021q driver: when enabling vlan subinterface on eth interface I'm getting ~50% packetloss due to packets are marked with incorrect tags (and eventually dropped by kernel since no vlans configured for such IDs).
Scenario:
[ 0.476950] cpufreq-nforce2: No nForce2 chipset.
[ 1.519133] forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
[ 1.519991] forcedeth 0000:00:0a.0: PCI INT A -> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22
[ 1.520037] forcedeth 0000:00:0a.0: setting latency timer to 64
[ 1.586526] forcedeth 0000:00:0a.0: ifname eth0, PHY OUI 0x732 @ 3, addr 00:26:18:40:21:61
[ 1.586542] forcedeth 0000:00:0a.0: highdma csum pwrctl gbit lnktim msi desc-v3
modprobe 8021q
- network still works properly, packets are comming not marked at all.
ip li add link eth0 name vl6 type vlan id 6
- from this moment massive packetdrop starting to happen, almost half of the *incoming* packets are shown in tcpdump as
14:15:52.859296 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 64, p 3, ethertype IPv4, [|ip]
14:15:56.869572 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 2112, p 7, ethertype IPv4, [|ip]
mostly only these two tags appears (64 & 2112). Moreover this happens as on native vlan level (pure ethernet) so on tagged subinterface (as if qinq double tagging) for properly tagged with ID 6 incomming packets.
I've tried disabling all offloads:
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off
- doesn't have any effect.
Once executing
ip li del vl6 type vlan
misterious tags disappear and everything works smoothly. Don't know who injects that garbage into frames - 8021q or forcedeth driver :(
Any ideas or suggestions to narrow the problem down?
Additional data.
Link level data dump example for broken frame:
12:35:32.175523 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 2112, p 2, ethertype IPv4, [|ip]
0x0000: 0026 1840 2161 0013 f71e fee4 8100 4840
0x0010: 0800 4500 0054 7a12 0000 4001 eb0f
0x0C-0D - TPID: ethertype 802.1Q (0x8100)
0x0E-0F - TCI (0100100001000000) PCP 010, CFI 0, VID 100001000000/0x840/2112
0x10-11 - ethertype IPv4
normal ping reply follows, which appears untagged in 50% cases with vlan configured and 100% cases without.
Interface is plugged into openwrt box into non-switched (wan) gigabit port with vid 6 subinterface configured.
Regards,
Ruslan
From: Eric Dumazet <hidden> Date: 2011-08-30 13:23:46
Le mardi 30 août 2011 à 14:51 +0200, Ruslan N. Marchenko a écrit :
Hi guys,
I've faced with strange behaviour of 8021q driver: when enabling vlan subinterface on eth interface I'm getting ~50% packetloss due to packets are marked with incorrect tags (and eventually dropped by kernel since no vlans configured for such IDs).
Scenario:
[ 0.476950] cpufreq-nforce2: No nForce2 chipset.
[ 1.519133] forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
[ 1.519991] forcedeth 0000:00:0a.0: PCI INT A -> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22
[ 1.520037] forcedeth 0000:00:0a.0: setting latency timer to 64
[ 1.586526] forcedeth 0000:00:0a.0: ifname eth0, PHY OUI 0x732 @ 3, addr 00:26:18:40:21:61
[ 1.586542] forcedeth 0000:00:0a.0: highdma csum pwrctl gbit lnktim msi desc-v3
modprobe 8021q
- network still works properly, packets are comming not marked at all.
ip li add link eth0 name vl6 type vlan id 6
- from this moment massive packetdrop starting to happen, almost half of the *incoming* packets are shown in tcpdump as
14:15:52.859296 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 64, p 3, ethertype IPv4, [|ip]
14:15:56.869572 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 2112, p 7, ethertype IPv4, [|ip]
mostly only these two tags appears (64 & 2112). Moreover this happens as on native vlan level (pure ethernet) so on tagged subinterface (as if qinq double tagging) for properly tagged with ID 6 incomming packets.
I've tried disabling all offloads:
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off
- doesn't have any effect.
Once executing
ip li del vl6 type vlan
misterious tags disappear and everything works smoothly. Don't know who injects that garbage into frames - 8021q or forcedeth driver :(
Any ideas or suggestions to narrow the problem down?
Additional data.
Link level data dump example for broken frame:
12:35:32.175523 00:13:f7:1e:fe:e4 > 00:26:18:40:21:61, ethertype 802.1Q (0x8100), length 102: vlan 2112, p 2, ethertype IPv4, [|ip]
0x0000: 0026 1840 2161 0013 f71e fee4 8100 4840
0x0010: 0800 4500 0054 7a12 0000 4001 eb0f
0x0C-0D - TPID: ethertype 802.1Q (0x8100)
0x0E-0F - TCI (0100100001000000) PCP 010, CFI 0, VID 100001000000/0x840/2112
0x10-11 - ethertype IPv4
normal ping reply follows, which appears untagged in 50% cases with vlan configured and 100% cases without.
Interface is plugged into openwrt box into non-switched (wan) gigabit port with vid 6 subinterface configured.
From: Ruslan N. Marchenko <hidden> Date: 2011-08-30 13:46:27
On Tue, Aug 30, 2011 at 03:23:48PM +0200, Eric Dumazet wrote:
What kernel version are you using ?
Oh, sorry for missing it, it runs on
Linux ruff.mobi 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:05:14 UTC 2011 i686 i686 i386 GNU/Linux
Just fyi - the openwrt box to which it is connected is
Linux OpenWrt 2.6.39.2 #2 Fri Aug 12 09:36:23 EEST 2011 mips GNU/Linux
although packet drop happens even if there're no vlans configured on remote side.
Regards,
Ruslan
From: Ruslan N. Marchenko <hidden> Date: 2011-08-30 14:23:12
On Tue, Aug 30, 2011 at 03:46:24PM +0200, Ruslan N. Marchenko wrote:
On Tue, Aug 30, 2011 at 03:23:48PM +0200, Eric Dumazet wrote:
quoted
What kernel version are you using ?
Oh, sorry for missing it, it runs on
Linux ruff.mobi 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:05:14 UTC 2011 i686 i686 i386 GNU/Linux
Just fyi - the openwrt box to which it is connected is
Linux OpenWrt 2.6.39.2 #2 Fri Aug 12 09:36:23 EEST 2011 mips GNU/Linux
although packet drop happens even if there're no vlans configured on remote side.
Here is double-tag sample:
16:20:31.151268 e0:46:9a:4e:88:1d > 00:26:18:40:21:62, ethertype 802.1Q (0x8100), length 106: vlan 2112, p 7, ethertype 802.1Q, vlan 6, p 0, ethertype IPv4, [|ip]
0x0000: 0026 1840 2162 e046 9a4e 881d 8100 e840
0x0010: 8100 0006 0800 4500 0054 abec 0000
Regards,
Ruslan
From: Eric Dumazet <hidden> Date: 2011-08-30 14:42:41
Le mardi 30 août 2011 à 16:23 +0200, Ruslan N. Marchenko a écrit :
On Tue, Aug 30, 2011 at 03:46:24PM +0200, Ruslan N. Marchenko wrote:
quoted
On Tue, Aug 30, 2011 at 03:23:48PM +0200, Eric Dumazet wrote:
quoted
What kernel version are you using ?
Oh, sorry for missing it, it runs on
Linux ruff.mobi 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:05:14 UTC 2011 i686 i686 i386 GNU/Linux
Just fyi - the openwrt box to which it is connected is
Linux OpenWrt 2.6.39.2 #2 Fri Aug 12 09:36:23 EEST 2011 mips GNU/Linux
although packet drop happens even if there're no vlans configured on remote side.
Latest kernel should be fine. Some patches need to be backported by
Ubuntu team, if you can test them.
commit 9331db4f00cfee8a79d2147ac83723ef436b9759
Author: Jiri Pirko [off-list ref]
Date: Wed Aug 17 23:50:37 2011 -0700
forcedeth: call vlan_mode only if hw supports vlans
If hw does not support vlans, dont call nv_vlan_mode because it has no point.
I believe that this should fix issues on older non-vlan supportive
chips (like Ingo has).
Reported-ty: Ingo Molnar [off-list ref]
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
commit 0891b0e08937aaec2c4734acb94c5ff8042313bb
Author: Jiri Pirko [off-list ref]
Date: Tue Jul 26 10:19:28 2011 +0000
forcedeth: fix vlans
For some reason, when rxaccel is disabled, NV_RX3_VLAN_TAG_PRESENT is
still set and some pseudorandom vids appear. So check for
NETIF_F_HW_VLAN_RX as well. Also set correctly hw_features and set vlan
mode on probe.
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
commit 3326c784c9f492e988617d93f647ae0cfd4c8d09
Author: Jiri Pirko [off-list ref]
Date: Wed Jul 20 04:54:38 2011 +0000
forcedeth: do vlan cleanup
- unify vlan and nonvlan rx path
- kill np->vlangrp and nv_vlan_rx_register
- allow to turn on/off rx vlan accel via ethtool (set_features)
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
From: Ruslan N. Marchenko <hidden> Date: 2011-08-31 13:10:39
On Tue, Aug 30, 2011 at 04:42:42PM +0200, Eric Dumazet wrote:
Le mardi 30 août 2011 à 16:23 +0200, Ruslan N. Marchenko a écrit :
quoted
On Tue, Aug 30, 2011 at 03:46:24PM +0200, Ruslan N. Marchenko wrote:
quoted
On Tue, Aug 30, 2011 at 03:23:48PM +0200, Eric Dumazet wrote:
quoted
What kernel version are you using ?
Oh, sorry for missing it, it runs on
Linux ruff.mobi 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:05:14 UTC 2011 i686 i686 i386 GNU/Linux
Just fyi - the openwrt box to which it is connected is
Linux OpenWrt 2.6.39.2 #2 Fri Aug 12 09:36:23 EEST 2011 mips GNU/Linux
although packet drop happens even if there're no vlans configured on remote side.
Latest kernel should be fine. Some patches need to be backported by
Ubuntu team, if you can test them.
commit 9331db4f00cfee8a79d2147ac83723ef436b9759
Author: Jiri Pirko [off-list ref]
Date: Wed Aug 17 23:50:37 2011 -0700
forcedeth: call vlan_mode only if hw supports vlans
If hw does not support vlans, dont call nv_vlan_mode because it has no point.
I believe that this should fix issues on older non-vlan supportive
chips (like Ingo has).
Reported-ty: Ingo Molnar [off-list ref]
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
commit 0891b0e08937aaec2c4734acb94c5ff8042313bb
Author: Jiri Pirko [off-list ref]
Date: Tue Jul 26 10:19:28 2011 +0000
forcedeth: fix vlans
For some reason, when rxaccel is disabled, NV_RX3_VLAN_TAG_PRESENT is
still set and some pseudorandom vids appear. So check for
NETIF_F_HW_VLAN_RX as well. Also set correctly hw_features and set vlan
mode on probe.
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
commit 3326c784c9f492e988617d93f647ae0cfd4c8d09
Author: Jiri Pirko [off-list ref]
Date: Wed Jul 20 04:54:38 2011 +0000
forcedeth: do vlan cleanup
- unify vlan and nonvlan rx path
- kill np->vlangrp and nv_vlan_rx_register
- allow to turn on/off rx vlan accel via ethtool (set_features)
Signed-off-by: Jiri Pirko [off-list ref]
Signed-off-by: David S. Miller [off-list ref]
Thanks Eric for quick hint, atom n270 however is not as fast in compiling kernel :D
I'm trying to backport the patch (and dependencies) and compile kernel with it, takes lot of time though. Will report back and raise a report/request to ubuntu kernel team. Anyway it seems exactly what I have.
Cheers,
Ruslan