ipv6 routing broken in 2.6.17-rc3,4

On my home 6to4 gw, ipv6 routing seems to be broken and everything is sent to 6to4 tunnel (the default route). It worked with fine for a long time and with 2.6.17-rc2-g4d5c34ec and it's broken with vmlinuz-2.6.17-rc3-g3cd73eed and 2.6.17-rc4-g9be2f7c3 (yesterdays kernel).

Example (I add an unreachable route to an ipv6 address to force going there by ipv4 and it still uses the route):

vaarikas:~# ip -6 route add unreachable 2001:7a8:1:5::14
vaarikas:~# ping6 2001:7a8:1:5::14
PING 2001:7a8:1:5::14(2001:7a8:1:5::14) 56 data bytes
64 bytes from 2001:7a8:1:5::14: icmp_seq=1 ttl=55 time=56.7 ms
64 bytes from 2001:7a8:1:5::14: icmp_seq=2 ttl=55 time=55.5 ms

--- 2001:7a8:1:5::14 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 55.573/56.155/56.738/0.628 ms
vaarikas:~# ip -6 route
::/96 via :: dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295
unreachable 2001:7a8:1:5::14 dev lo  metric 1024  expires 21334221sec error -101 mtu 16436 advmss 16376 hoplimit 4294967295
2002:5283:297e:2::/64 dev eth0  metric 256  expires 21332659sec mtu 1500 advmss 1440 hoplimit 4294967295
2002:5283:297e::/48 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295
2000::/3 via ::192.88.99.1 dev tun6to4  metric 1  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires 21332650sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev eth1  metric 256  expires 21332654sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295
ff00::/8 dev eth1  metric 256  expires 21332654sec mtu 1500 advmss 1440 hoplimit 4294967295
ff00::/8 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295
ff00::/8 dev eth0  metric 256  expires 21332650sec mtu 1500 advmss 1440 hoplimit 4294967295
unreachable default dev lo  proto none  metric -1  error -101 hoplimit 255

Additional example: nothing goes in the direction of eth0, the packets that should go there also go to tun6to4 interface as confirmed by tcpdump. I have a laptop on eth0 and when I tracepath6 some external address, I get this from tun4to4 device on the gw:

vaarikas:~# tcpdump -n -s 1500 -vv -i tun6to4
tcpdump: WARNING: tun6to4: no IPv4 address assigned
tcpdump: listening on tun6to4, link-type RAW (Raw IP), capture size 1500 bytes
09:14:08.053137 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, time exceeded in-transit, length 1240 for 2001:ad0::10
09:14:09.050230 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, time exceeded in-transit, length 1240 for 2001:ad0::10
09:14:10.091294 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, time exceeded in-transit, length 1240 for 2001:ad0::10
09:14:11.090985 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, packet too big, length 1240, mtu 1480
09:14:12.090354 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, packet too big, length 1240, mtu 1480
09:14:13.090373 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, packet too big, length 1240, mtu 1480
09:14:14.091355 IP6 (hlim 64, next-header: ICMPv6 (58), length: 1240) 2002:5283:297e::42 > 2002:5283:297e:2:210:60ff:fe5a:d6f5: [icmp6 sum ok] ICMP6, packet too big, length 1240, mtu 1480
...

-- 
Meelis Roos (mroos@linux.ee)

On Mon, 15 May 2006, Meelis Roos wrote:

On my home 6to4 gw, ipv6 routing seems to be broken and everything is sent to 6to4 tunnel (the default route). It worked with fine for a long time and with 2.6.17-rc2-g4d5c34ec and it's broken with vmlinuz-2.6.17-rc3-g3cd73eed and 2.6.17-rc4-g9be2f7c3 (yesterdays kernel).

...

vaarikas:~# ip -6 route
2002:5283:297e::/48 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295

You're using the wrong prefix length.  The right one is /16.  Does that work?

Maybe the 6to4 hack which didn't heed routing table (so either prefix length was OK) was removed or changed...

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

quoted

vaarikas:~# ip -6 route
2002:5283:297e::/48 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295

You're using the wrong prefix length.  The right one is /16.  Does that work?

Will try soon today. But the other routes had mor specific masks anyway -
/64 and /128, so why should it change anything?

-- 
Meelis Roos (mroos@linux.ee)

quoted

vaarikas:~# ip -6 route
2002:5283:297e::/48 dev tun6to4  metric 256  expires 21332659sec mtu 1480 advmss 1420 hoplimit 4294967295

You're using the wrong prefix length.  The right one is /16.  Does that work?

Changed tun6to4 to 2002::/16, still the same (packets for eth0 and the
unreachable destination still go to tun6to4).

# ip -6 route
::/96 via :: dev tun6to4  metric 256  expires 21334342sec mtu 1480 advmss 1420 hoplimit 4294967295
unreachable 2001:7a8:1:5::14 dev lo  metric 1024  expires 21334363sec error -101 mtu 16436 advmss 16376 hoplimit 4294967295
2002:5283:2811:2::/64 dev eth0  metric 256  expires 21334342sec mtu 1500 advmss 1440 hoplimit 4294967295
2002::/16 dev tun6to4  metric 256  expires 21334342sec mtu 1480 advmss 1420 hoplimit 4294967295
2000::/3 via ::192.88.99.1 dev tun6to4  metric 1  expires 21334342sec mtu 1480 advmss 1420 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires 21332676sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev eth1  metric 256  expires 21334336sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev tun6to4  metric 256  expires 21334342sec mtu 1480 advmss 1420 hoplimit 4294967295
ff00::/8 dev eth1  metric 256  expires 21334336sec mtu 1500 advmss 1440 hoplimit 4294967295
ff00::/8 dev tun6to4  metric 256  expires 21334342sec mtu 1480 advmss 1420 hoplimit 4294967295
ff00::/8 dev eth0  metric 256  expires 21332676sec mtu 1500 advmss 1440 hoplimit 4294967295
unreachable default dev lo  proto none  metric -1  error -101 hoplimit 255

-- 
Meelis Roos (mroos@linux.ee)

quoted

On my home 6to4 gw, ipv6 routing seems to be broken and everything is sent to 6to4 tunnel (the default route). It worked with fine for a long time and with 2.6.17-rc2-g4d5c34ec and it's broken with vmlinuz-2.6.17-rc3-g3cd73eed and 2.6.17-rc4-g9be2f7c3 (yesterdays kernel).

So far I have narrowed it down to break between 2.6.16 (working) and 2.6.17-rc1 (not working).

-- 
Meelis Roos (mroos@linux.ee)

(To YOSHIFUJI Hideaki: this is about the
http://comments.gmane.org/gmane.linux.network/35262 thread)

Tracked it down to IPV6 merge at 2006-03-21:
commit cd85f6e2f58282186ad720fc18482be228f0b972 is good (right before the bunch of patches)
commit b00055aacdb172c05067612278ba27265fcd05ce is bad (right after the bunch of changes)

http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=shortlog;h=705af309505681f197f81618440954d10f120dc0;pg=44 shows the list of committs at that range.

-- 
Meelis Roos (mroos@linux.ee)

In article [off-list ref] (at Fri, 26 May 2006 02:24:19 +0300 (EEST)), Meelis Roos [off-list ref] says:

(To YOSHIFUJI Hideaki: this is about the
http://comments.gmane.org/gmane.linux.network/35262 thread)

Tracked it down to IPV6 merge at 2006-03-21:
commit cd85f6e2f58282186ad720fc18482be228f0b972 is good (right before 
the bunch of patches)
commit b00055aacdb172c05067612278ba27265fcd05ce is bad (right after the 
bunch of changes)

I guess rt6_select() is returning &ip6_null_entry and
the caller is finding next best route; e.g. default route.

Does this solve your problem?

diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 0190e39..93eb33c 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c

@@ -280,10 +280,12 @@ static int inline rt6_check_neigh(struct
 {
 	struct neighbour *neigh = rt->rt6i_nexthop;
 	int m = 0;
-	if (neigh) {
+	if (rt->rt6i_flags & RTF_NONEXTHOP)
+		m = 1;
+	else if (neigh) {
 		read_lock_bh(&neigh->lock);
 		if (neigh->nud_state & NUD_VALID)
-			m = 1;
+			m = 2;
 		read_unlock_bh(&neigh->lock);
 	}
 	return m;

@@ -292,15 +294,18 @@ static int inline rt6_check_neigh(struct
 static int rt6_score_route(struct rt6_info *rt, int oif,
 			   int strict)
 {
-	int m = rt6_check_dev(rt, oif);
+	int m, n
+		
+	m = rt6_check_dev(rt, oif);
 	if (!m && (strict & RT6_SELECT_F_IFACE))
 		return -1;
 #ifdef CONFIG_IPV6_ROUTER_PREF
 	m |= IPV6_DECODE_PREF(IPV6_EXTRACT_PREF(rt->rt6i_flags)) << 2;
 #endif
-	if (rt6_check_neigh(rt))
+	n = rt6_check_neigh(rt);
+	if (n > 1)
 		m |= 16;
-	else if (strict & RT6_SELECT_F_REACHABLE)
+	else if (!n && strict & RT6_SELECT_F_REACHABLE)
 		return -1;
 	return m;
 }