Re: Trouble getting a stable miredo relay

	Hello,

Le jeudi 12 mars 2009 16:06:13 Wouter de Jong, vous avez écrit :

I'm trying to get a miredo relay up & running.
While it should be so easy, I can't get it stable...

After a very short period (< 5-10 minutes) of announcing 2001::/32
via BGP to the rest of the world,
I see a lot of packetloss towards hosts in our network that I reach
through our relay from teredo clients.
Sometimes the addresses are not reachable at all (not even the
relay-address).

This is symptomatic of an overflow in the route/neighbor caches of the kernel. 
Please try to increase the size (a lot) manually via sysctl. I don't know 
anything sane userland/TUNTAP can do about this.

From native IPv6 hosts from outside -> native IPv6 hosts inside our
network and vice versa, there is no packetloss at all.

I tried various hardware (SuperMicro Dual Xeon, Dell 860 Dual-Core
machine, HP DL360, etc)... all ending up with the same result. Tried
various distro's (CentOS 5.2, Fedora 10, Ubuntu server 8.10,
Debian 4.0r7). Distro specific packages, self-compile (against Judy,
without Judy).... no change.

I tried separating the IPv4 traffic from the IPv6 traffic (both on
different nic's), tried tweaking sysctl parameters .... all to no
avail. 

At no point the relay took more than 10Mbit/s.

Miredo holds a paradoxical situation whereby it (in userland) can easily 
handle millions of peers, but the kernel starts failing at 1024 of them.

See also 
http://kerneltrap.org/mailarchive/linux-netdev/2008/8/31/3146914/thread and 
http://kerneltrap.org/mailarchive/linux-netdev/2008/8/31/3147634/thread

You don't need much bandwidth to hit 1024 peers. You just to export your route 
to sufficiently many people. I don't know why there is a neighbors cache for 
no-ARP interfaces in the first place. I guess nobody ever bothered to 
conditionnaly disable the code.

When I restart miredo, it usually works again for a minute (but not
always).  With tcpdump i do see a lot of teredo traffic when it
appears to give unreachable messages for hosts.

Best regards,

-- 
Rémi Denis-Courmont
http://www.remlab.net/