[PATCH] net: compute a more reasonable default ip6_rt_max_size

The algorithm is based on ipv4 and alloc_large_system_hash().

The following data is from a x86_64 box I tested:

128MB
$ cat /proc/sys/net/ipv{4,6}/route/max_size
16384
22444

512MB
$ cat /proc/sys/net/ipv{4,6}/route/max_size
65536
99856

1GB
$ cat /proc/sys/net/ipv{4,6}/route/max_size
524288
203068

2GB
$ cat /proc/sys/net/ipv{4,6}/route/max_size
1048576
524288

4GB
$ cat /proc/sys/net/ipv{4,6}/route/max_size
2097152
524288

Signed-off-by: Arun Sharma <redacted>
Cc: netdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: David Miller <davem@davemloft.net>
---
 net/ipv6/route.c |   21 ++++++++++++++++++++-
 1 files changed, 20 insertions(+), 1 deletions(-)

diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 49d6ce1..c89ebbb 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c

@@ -2827,6 +2827,16 @@ struct ctl_table * __net_init ipv6_route_sysctl_init(struct net *net)
 }
 #endif
 
+static __initdata unsigned long ip6_rt_entries;
+static int __init set_rt_entries(char *str)
+{
+	if (!str)
+		return 0;
+	ip6_rt_entries = simple_strtoul(str, &str, 0);
+	return 1;
+}
+__setup("ip6_rt_entries=", set_rt_entries);
+
 static int __net_init ip6_route_net_init(struct net *net)
 {
 	int ret = -ENOMEM;

@@ -2872,8 +2882,17 @@ static int __net_init ip6_route_net_init(struct net *net)
 			 ip6_template_metrics, true);
 #endif
 
+	/* Compute a reasonable default based on what we do for ipv4
+	 * total size = 1/16th of total RAM
+	 * No more than 512k entries unless overridden on kernel cmdline */
+        if (ip6_rt_entries == 0) {
+		ip6_rt_entries = (totalram_pages << PAGE_SHIFT) >> 4;
+		ip6_rt_entries /= sizeof(struct rt6_info);
+		ip6_rt_entries = min(512 * 1024UL, ip6_rt_entries);
+        }
+
 	net->ipv6.sysctl.flush_delay = 0;
-	net->ipv6.sysctl.ip6_rt_max_size = 4096;
+	net->ipv6.sysctl.ip6_rt_max_size = ip6_rt_entries;
 	net->ipv6.sysctl.ip6_rt_gc_min_interval = HZ / 2;
 	net->ipv6.sysctl.ip6_rt_gc_timeout = 60*HZ;
 	net->ipv6.sysctl.ip6_rt_gc_interval = 30*HZ;

-- 
1.7.8.4

From: Arun Sharma <redacted>
Date: Fri, 25 May 2012 13:15:34 -0700

+	/* Compute a reasonable default based on what we do for ipv4
+	 * total size = 1/16th of total RAM
+	 * No more than 512k entries unless overridden on kernel cmdline */

Please format this comment correctly:

	/* Compute a reasonable default based on what we do for ipv4
	 * total size = 1/16th of total RAM
	 * No more than 512k entries unless overridden on kernel cmdline
	 */

On Fri, 2012-05-25 at 13:15 -0700, Arun Sharma wrote:

The algorithm is based on ipv4 and alloc_large_system_hash().

Why is it needed at all ?

IPv4 has a route cache with potentially millions of entries, not IPv6.

On 5/25/12 1:47 PM, Eric Dumazet wrote:

On Fri, 2012-05-25 at 13:15 -0700, Arun Sharma wrote:

quoted

The algorithm is based on ipv4 and alloc_large_system_hash().

Why is it needed at all ?

IPv4 has a route cache with potentially millions of entries, not IPv6.

With the default size of 4096 for the ipv6 routing table, entries can 
get garbage collected and hosts could lose their default route and 
therefore lose connectivity.

We actually saw it happen.

  -Arun

From: Arun Sharma <redacted>
Date: Fri, 25 May 2012 15:22:54 -0700

On 5/25/12 1:47 PM, Eric Dumazet wrote:

quoted

On Fri, 2012-05-25 at 13:15 -0700, Arun Sharma wrote:

quoted

The algorithm is based on ipv4 and alloc_large_system_hash().

Why is it needed at all ?

IPv4 has a route cache with potentially millions of entries, not IPv6.

With the default size of 4096 for the ipv6 routing table, entries can
get garbage collected and hosts could lose their default route and
therefore lose connectivity.

We actually saw it happen.

Under no circumstances should administrator configured ipv6 routes be
garbage collected, that is a bug.

On 5/25/12 3:51 PM, David Miller wrote:

From: Arun Sharma<redacted>
Date: Fri, 25 May 2012 15:22:54 -0700

quoted

On 5/25/12 1:47 PM, Eric Dumazet wrote:

quoted

On Fri, 2012-05-25 at 13:15 -0700, Arun Sharma wrote:

quoted

The algorithm is based on ipv4 and alloc_large_system_hash().

Why is it needed at all ?

IPv4 has a route cache with potentially millions of entries, not IPv6.

With the default size of 4096 for the ipv6 routing table, entries can
get garbage collected and hosts could lose their default route and
therefore lose connectivity.

We actually saw it happen.

Under no circumstances should administrator configured ipv6 routes be
garbage collected, that is a bug.

These were not admin configured routes. They were discovered via ipv6 
neighbor discovery.

  -Arun

From: Arun Sharma <redacted>
Date: Fri, 25 May 2012 17:08:59 -0700

On 5/25/12 3:51 PM, David Miller wrote:

quoted

From: Arun Sharma<redacted>
Date: Fri, 25 May 2012 15:22:54 -0700

quoted

On 5/25/12 1:47 PM, Eric Dumazet wrote:

quoted

On Fri, 2012-05-25 at 13:15 -0700, Arun Sharma wrote:

quoted

The algorithm is based on ipv4 and alloc_large_system_hash().

Why is it needed at all ?

IPv4 has a route cache with potentially millions of entries, not IPv6.

With the default size of 4096 for the ipv6 routing table, entries can
get garbage collected and hosts could lose their default route and
therefore lose connectivity.

We actually saw it happen.

Under no circumstances should administrator configured ipv6 routes be
garbage collected, that is a bug.

These were not admin configured routes. They were discovered via ipv6
neighbor discovery.

Then such default routes should either be:

1) Passed over by GC

2) Trigger neighbour discovery when GC'd

On 5/25/12 5:11 PM, David Miller wrote:

quoted

These were not admin configured routes. They were discovered via ipv6
neighbor discovery.

Then such default routes should either be:

1) Passed over by GC

2) Trigger neighbour discovery when GC'd

It's possible that there is a bug somewhere - we didn't get a chance to 
dig deeper. What we observed is that as we got close to the 4096 limit, 
some hosts were becoming unreachable. A modest increase in the routing 
table size made things better.

  -Arun

On Fri, 2012-05-25 at 17:44 -0700, Arun Sharma wrote:

On 5/25/12 5:11 PM, David Miller wrote:

quoted

quoted

These were not admin configured routes. They were discovered via ipv6
neighbor discovery.

Then such default routes should either be:

1) Passed over by GC

2) Trigger neighbour discovery when GC'd

It's possible that there is a bug somewhere - we didn't get a chance to 
dig deeper. What we observed is that as we got close to the 4096 limit, 
some hosts were becoming unreachable. A modest increase in the routing 
table size made things better.

  -Arun

But your patch is not a  "modest increase", so whats the deal ?

A modest increase would be 8192 instead of 4096, regardless of RAM size.

On Sat, 2012-05-26 at 05:39 +0200, Eric Dumazet wrote:

But your patch is not a  "modest increase", so whats the deal ?

A modest increase would be 8192 instead of 4096, regardless of RAM size.

More over, a boot parameter to tweak it is absolutely not needed

sysctl -w net.ipv6.route.max_size=16384

or

echo 16384 >/proc/sys/net/ipv6/route/max_size

IPv4 has to allocate a hash table at boot time, and this hash table is
not resized. Thus some really special purpose machines need a boot
param.

On 5/25/12 8:39 PM, Eric Dumazet wrote:

But your patch is not a  "modest increase", so whats the deal ?

A modest increase would be 8192 instead of 4096, regardless of RAM size.

Yes - 8192 solves our immediate problem, but I was worrying that the 
problem might resurface as ipv6 adoption becomes more widespread.

We were testing a pre-3.0 kernel that didn't have Dave's DST_NOCOUNT 
patch. Will retest with that patch applied.

 > More over, a boot parameter to tweak it is absolutely not needed

Agreed. Will remove that part.

Still not sure why you'd like to go for one size regardless of 
totalram_pages.

  -Arun

On Sat, 2012-05-26 at 20:54 -0700, Arun Sharma wrote:

On 5/25/12 8:39 PM, Eric Dumazet wrote:

quoted

But your patch is not a  "modest increase", so whats the deal ?

A modest increase would be 8192 instead of 4096, regardless of RAM size.

Yes - 8192 solves our immediate problem, but I was worrying that the 
problem might resurface as ipv6 adoption becomes more widespread.

Going from 4096 to 8192 is modest increase. If you put 65536, it should
be enough for the next years.

Your patch was increasing 4096 to 524288 (for 2GB of ram), which sounds
not modest at all.

We were testing a pre-3.0 kernel that didn't have Dave's DST_NOCOUNT 
patch. Will retest with that patch applied.

Good

 > More over, a boot parameter to tweak it is absolutely not needed

Agreed. Will remove that part.

Still not sure why you'd like to go for one size regardless of 
totalram_pages.

Because size of IPv6 route table is not depending on RAM size, but on
number or IPv6 routes.

A router runs a piece of software complex enough to be able to adjust
the limit when needed, don't you think so ?

Your patch basically removes the whole idea of having a limit in the
first place. Why do we have a limit if you set it to four order of
magnitudes bigger than necessary ?