Re: Scalability of interface creation and deletion

--On 8 May 2011 05:44:28 -0700 "Paul E. McKenney" 
[off-list ref] wrote:

OK, let's break it out...

4.022 seconds for 100 interfaces means about 40 milliseconds per
interface.

It's 200 interfaces as they are veth pairs, so 20ms.

My guess is that you have CONFIG_NO_HZ=y, which means that RCU needs to
figure out that various CPUs are in dyntick-idle state, which is a minimum
of 6 jiffies.

That sounds like the main cause.

It could be longer if a given CPU happens to be in IRQ
when RCU checks, so call it 9 jiffies.

That sounds unlikely to happen much of the time. Looking at /proc/interrupts
not much is going on.

If you are doing the interfaces
synchronously, you will likely have to wait for a prior grace period (due
to background activity).

Not sure I understand that (there's nothing else going on), but...

So I can easily imagine 18 milliseconds for
HZ=1000.  40 milliseconds sounds a bit high, but perhaps not impossible.

Add to that the fact that there are 2 x synchronize_sched() and one
rcu_barrier() per interface removal. If I ignore your IRQ idea, that's
6 x 3 = 18ms per interface removal at CONFIG_HZ=10, 180ms at
CONFIG_HZ=100.

-- 
Alex Bligh