On Mon, May 18, 2009 at 05:42:41PM +0200, Ingo Molnar wrote:

* Paul E. McKenney [off-list ref] wrote:

quoted

quoted

i might be missing something fundamental here, but why not just 
have per CPU helper threads, all on the same waitqueue, and wake 
them up via a single wake_up() call? That would remove the SMP 
cross call (wakeups do immediate cross-calls already).

My concern with this is that the cache misses accessing all the 
processes on this single waitqueue would be serialized, slowing 
things down. In contrast, the bitmask that smp_call_function() 
traverses delivers on the order of a thousand CPUs' worth of bits 
per cache miss.  I will give it a try, though.

At least if you go via the migration threads, you can queue up 
requests to them locally. But there's going to be cachemisses 
_anyway_, since you have to access them all from a single CPU, and 
then they have to fetch details about what to do, and then have to 
notify the originator about completion.

Ah, so you are suggesting that I use smp_call_function() to run code on
each CPU that wakes up that CPU's migration thread?  I will take a look
at this.

							Thanx, Paul