On Wed, Sep 07, 2011 at 12:57:44PM +0200, Thomas Gleixner wrote:

The problem is that if you enable interrupts on the CPU _BEFORE_ it is
set online AND active, then you can end up waking up kernel threads
which are bound to that CPU and the scheduler will happily schedule
them on an online CPU. That makes them lose the cpu affinity to the
CPU as well and hell breaks lose.

How can that happen?

1. The only interrupts we're likely to receive are the local timer
   interrupts - we have not routed any other interrupts to this CPU.

2. We will not schedule on this CPU except at explicit scheduling
   points (such as contended mutexes or explicit calls to schedule)
   as we have a call to preempt_disable().

Frank has observed this with softirq threads, but the same thing is
true for any other CPU bound thread like the worker stuff.

So who is scheduling a workqueue from the local timer?

So moving the online, active thing BEFORE enabling interrupt is the
only sensible solution.

Yes, that'll be why even x86 enables interrupts before setting the CPU
online for the delay calibration.