On Wednesday 08 August 2007 13:42:32 you wrote:

Ingo and Thomas,

John and I have been discussing all the "run softirq from IRQ thread"
lately and discovered something nasty.

Now it is a nice optimization to run softirqs from the IRQ thread, but
it may not be feasible because of the semantics of the IRQ thread
compared with the softirq thread. Namely, the softirq thread is bound to
a single CPU and the IRQ thread is not.

We use to think that it would be fine to simply bind an IRQ thread to a
single CPU, either at the start of the IRQ thread code, or just while it
is running the softirq code.  But this has a major flaw as John Stultz
discovered.

If a RT hog that is of higher priority than the IRQ thread preempts the
IRQ thread while it is bound to the CPU (more likely with the latest
code that always binds the IRQ thread to 1 CPU), then that IRQ is, in
essence, masked.  That means no more actions will be taken place by that
IRQ while the RT thread is running.  Normally, one would expect, that if
the IRQ has its affinity set to all CPUS, if a RT thread were to preempt
the IRQ thread and run for a long time, it would be expected that the
IRQ thread would migrate to another CPU and finish. Letting more
interrupts from the IRQ line in (remember that the IRQ line is masked
until the IRQ finishes its handler).

This patch will only run the softirq functions if the IRQ thread and the
softirq thread have the same priority **and** the IRQ thread is already
bound to a single CPU.  If we are running on UP or the IRQ thread is
bound to a single CPU, we already have the possibility of having a RT
hog starve the IRQ.  But we should not add that scenario when the IRQ
thread has its affinity set to run on other CPUS that don't have RT hogs
on them.

It seems to me that this patch will reduce the frequency of irqd/softirqd 
starvation, but the core problem still exists: softirq tasks can't migrate to 
other CPUs to perform their work if a higher priority task preempts them.  
I'm wondering if we want to keep special casing things to minimize the 
problem or not - seems to me the worst case is still the same - and isn't the 
worst case the only case that matters (for -rt)?

-- 
Darren Hart
IBM Linux Technology Center
Realtime Linux Team