Re: IRQ thread timeouts and affinity

On Thu, 09 Oct 2025 17:05:15 +0100,
Thierry Reding [off-list ref] wrote:

[1  <text/plain; us-ascii (quoted-printable)>]
On Thu, Oct 09, 2025 at 03:30:56PM +0100, Marc Zyngier wrote:

quoted

Hi Thierry,

On Thu, 09 Oct 2025 12:38:55 +0100,
Thierry Reding [off-list ref] wrote:

quoted

Which brings me to the actual question: what is the right way to solve
this? I had, maybe naively, assumed that the default CPU affinity, which
includes all available CPUs, would be sufficient to have interrupts
balanced across all of those CPUs, but that doesn't appear to be the
case. At least not with the GIC (v3) driver which selects one CPU (CPU 0
in this particular case) from the affinity mask to set the "effective
affinity", which then dictates where IRQs are handled and where the
corresponding IRQ thread function is run.

There's a (GIC-specific) answer to that, and that's the "1 of N"
distribution model. The problem is that it is a massive headache (it
completely breaks with per-CPU context).

Heh, that started out as a very promising first paragraph but turned
ugly very quickly... =)

quoted

We could try and hack this in somehow, but defining a reasonable API
is complicated. The set of CPUs receiving 1:N interrupts is a *global*
set, which means you cannot have one interrupt targeting CPUs 0-1, and
another targeting CPUs 2-3. You can only have a single set for all 1:N
interrupts. How would you define such a set in a platform agnostic
manner so that a random driver could use this? I definitely don't want
to have a GIC-specific API.

I see. I've been thinking that maybe the only way to solve this is using
some sort of policy. A very simple policy might be: use CPU 0 as the
"default" interrupt (much like it is now) because like you said there
might be assumptions built-in that break when the interrupt is scheduled
elsewhere. But then let individual drivers opt into the 1:N set, which
would perhaps span all available CPUs but the first one. From an API PoV
this would just be a flag that's passed to request_irq() (or one of its
derivatives).

The $10k question is how do you pick the victim CPUs? I can't see how
to do it in a reasonable way unless we decide that interrupts that
have an affinity matching cpu_possible_mask are 1:N. And then we're
left with wondering what to do about CPU hotplug.

quoted

Overall, there is quite a lot of work to be done in this space: the
machine I'm typing this from doesn't have affinity control *at
all*. Any interrupt can target any CPU,

Well, that actually sounds pretty nice for the use-case that we have...

quoted

                                        and if Linux doesn't expect
that, tough.

... but yeah, it may also break things.

Yeah. With GICv3, only SPIs can be 1:N, but on this (fruity) box, even
MSIs can be arbitrarily moved from one CPU to another. This is a
ticking bomb.

I'll see if I can squeeze out some time to look into this -- no
promises though.

	M.

-- 
Without deviation from the norm, progress is not possible.