On Fri, May 22, 2020 at 10:39:23AM +0800, Ming Lei wrote:

On Thu, May 21, 2020 at 12:15:52PM -0700, Bart Van Assche wrote:

quoted

On 2020-05-20 21:33, Ming Lei wrote:

quoted

No.

If vector 3 is for covering hw queue 12 ~ 15, the vector shouldn't be
shutdown when cpu 14 is offline.

quoted

Also I am pretty sure that we don't do this way with managed IRQ. And

non-managed IRQ will be migrated to other online cpus during cpu offline,
so not an issue at all. See migrate_one_irq().

Thanks for the pointer to migrate_one_irq().

However, I'm not convinced the above statement is correct. My
understanding is that the block driver knows which interrupt vector has
been associated with which hardware queue but the blk-mq core not. It
seems to me that patch 6/6 of this series is based on the following
assumptions:
(a) That the interrupt that is associated with a hardware queue is
    processed by one of the CPU's in hctx->cpumask.
(b) That hardware queues do not share interrupt vectors.

I don't think that either assumption is correct.

What the patch tries to do is just:

- when the last cpu of hctx->cpumask is going to become offline, mark
this hctx as inactive, then drain any inflight IO requests originated
from this hctx

The correctness is that once we stops to produce request, we can drain
any in-flight requests before shutdown the last cpu of hctx. Then finally
this hctx becomes quiesced completely. Do you think this way is wrong?
If yes, please prove it.

I don't think this applies to what Bart is saying, but there is a
pathological case where things break down: if a driver uses managed
irq's, but doesn't use the same affinity for the hctx's, an offline cpu
may have been the only one providing irq handling for an online hctx.

I feel like that should be a driver bug if it were to set itself up that
way, but I don't find anything enforces that.