On 2020-12-08 13:52:25 [+0100], To Daniel Wagner wrote:

On 2020-12-08 13:41:48 [+0100], Daniel Wagner wrote:

quoted

On Tue, Dec 08, 2020 at 12:49:36PM +0100, Sebastian Andrzej Siewior wrote:

quoted

On 2020-12-08 12:36:53 [+0100], Daniel Wagner wrote:

quoted

Obvious in this configuration there are no remote completions (verified
it).

do you complete on a remote CPU if you limit the queues to one (this is
untested of course)?

nvme0n1/ completed   11913011 remote    6718563 56.40%

yes, but how is this relevant? I thought Jens complain was about the
additional indirection via the softirq context

-		rq->q->mq_ops->complete(rq);
+	blk_mq_trigger_softirq(rq);

and not the remote completion path. I can benchmark it out but I don't
know if it's really helping in the discussion.

… blurp

Yes, you are right. Even cross-CPU completion for single-queue was
already completing in softirq. So the only change is for multiqueue
devices which you just demonstrated that it does not happen.
Thank you!

Sebastian