On Wed, Nov 14, 2018 at 10:15 AM jianchao.wang
[off-list ref] wrote:

Hi Jens

Thanks for your kindly response.

On 11/13/18 9:44 PM, Jens Axboe wrote:

quoted

On 11/13/18 2:56 AM, Jianchao Wang wrote:

quoted

When issue request directly and the task is migrated out of the
original cpu where it allocates request, hctx could be ran on
the cpu where it is not mapped.
To fix this,
 - insert the request forcibly if BLK_MQ_F_BLOCKING is set.
 - check whether the current is mapped to the hctx, if not, insert
   forcibly.
 - invoke __blk_mq_issue_directly under preemption disabled.

I'm not too crazy about this one, adding a get/put_cpu() in the hot
path, and a cpumask test. The fact is that most/no drivers care
about strict placement. We always try to do so, if convenient,
since it's faster, but this seems to be doing the opposite.

I'd be more inclined to have a driver flag if it needs guaranteed
placement, using one an ops BLK_MQ_F_STRICT_CPU flag or similar.

What do you think?

I'd inclined blk-mq should comply with a unified rule, no matter the
issuing directly path or inserting one. Then blk-mq would have a simpler
model. And also this guarantee could be a little good for drivers,
especially the case where cpu and hw queue mapping is 1:1.

I guess it is quite hard to respect this rule 100%, such as in case of
CPU hotplug.

Thanks,
Ming Lei