From: Josef Bacik <josef@toxicpanda.com>
blk_mq_get_tag() checks BLK_MQ_S_INACTIVE after getting a tag and gives
it back if the flag is set, so the caller retries on the hctx of the CPU
it runs on now. The batched allocation for plugged submitters doesn't,
so a task migrated off a dying CPU can fill plug->cached_rqs from a
drained hctx, and those requests never complete.
Check the flag in blk_mq_get_tags() too and release the tags if it's
set. The caller falls back to blk_mq_get_tag(), which fails the same way
and starts over.
Fixes: 47c122e35d7e ("block: pre-allocate requests if plug is started and is a batch")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
block/blk-mq-tag.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index 2b751bfa50e1..aaea2e2c2f23 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -139,6 +139,18 @@ unsigned long blk_mq_get_tags(struct blk_mq_alloc_data *data, int nr_tags,
return 0;
ret = __sbitmap_queue_get_batch(bt, nr_tags, offset);
*offset += tags->nr_reserved_tags;
+
+ /*
+ * Give up this allocation if the hctx is inactive, like blk_mq_get_tag()
+ * does. The caller falls back to it and will retry on an active hctx.
+ */
+ if (ret && unlikely(test_bit(BLK_MQ_S_INACTIVE, &data->hctx->state))) {
+ unsigned int i;
+
+ for_each_set_bit(i, &ret, BITS_PER_LONG)
+ blk_mq_put_tag(tags, data->ctx, *offset + i);
+ return 0;
+ }
return ret;
}
--
2.56.0