Re: [PATCH 3/4] blk-mq: use hw tag for scheduling if hw tag space is big enough

On Mon, May 01, 2017 at 03:06:16PM +0000, Bart Van Assche wrote:

On Sat, 2017-04-29 at 18:35 +0800, Ming Lei wrote:

quoted

On Fri, Apr 28, 2017 at 06:09:40PM +0000, Bart Van Assche wrote:

quoted

On Fri, 2017-04-28 at 23:15 +0800, Ming Lei wrote:

quoted

+static inline bool blk_mq_sched_may_use_hw_tag(struct request_queue *q)
+{
+	if (q->tag_set->flags & BLK_MQ_F_TAG_SHARED)
+		return false;
+
+	if (blk_mq_get_queue_depth(q) < q->nr_requests)
+		return false;
+
+	return true;
+}

The only user of shared tag sets I know of is scsi-mq. I think it's really
unfortunate that this patch systematically disables BLK_MQ_F_SCHED_USE_HW_TAG
for scsi-mq.

In previous patch, I actually allow driver to pass this flag, but this
feature is dropped in this post, just for making it simple & clean.
If you think we need it for shared tag set, I can add it in v1.

For shared tag sets, I suggest to not enable it at default, because
scheduler is per request queue now, and generaly more requests available,
better it performs.  When tags are shared among several request
queues, one of them may use tags up for its own scheduling, then
starve others. But it should be possible and not difficult to allocate
requests fairly for scheduling in this case if we switch to per-hctx
scheduling.

Hello Ming,

Have you noticed that there is already a mechanism in the block layer to
avoid starvation if a tag set is shared? The hctx_may_queue() function
guarantees that each user that shares a tag set gets at least some tags.
The .active_queues counter keeps track of the number of hardware queues
that share a tag set.

OK, from hctx_may_queue(), each hw queue can be allocated at most
tags of (queue depth / nr_hw_queues), and we can allow to use
hw tag for scheduler too if the following condition is ture for
shared tags:

	q->nr_requests <= (blk_mq_get_queue_depth(q) / nr_hw_queues)

It should often be true for some scsi devices(such as virtio-scsi)
in some configurations.


thanks,
Ming