On Mon, May 11, 2020 at 02:23:14AM -0700, Sagi Grimberg wrote:

quoted

quoted

quoted

Basically, my idea is to dequeue request one by one, and for each
dequeued request:

- we try to get a budget and driver tag, if both succeed, add the
request to one per-task list which can be stored in stack variable,
then continue to dequeue more request

- if either budget or driver tag can't be allocated for this request,
marks the last request in the per-task list as .last, and send the
batching requests stored in the list to LLD

- when queueing batching requests to LLD, if one request isn't queued
to driver successfully, calling .commit_rqs() like before, meantime
adding the remained requests in the per-task list back to scheduler
queue or hctx->dispatch.

Sounds good to me.

quoted

One issue is that this way might degrade sequential IO performance if
the LLD just tells queue busy to blk-mq via return value of .queue_rq(),
so I guess we still may need one flag, such as BLK_MQ_F_BATCHING_SUBMISSION.

Why is that degrading sequential I/O performance? because the specific

Some devices may only return BLK_STS_RESOURCE from .queue_rq(), then more
requests are dequeued from scheduler queue if we always queue batching IOs
to LLD, and chance of IO merge is reduced, so sequential IO performance will
be effected.

Such as some scsi device which doesn't use sdev->queue_depth for
throttling IOs.

For virtio-scsi or virtio-blk, we may stop queue for avoiding the
potential affect.

Do we have a way to characterize such devices? I'd assume that most

It may not be easy.

devices will benefit from the batching so maybe the flag needs to be
inverted? BLK_MQ_F_DONT_BATCHING_SUBMISSION?

Actually I'd rather to not add any flag, and we may use some algorithm
(maybe EWMA or other intelligent stuff, or use hctx->dispatch_busy directly)
to figure out one dynamic batching number which is used to dequeue requests
from scheduler or sw queue.

Then just one single dispatch io code path is enough.

Thanks, 
Ming