Re: [PATCH 3/4] block: move queue enter logic into blk_mq_submit_bio()

On 11/4/21 3:10 AM, Christoph Hellwig wrote:

On Wed, Nov 03, 2021 at 12:32:21PM -0600, Jens Axboe wrote:

quoted

Retain the old logic for the fops based submit, but for our internal
blk_mq_submit_bio(), move the queue entering logic into the core
function itself.

Can you explain the why?  I guess you want to skip the extra reference
for the cached requests now that they already have one.  But please
state that, and explain why it is a fix, as to me it just seems like
another little optimization.

It's just pointless to grab double references, and counter productive
too.

quoted

We need to be a bit careful if going into the scheduler, as a scheduler
or queue mappings can arbitrarily change before we have entered the queue.
Have the bio scheduler mapping do that separately, it's a very cheap
operation compared to actually doing merging locking and lookups.

So just don't do the merges for cache requets and side step this
extra bio_queue_enter for that case?

I'd be fine with that, but it's a bit of a chicken and egg situation as
we don't know. I guess we could move the plugged request check earlier,
and just bypass merging there. Though that makes it a special case
thing, and it's generally useful now. Not sure that would be a good
idea.

quoted

-	if (unlikely(bio_queue_enter(bio) != 0))
-		return;
-
 	if (!submit_bio_checks(bio) || !blk_crypto_bio_prep(&bio))
-		goto queue_exit;
+		return;

This is broken, we really ant the submit checks under freeze
protection to make sure the parameters can't be changed underneath
us.

Which parameters are you worried about in submit_bio_checks()? I don't
immediately see anything that would make me worry about it.

quoted

+static inline bool blk_mq_queue_enter(struct request_queue *q, struct bio *bio)
+{
+	if (!blk_try_enter_queue(q, false) && bio_queue_enter(bio))
+		return false;
+	return true;
+}

This looks weird, as blk_try_enter_queue is already called by
bio_queue_enter.

It's just for avoiding a pointless call into bio_queue_enter(), which
isn't needed it blk_try_enter_queue() is successful. The latter is short
and small and can be inlined, while bio_queue_enter() is a lot bigger.

quoted

 	} else {
 		struct blk_mq_alloc_data data = {
 			.q		= q,

@@ -2528,6 +2534,11 @@ void blk_mq_submit_bio(struct bio *bio)
 			.cmd_flags	= bio->bi_opf,
 		};
 
+		if (unlikely(!blk_mq_queue_enter(q, bio)))
+			return;
+
+		rq_qos_throttle(q, bio);
+

At some point the code in this !cached branch really needs to move
into a helper..

Like in the next patch?

-- 
Jens Axboe