Re: [PATCH] block: throttle: don't add one extra jiffy mistakenly for bps limit

On Fri, Feb 21, 2025 at 11:39:17AM +0800, Yu Kuai wrote:

Hi,

在 2025/02/21 10:55, Ming Lei 写道:

quoted

Hi Yukuai,

On Thu, Feb 20, 2025 at 09:38:12PM +0800, Yu Kuai wrote:

quoted

Hi,

在 2025/02/20 19:17, Ming Lei 写道:

quoted

When the current bio needs to be throttled because of bps limit, the wait
time for the extra bytes may be less than 1 jiffy, tg_within_bps_limit()
adds one extra 1 jiffy.

However, when taking roundup time into account, the extra 1 jiffy
may become not necessary, then bps limit becomes not accurate. This way
causes blktests throtl/001 failure in case of CONFIG_HZ_100=y.

Fix it by not adding the 1 jiffy in case that the roundup time can
cover it.

Cc: Tejun Heo <tj@kernel.org>
Cc: Yu Kuai <redacted>
Signed-off-by: Ming Lei <redacted>
---
   block/blk-throttle.c | 6 +++---
   1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 8d149aff9fd0..8348972c517b 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c

@@ -729,14 +729,14 @@ static unsigned long tg_within_bps_limit(struct throtl_grp *tg, struct bio *bio,
   	extra_bytes = tg->bytes_disp[rw] + bio_size - bytes_allowed;
   	jiffy_wait = div64_u64(extra_bytes * HZ, bps_limit);
-	if (!jiffy_wait)
-		jiffy_wait = 1;
-
   	/*
   	 * This wait time is without taking into consideration the rounding
   	 * up we did. Add that time also.
   	 */
   	jiffy_wait = jiffy_wait + (jiffy_elapsed_rnd - jiffy_elapsed);
+	if (!jiffy_wait)
+		jiffy_wait = 1;

Just wonder, will wait (0, 1) less jiffies is better than wait (0, 1)
more jiffies.

How about following changes?

Thanks,
Kuai

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 8d149aff9fd0..f8430baf3544 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c

@@ -703,6 +703,7 @@ static unsigned long tg_within_bps_limit(struct

throtl_grp *tg, struct bio *bio,
                                 u64 bps_limit)
  {
         bool rw = bio_data_dir(bio);
+       long long carryover_bytes;
         long long bytes_allowed;
         u64 extra_bytes;
         unsigned long jiffy_elapsed, jiffy_wait, jiffy_elapsed_rnd;

@@ -727,10 +728,11 @@ static unsigned long tg_within_bps_limit(struct

throtl_grp *tg, struct bio *bio,

         /* Calc approx time to dispatch */
         extra_bytes = tg->bytes_disp[rw] + bio_size - bytes_allowed;
-       jiffy_wait = div64_u64(extra_bytes * HZ, bps_limit);
+       jiffy_wait = div64_u64_rem(extra_bytes * HZ, bps_limit,
carryover_bytes);

Hi, Thanks for the test.

This is a mistake, carryover_bytes is much bigger than expected :(
That's why the result is much worse. My bad.

quoted

quoted

&carryover_bytes

quoted

+       /* carryover_bytes is dispatched without waiting */
         if (!jiffy_wait)

The if condition shound be removed.

quoted

quoted

-               jiffy_wait = 1;
+               tg->carryover_bytes[rw] -= carryover_bytes;

         /*
          * This wait time is without taking into consideration the rounding

quoted

+
   	return jiffy_wait;

Looks result is worse with your patch:

throtl/001 (basic functionality)                             [failed]
     runtime  6.488s  ...  28.862s
     --- tests/throtl/001.out	2024-11-21 09:20:47.514353642 +0000
     +++ /root/git/blktests/results/nodev/throtl/001.out.bad	2025-02-21 02:51:36.723754146 +0000
     @@ -1,6 +1,6 @@
      Running throtl/001
     +13
      1
     -1
     -1
     +13
      1
     ...
     (Run 'diff -u tests/throtl/001.out /root/git/blktests/results/nodev/throtl/001.out.bad' to see the entire diff)

And I realize now that throtl_start_new_slice() will just clear
the carryover_bytes, I tested in my VM and with following changes,
throtl/001 never fail with CONFIG_HZ_100.

If carryover_bytes can cover this issue, I think it is preferred.

Can you explain a bit why `carryover_bytes/HZ` is subtracted instead of
carryover_bytes?

Also tg_within_bps_limit() may return 0 now, which isn't expected.


Thanks, 
Ming