On 2/17/23 7:27 AM, Pankaj Raghav wrote:

Hi Jens,

On 2023-02-16 05:08, Jens Axboe wrote:

quoted

I think your numbers are skewed because brd isn't flagg nowait, can you
try with this?

I ran some quick testing here, using the current tree:

		without patch		with patch
io_uring	~430K IOPS		~3.4M IOPS
libaio		~895K IOPS		~895K IOPS

which is a pretty substantial difference...

I rebased my blk-mq changes on top of your nowait patches, but still I see a
regression with blk-mq. When I tried to trace and run perf, nothing odd
stood out, except for the normal blk-mq overhead.

Could you try it in your setup and see if you are noticing a similar trend?
Because based on the numbers you shared yesterday, I didn't see this regression.

fio script I run to benchmark:

$ fio --name=<workload>  --rw=<workload>  --ramp_time=5s --size=1G
--io_size=10G --loop=4 --cpus_allowed=1 --filename=/dev/ram0 --direct=1
--iodepth=128 --ioengine=<engine>

+-----------+-----------+--------+--------+
| io_uring  | bio(base) | blk-mq | delta  |
+-----------+-----------+--------+--------+
|   read    |    577    |  446   | -22.7  |
| randread  |    504    |  416   | -17.46 |
|   write   |    554    |  424   | -23.47 |
| randwrite |    484    |  381   | -21.28 |
+-----------+-----------+--------+--------+

+-----------+-----------+--------+--------+
|  libaio   | bio(base) | blk-mq | delta  |
+-----------+-----------+--------+--------+
|   read    |    412    |  341   | -17.23 |
| randread  |    389    |  335   | -13.88 |
|   write   |    401    |  329   | -17.96 |
| randwrite |    351    |  304   | -13.39 |
+-----------+-----------+--------+--------+

This is pretty much expected, as blk-mq adds a bunch of things that
brd doesn't really care about. One example of such would be tag
management.

My reaction to your initial report wasn't a surprise that blk-mq
would be slower than bio based for this use case, rather that
io_uring was slower than libaio.

-- 
Jens Axboe