Re: [PATCH 1/1] block: Convert hd_struct in_flight from atomic to percpu

On 06/28/2017 04:49 PM, Jens Axboe wrote:

On 06/28/2017 03:12 PM, Brian King wrote:

quoted

This patch converts the in_flight counter in struct hd_struct from a
pair of atomics to a pair of percpu counters. This eliminates a couple
of atomics from the hot path. When running this on a Power system, to
a single null_blk device with 80 submission queues, irq mode 0, with
80 fio jobs, I saw IOPs go from 1.5M IO/s to 11.4 IO/s.

This has been done before, but I've never really liked it. The reason is
that it means that reading the part stat inflight count now has to
iterate over every possible CPU. Did you use partitions in your testing?
How many CPUs were configured? When I last tested this a few years ago

I did not use partitions. I was running this on a 4 socket Power 8 machine with
5 cores per socket, running with 4 threads per core, so a total of 80
logical CPUs were usable in Linux.

I was missing the fact that part_round_stats_single calls part_in_flight
and had only noticed the sysfs and procfs users of part_in_flight previously.

-Brian

-- 
Brian King
Power Linux I/O
IBM Linux Technology Center