Re: [PATCH 2/3] block: switch to per-cpu in-flight counters
From: Mike Snitzer <hidden>
Date: 2018-11-30 21:44:44
Also in:
dm-devel
Subsystem:
block layer, the rest · Maintainers:
Jens Axboe, Linus Torvalds
On Tue, Nov 27 2018 at 7:42pm -0500, Mikulas Patocka [off-list ref] wrote:
quoted hunk ↗ jump to hunk
Now when part_round_stats is gone, we can switch to per-cpu in-flight counters. We use the local-atomic type local_t, so that if part_inc_in_flight or part_dec_in_flight is reentrantly called from an interrupt, the value will be correct. The other counters could be corrupted due to reentrant interrupt, but the corruption only results in slight counter skew - the in_flight counter must be exact, so it needs local_t. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> --- block/bio.c | 4 ++-- block/blk-core.c | 4 ++-- block/blk-merge.c | 2 +- block/genhd.c | 47 +++++++++++++++++++++++++++++++++++------------ drivers/md/dm.c | 4 +--- include/linux/genhd.h | 7 ++++--- 6 files changed, 45 insertions(+), 23 deletions(-) Index: linux-block/block/genhd.c ===================================================================--- linux-block.orig/block/genhd.c 2018-11-28 00:09:59.000000000 +0100 +++ linux-block/block/genhd.c 2018-11-28 00:09:59.000000000 +0100@@ -45,53 +45,76 @@ static void disk_add_events(struct gendi static void disk_del_events(struct gendisk *disk); static void disk_release_events(struct gendisk *disk); -void part_inc_in_flight(struct request_queue *q, struct hd_struct *part, int rw) +void part_inc_in_flight(struct request_queue *q, int cpu, struct hd_struct *part, int rw) { if (queue_is_mq(q)) return; - atomic_inc(&part->in_flight[rw]); + local_inc(per_cpu_ptr(part->dkstats, cpu)->in_flight); if (part->partno) - atomic_inc(&part_to_disk(part)->part0.in_flight[rw]); + local_inc(per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight); } -void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, int rw) +void part_dec_in_flight(struct request_queue *q, int cpu, struct hd_struct *part, int rw) { if (queue_is_mq(q)) return; - atomic_dec(&part->in_flight[rw]); + local_dec(per_cpu_ptr(part->dkstats, cpu)->in_flight); if (part->partno) - atomic_dec(&part_to_disk(part)->part0.in_flight[rw]); + local_dec(per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight); }
The above is wrong, needs this incremental fix:
diff --git a/block/genhd.c b/block/genhd.c
index 2ed431098953..3397288a2926 100644
--- a/block/genhd.c
+++ b/block/genhd.c@@ -50,9 +50,9 @@ void part_inc_in_flight(struct request_queue *q, int cpu, struct hd_struct *part if (queue_is_mq(q)) return; - local_inc(per_cpu_ptr(part->dkstats, cpu)->in_flight); + local_inc(&per_cpu_ptr(part->dkstats, cpu)->in_flight[rw]); if (part->partno) - local_inc(per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight); + local_inc(&per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight[rw]); } void part_dec_in_flight(struct request_queue *q, int cpu, struct hd_struct *part, int rw)
@@ -60,9 +60,9 @@ void part_dec_in_flight(struct request_queue *q, int cpu, struct hd_struct *part if (queue_is_mq(q)) return; - local_dec(per_cpu_ptr(part->dkstats, cpu)->in_flight); + local_dec(&per_cpu_ptr(part->dkstats, cpu)->in_flight[rw]); if (part->partno) - local_dec(per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight); + local_dec(&per_cpu_ptr(part_to_disk(part)->part0.dkstats, cpu)->in_flight[rw]); } unsigned int part_in_flight(struct request_queue *q, struct hd_struct *part)