Re: [PATCH v3] blk-throtl: Introduce sync and async queues for blk-throtl

On Thu, Jan 05, 2023 at 07:35:59AM -1000, Tejun Heo [off-list ref] wrote:

Hard limits tend to make this sort of problems a lot more pronounced because
the existing mechanisms tend to break down for the users which are severely
throttled down even while the device as a whole is fairly idle. cpu.max
often triggers severe priority inversions too, so it isn't too surprising
that people hit severe priority inversion issues w/ io.max.

To be on the same page:
1) severe PI == priority inversion across cgroups (progated e.g. via
   global locks (as with cpu.max) or FS journal (as with io.max)),
2) ordinary PI == priority inversion contained within a single cgroup,
   i.e. no different from an under-provisioned system.

The reported issue sounds like 2) but even with the separated queues 1)
is still possible :-/

Another problem with blk-throttle is that it doesn't prioritize shared IOs
identified by bio_issue_as_root_blkg() like iolatency and iocost do, so
there can be very severe priority inversions when e.g. journal commit gets
trapped in a low priority cgroup further exacerbating issues like this.

Thanks for the broader view. So the separated queues are certainly an
improvement but ultimately a mechanism based on bio_issue_as_root_blkg()
predicate and deferred throttling would be better? Or is permanent limit
enforcement more important?

Thanks,
Michal