On Mon, Apr 26, 2021 at 08:57:31AM -0600, Jens Axboe wrote:
I was separately curious about this as I have a (as of yet unposted)
patchset that recycles bio allocations, as we spend quite a bit of time
doing that for high rate polled IO. It's good for taking the above 2.97M
IOPS to 3.2-3.3M IOPS, and it'd obviously be a bit more problematic with
required RCU freeing of bio's. Even without the alloc cache, using RCU
will ruin any potential cache locality on back-to-back bio free + bio
alloc.
That sucks indeed. How do you recycle the bios? If we make sure the
bio is only ever recycled as a bio and bi_bdev remaings valid long
enough we might not need the rcu free. Even without your recycling
we could probably do something nasty using SLAB_TYPESAFE_BY_RCU.