Re: [RFC PATCH] Introducing lockless cache built on top of slab allocator

On Mon, Sep 20, 2021 at 11:07:36AM +0200, Vlastimil Babka wrote:

On 9/20/21 03:53, Matthew Wilcox wrote:

quoted

On Mon, Sep 20, 2021 at 01:09:38AM +0000, Hyeonggon Yoo wrote:

quoted

Hello Matthew, Thanks to give me a comment! I appreciate it.
Yeah, we can implement lockless cache using kmem_cache_alloc_{bulk, free}
but kmem_cache_alloc_{free,bulk} is not enough.

quoted

I'd rather see this be part of the slab allocator than a separate API.

And I disagree on this. for because most of situation, we cannot
allocate without lock, it is special case for IO polling.

To make it as part of slab allocator, we need to modify existing data
structure. But making it part of slab allocator will be waste of memory
because most of them are not using this.

Oh, it would have to be an option.  Maybe as a new slab_flags_t flag.
Or maybe a kmem_cache_alloc_percpu_lockless().

I've recently found out that similar attempts (introduce queueing to SLUB)
have been done around 2010. See e.g. [1] but there will be other threads to
search at lore too. Haven't checked yet while it wasn't ultimately merged, 
I guess Christoph and David could remember (this was before my time).

There was attempt on SLUB with queueing as you said.
I searched a bit and found [2] and [3].

- SLUB with queueing (V2) beats SLAB netperf TCP_RR, 2010-07
[2] https://lore.kernel.org/lkml/alpine.DEB.2.00.1007121010420.14328@router.home/T/#m5a31c7caa28b93a00de3af6d547b79273449f5ba (local)

- The Unified slab allocator (V4), 2010-10
[3] https://linux-mm.kvack.narkive.com/e595iCuz/unifiedv4-00-16-the-unified-slab-allocator-v4#post47

Looking at [3], There was still some regression comparing "SLUB with queueing"
with SLAB. And I couldn't find patch series after [3] yet. I'll add link
if I find.

I guess making it opt-in only for caches where performance improvement was
measured would make it easier to add, as for some caches it would mean no
improvement, but increased memory usage. But of course it makes the API more
harder to use.

Do you mean "lockless cache" it should be separate from slab because some caches
doesn't benefit at all?

I'd be careful about the name "lockless", as that's ambiguous. Is it "mostly
lockless" therefore fast, but if the cache is empty, it will still take
locks as part of refill?

It is actually "mostly lockless" so it is ambiguous.
Can you suggest a name? like try_lockless or anything?

Or is it lockless always, therefore useful in
contexts that can take no locks, but then the caller has to have fallbacks
in case the cache is empty and nothing is allocated?

[1] https://lore.kernel.org/linux-mm/20100804024531.914852850@linux.com/T/#u (local)