On Mon, Sep 20, 2021 at 02:53:34AM +0100, Matthew Wilcox wrote:

On Mon, Sep 20, 2021 at 01:09:38AM +0000, Hyeonggon Yoo wrote:

quoted

Hello Matthew, Thanks to give me a comment! I appreciate it.

On Sun, Sep 19, 2021 at 08:17:44PM +0100, Matthew Wilcox wrote:

quoted

On Sun, Sep 19, 2021 at 04:42:39PM +0000, Hyeonggon Yoo wrote:

quoted

It is just simple proof of concept, and not ready for submission yet.
There can be wrong code (like wrong gfp flags, or wrong error handling,
etc) it is just simple proof of concept. I want comment from you.

Have you read:

https://www.usenix.org/legacy/event/usenix01/full_papers/bonwick/bonwick_html/
The relevant part of that paper is section 3, magazines.  We should have
low and high water marks for number of objects

I haven't read that before, but after reading it seems not different from
SLAB's percpu queuing.
 

quoted

and we should allocate
from / free to the slab allocator in batches.  Slab has bulk alloc/free
APIs already.

There's kmem_cache_alloc_{bulk,free} functions for bulk
allocation. But it's designed for large number of allocation
to reduce locking cost, not for percpu lockless allocation.

What I'm saying is that rather than a linked list of objects, we should
have an array of, say, 15 pointers per CPU (and a count of how many
allocations we have).  If we are trying to allocate and have no objects,
call kmem_cache_alloc_bulk() for 8 objects.  If we are trying to free
and have 15 objects already, call kmem_cache_free_bulk() for the last
8 objects and set the number of allocated objects to 7.

(maybe 8 and 15 are the wrong numbers.  this is just an example)

Ah, Okay. it seems better to use array. Using cache for list is
unnecessary cost. array is simpler.

quoted

Yeah, we can implement lockless cache using kmem_cache_alloc_{bulk, free}
but kmem_cache_alloc_{free,bulk} is not enough.

quoted

I'd rather see this be part of the slab allocator than a separate API.

And I disagree on this. for because most of situation, we cannot
allocate without lock, it is special case for IO polling.

To make it as part of slab allocator, we need to modify existing data
structure. But making it part of slab allocator will be waste of memory
because most of them are not using this.

Oh, it would have to be an option.  Maybe as a new slab_flags_t flag.
Or maybe a kmem_cache_alloc_percpu_lockless().

Oh, Now I got what you mean. That is a good improvement!

For example,
there is a slab_flags_t flag like SLAB_LOCKLESS.
and a cache created with SLAB_LOCKLESS flag can allocate 
using both kmem_cache_alloc, or kmem_cache_alloc_percpu_lockless
depending on situation? (I suggest kmem_cache_alloc_lockless is better name)

it seems MUCH better. (because it prevents duplicating a cache)

I'll send RFC v2 soon.
Thank you so much Matthew.

If there's misunderstanding from me, please let me know.

Thanks,
Hyeonggon Yoo