Hi Mathieu, all,

On Tue, Nov 22, 2022 at 03:39PM -0500, Mathieu Desnoyers wrote:

Extend the rseq ABI to expose NUMA node ID, mm_cid, and mm_numa_cid
fields.

The NUMA node ID field allows implementing a faster getcpu(2) in libc.

The per-memory-map concurrency id (mm_cid) [1] allows ideal scaling
(down or up) of user-space per-cpu data structures. The concurrency ids
allocated within a memory map are tracked by the scheduler, which takes
into account the number of concurrently running threads, thus implicitly
considering the number of threads, the cpu affinity, the cpusets
applying to those threads, and the number of logical cores on the
system.

The NUMA-aware concurrency id (mm_numa_cid) is similar to the mm_cid,
except that it keeps track of the NUMA node ids with which each cid has
been associated. On NUMA systems, when a NUMA-aware concurrency ID is
observed by user-space to be associated with a NUMA node, it is
guaranteed to never change NUMA node unless a kernel-level NUMA
configuration change happens. This is useful for NUMA-aware per-cpu data
structures running in environments where a process or a set of processes
belonging to cpuset are pinned to a set of cores which belong to a
subset of the system's NUMA nodes.

[...]

Just out of curiosity: is anyone aware of any libraries that have
started using CIDs? It looks like the cost of CID assignment is always
paid (even though it should be small), I'm trying to understand if after
1.5 years there are common libraries that have started using it and what
their exact usecase is.

I'm aware that TCMalloc was the inspiration for vCPUs [1], then renamed to
CIDs, but am wondering if other users are out there.

Thanks,
-- Marco

[1] https://lore.kernel.org/lkml/20220218210633.23345-10-mathieu.desnoyers@efficios.com/ (local)