Re: [RFC] futex2: add NUMA awareness

Hi Andrey,

Thanks for the feedback.

Às 08:01 de 14/07/22, Andrey Semashev escreveu:

On 7/14/22 06:18, André Almeida wrote:

[...]

quoted

Feedback? Who else should I CC?

Just a few questions:

Do I understand correctly that notifiers won't be able to wake up
waiters unless they know on which node they are waiting?

If userspace is using NUMA_FLAG, yes. Otherwise all futexes would be
located in the default node, and userspace doesn't need to know which
one is the default.

Is it possible to wait on a futex on different nodes?

Yes, given that you specify `.hint = id` with the proper node id.

Is it possible to wake waiters on a futex on all nodes? When a single
(or N, where N is not "all") waiter is woken, which node is selected? Is
there a rotation of nodes, so that nodes are not skewed in terms of
notified waiters?

Regardless of which node the waiter process is running, what matter is
in which node the futex hash table is. So for instance if we have:

	struct futex32_numa f = {.value = 0, hint = 2};

And now we add some waiters for this futex:

Thread 1, running on node 3:

	futex_wait(&f, 0, FUTEX_NUMA | FUTEX_32, NULL);

Thread 2, running on node 0:

	futex_wait(&f, 0, FUTEX_NUMA | FUTEX_32, NULL);

Thread 3, running on node 2:

	futex_wait(&f, 0, FUTEX_NUMA | FUTEX_32, NULL);

And then, Thread 4, running on node 3:

	futex_wake(&f, 2, FUTEX_NUMA | FUTEX_32);

Now, two waiter would wake up (e.g. T1 and T3, node 3 and 2) and they
are from different nodes. futex_wake() doesn't provide guarantees of
which waiter will be selected, so I can't say which node would be
selected. There's no policy for fairness/starvation for futex_wake(). Do
you think this would be important for the NUMA case?

Let me know if this clarifies your questions.