Re: [PATCH 2/4 v0.5] sched/umcg: RFC: add userspace atomic helpers

On Tue, Sep 14, 2021 at 11:29 AM Peter Oskolkov [off-list ref] wrote:

On Tue, Sep 14, 2021 at 11:05 AM Peter Zijlstra [off-list ref] wrote:

quoted

On Tue, Sep 14, 2021 at 09:29:00AM -0700, Peter Oskolkov wrote:

quoted

In the version of the patchset that I'm preparing to send I've decided
to punt on the issue and just ask the userspace to deal with locking
the memory as it sees fit: mlock() is available and as far as I can

Sadly mlock() does not imply no faults. Someone had a too literal
reading of the POSIX-RT spec (of which mlock is part) and figured that
all that was required was to keep the page in memory, not avoid faults.

Linux has had this bahviour for ages, PREEMPT_RT has tried to change
this, but so far to no avail. At some point sys_mpin() was proposed to
meet the original POSIX-RT intent, but afaict that never actually
happened.

In short, mlock() does not avoid minor faults, or even migration faults,
which can take a fair while to resolve.

Ok, I'll go with transiently pinning pages in
__syscall_enter_from_user_work(), as you suggested. Seems easy enough
to do.

Actually, I think pinning these pages when the worker exits to the
userspace (i.e. is scheduled on a CPU) and releasing them when the
worker is descheduled (blocks) would be better - this way we will be
able to wake the server not only on blocking syscalls but also on
pagefaults (on other pages) as well.

Do you think this approach is acceptable?

Thanks for the suggestion!