Thread (58 messages) 58 messages, 12 authors, 2021-11-30

Re: [PATCH] Increase default MLOCK_LIMIT to 8 MiB

From: Jens Axboe <axboe@kernel.dk>
Date: 2021-11-04 14:44:31
Also in: io-uring, lkml

On 11/4/21 8:27 AM, Cyril Hrubis wrote:
Hi!
quoted
quoted
This limit has not been updated since 2008, when it was increased to 64
KiB at the request of GnuPG. Until recently, the main use-cases for this
feature were (1) preventing sensitive memory from being swapped, as in
GnuPG's use-case; and (2) real-time use-cases. In the first case, little
memory is called for, and in the second case, the user is generally in a
position to increase it if they need more.

The introduction of IOURING_REGISTER_BUFFERS adds a third use-case:
preparing fixed buffers for high-performance I/O. This use-case will
take as much of this memory as it can get, but is still limited to 64
KiB by default, which is very little. This increases the limit to 8 MB,
which was chosen fairly arbitrarily as a more generous, but still
conservative, default value.
---
It is also possible to raise this limit in userspace. This is easily
done, for example, in the use-case of a network daemon: systemd, for
instance, provides for this via LimitMEMLOCK in the service file; OpenRC
via the rc_ulimit variables. However, there is no established userspace
facility for configuring this outside of daemons: end-user applications
do not presently have access to a convenient means of raising their
limits.

The buck, as it were, stops with the kernel. It's much easier to address
it here than it is to bring it to hundreds of distributions, and it can
only realistically be relied upon to be high-enough by end-user software
if it is more-or-less ubiquitous. Most distros don't change this
particular rlimit from the kernel-supplied default value, so a change
here will easily provide that ubiquity.
Agree with raising this limit, it is ridiculously low and we often get
reports from people that can't even do basic rings with it. Particularly
when bpf is involved as well, as it also dips into this pool.

On the production side at facebook, we do raise this limit as well.
We are raising this limit to 2MB for LTP testcases as well, otherwise we
get failures when we run a few bpf tests in quick succession.> 
Acked-by: Cyril Hrubis <chrubis@suse.cz>
Andrew, care to pick this one up for 5.16?

-- 
Jens Axboe
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help