On 24.11.21 19:35, Jason Gunthorpe wrote:

On Wed, Nov 24, 2021 at 05:43:58PM +0100, David Hildenbrand wrote:

quoted

On 24.11.21 16:34, Jason Gunthorpe wrote:

quoted

On Wed, Nov 24, 2021 at 03:14:00PM +0100, David Hildenbrand wrote:

quoted

I'm not aware of any where you can fragment 50% of all pageblocks in the
system as an unprivileged user essentially consuming almost no memory
and essentially staying inside well-defined memlock limits. But sure if
there are "many" people will be able to come up with at least one
comparable thing. I'll be happy to learn.

If the concern is that THP's can be DOS'd then any avenue that renders
the system out of THPs is a DOS attack vector. Including all the
normal workloads that people run and already complain that THPs get
exhausted.

A hostile userspace can only quicken this process.

We can not only fragment THP but also easily smaller compound pages,
with less impact though (well, as long as people want more than 0.1% per
user ...).

My point is as long as userspace can drive this fragmentation, by any
means, we can never have DOS proof higher order pages, so lets not
worry so much about one of many ways to create fragmentation.

That would be giving up on compound pages (hugetlbfs, THP, ...) on any
current Linux system that does not use ZONE_MOVABLE -- which is not
something I am not willing to buy into, just like our customers ;)

See my other mail, the upstream version of my reproducer essentially
shows what FOLL_LONGTERM is currently doing wrong with pageblocks. And
at least to me that's an interesting insight :)

I agree that the more extreme scenarios I can construct are a secondary
concern. But my upstream reproducer just highlights what can easily
happen in reality.

quoted

quoted

quoted

My position that FOLL_LONGTERM for unprivileged users is a strong no-go
stands as it is.

As this basically excludes long standing pre-existing things like
RDMA, XDP, io_uring, and more I don't think this can be the general
answer for mm, sorry.

Let's think about options to restrict FOLL_LONGTERM usage:

Which gives me the view that we should be talking about how to make
high order pages completely DOS proof, not about FOLL_LONGTERM.

Sure, one step at a time ;)

To me that is exactly what ZONE_MOVABLE strives to achieve, and I
think anyone who cares about QOS around THP must include ZONE_MOVABLE
in their solution.

For 100% yes.

In all of this I am thinking back to the discussion about the 1GB THP
proposal which was resoundly shot down on the grounds that 2MB THP
*doesn't work* today due to the existing fragmentation problems.

The point that "2MB THP" doesn't work is just wrong. pageblocks do their
job very well, but we can end up in corner case situations where more
and more pageblocks are getting fragmented. And people are constantly
improving these corner cases (e.g. proactive compaction).

Usually you have to allocate *a lot* of memory and put the system under
extreme memory pressure, such that unmovable allocations spill into
movable pageblocks and the other way around.

The thing about my reproducer is that it does that without any memory
pressure, and that is the BIG difference to everything else we have in
that regard. You can have an idle 1TiB system running my reproducer and
it will fragment half of of all pageblocks in the system while mlocking
~ 1GiB. And that highlights the real issue IMHO.

The 1 GB THP project is still going on BTW.

quoted

Another option would be not accounting FOLL_LONGTERM as RLIMIT_MEMLOCK,
but instead as something that explicitly matches the differing
semantics. 

Also a good idea, someone who cares about this should really put
pinned pages into the cgroup machinery (with correct accounting!)

quoted

At the same time, eventually work on proper alternatives with mmu
notifiers (and possibly without the any such limits) where possible
and required.

mmu_notifiers is also bad, it just offends a different group of MM
concerns :)

Yeah, I know, locking nightmare.

Something like io_ring is registering a bulk amount of memory and then
doing some potentially long operations against it.

The individual operations it performs are comparable to O_DIRECT I think
-- but no expert.

So to use a mmu_notifier scheme you'd have to block the mmu_notifier
invalidate_range_start until all the operations touching the memory
finish (and suspend new operations at the same time!).

Blocking the notifier like this locks up the migration/etc threads
completely, and is destructive to the OOM reclaim.

At least with a pinned page those threads don't even try to touch it
instead of getting stuck up.

Yes, if only we'd be pinning for a restricted amount of time ...

-- 
Thanks,

David / dhildenb