Re: [PATCH v3 3/5] mm: Add RCU-based VMA lookup helper that waits for writers

On Mon, Aug 3, 2026 at 4:28 AM Lorenzo Stoakes (ARM) [off-list ref] wrote:

On Sun, Aug 02, 2026 at 02:54:57PM -0700, Suren Baghdasaryan wrote:

quoted

From: Dave Hansen <dave.hansen@linux.intel.com>

== Background ==

There are basically two parallel ways to look up a VMA: the
traditional way, which is protected by mmap_read_lock, and the RCU-based
per-VMA lock way which is based on RCU and refcounts.

== Problem ==

The mmap_lock one is more straightforward to use but it has a big
disadvantage in that it can not be mixed with page faults since those
can take mmap_lock for read, which can deadlock when mixed with nested
page faults and parallel writers.
For example:

      mmap_read_lock(mm);
      // Another thread does mmap_write_lock().
      // New mmap_lock readers are blocked.
      vma = vma_lookup(mm, address);
      // This deadlocks on mmap_read_lock() if it faults:
      copy_from_user(address);
      mmap_read_unlock(mm);

The per-VMA lock can be mixed with faults, but they can fail and need to
be able to fall back to the traditional way.

== Solution ==

Add a variant of the RCU-based lookup that waits for writers. This is
basically the same as the existing RCU-based lookup, but on a failure to
lock it temporarily takes mmap_lock for read and waits for writers
to finish before locking the VMA, dropping the mmap_lock and returning
the locked VMA. This has some advantages:

 1. Callers do not need to have a fallback path for when they
    collide with writers.
 2. It can be used in contexts where page faults can happen because
    it can take the mmap_lock for read but never *holds* it.
 3. Its fast path does not require taking mmap_lock for read.

Basically, when applied correctly, this approach results in faster
*and* simpler code.

Would be nice to have a:

Suggested-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

Ack.

Here given https://lore.kernel.org/linux-mm/af4Zx0gJIWbdDeY2@lucifer/ (local) :)

quoted

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: "Liam R. Howlett" <redacted>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: linux-mm@kvack.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: netdev@vger.kernel.org
---
 include/linux/mmap_lock.h | 15 +++++++++++----
 mm/mmap_lock.c            | 29 +++++++++++++++++++++++++++++
 mm/userfaultfd.c          |  6 ++++--
 3 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index eb32b482434e..fdd8f5cf5722 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h

@@ -228,10 +228,12 @@ static inline void vma_refcount_put(struct vm_area_struct *vma)
 }

 /*
- * Use only while holding mmap read lock which guarantees that locking will not
- * fail (nobody can concurrently write-lock the vma). vma_start_read() should
+ * Use only while holding mmap read lock which guarantees that vma lock is not
+ * contended (nobody can concurrently write-lock the vma). vma_start_read() should
  * not be used in such cases because it might fail due to mm_lock_seq overflow.
  * This functionality is used to obtain vma read lock and drop the mmap read lock.
+ * VMA can't be detached while we are holding mmap lock, therefore in practice this
+ * function can fail only when there are so many readers that vm_refcnt overflows.
  */
 static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass)
 {

@@ -247,16 +249,21 @@ static inline bool vma_start_read_locked_nested(struct vm_area_struct *vma, int
 }

 /*
- * Use only while holding mmap read lock which guarantees that locking will not
- * fail (nobody can concurrently write-lock the vma). vma_start_read() should
+ * Use only while holding mmap read lock which guarantees that vma lock is not
+ * contended (nobody can concurrently write-lock the vma). vma_start_read() should
  * not be used in such cases because it might fail due to mm_lock_seq overflow.
  * This functionality is used to obtain vma read lock and drop the mmap read lock.
+ * VMA can't be detached while we are holding mmap lock, therefore in practice this
+ * function can fail only when there are so many readers that vm_refcnt overflows.
  */
 static inline bool vma_start_read_locked(struct vm_area_struct *vma)
 {
      return vma_start_read_locked_nested(vma, 0);
 }

+struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
+                                            unsigned long address);
+
 static inline void vma_end_read(struct vm_area_struct *vma)
 {
      vma_refcount_put(vma);

diff --git a/mm/mmap_lock.c b/mm/mmap_lock.c
index e20d01e8d38f..6ff05e68e61b 100644
--- a/mm/mmap_lock.c
+++ b/mm/mmap_lock.c

@@ -338,6 +338,35 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
      return NULL;
 }

+/*

Why not a kdoc comment?

Indeed. Will change.

quoted

+ * Find the VMA covering 'address' and lock it for reading. Waits for writers to
+ * finish if the VMA is being modified. Returns NULL if there is no VMA covering
+ * 'address'.
+ *
+ * Use only in code paths where no mmap_lock and no VMA lock is held.

Well, a VMA read lock can be held which would make this a noop essentially.

If a VMA write lock is held you're also ok as the mmap read lock will preclude
an mmap write lock, meaning vma_end_write_all() will have been called and the
write lock released.

So I think you can just drop this line?

Maybe instead of this I should say: Use when mmap_lock is not held,
otherwise use vma_start_read_locked() ? I think that was the original
reason for this comment.

quoted

+ *
+ * The fast path does not take mmap_lock.
+ */
+struct vm_area_struct *vma_start_read_unlocked(struct mm_struct *mm,
+                                            unsigned long address)
+{
+     struct vm_area_struct *vma;
+
+     /* Fast path: return stable VMA covering 'address': */
+     vma = lock_vma_under_rcu(mm, address);
+     if (vma)
+             return vma;
+
+     /* Slow path: preclude VMA writers by temporarily getting mmap read lock. */
+     mmap_read_lock(mm);
+     vma = vma_lookup(mm, address);
+     if (vma && !vma_start_read_locked(vma))

This maybe warrants an unlikely() given it can only happen if refcount
overflows? Also worth having a comment to that effect here?

vma_start_read_locked()
  vma_start_read_locked()
    vma_start_read_locked_nested()
      unlikely(!__refcount_inc_not_zero_limited_acquire())

already contains "unlikely" clause. Do we need to add it in all its users?
I can modify the comment for vma_start_read_locked() stating that
refcount overflow is very unlikely. Would that work or do you want
callers to have that too?

quoted

+             vma = NULL;
+     mmap_read_unlock(mm);
+
+     return vma;
+}
+
 static struct vm_area_struct *lock_next_vma_under_mmap_lock(struct mm_struct *mm,
                                                          struct vma_iterator *vmi,
                                                          unsigned long from_addr)

diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index edd90892f8cc..c3a0c38a3dc3 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c

@@ -129,8 +129,10 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
  *
  * Should be called without holding mmap_lock.
  *
- * Return: A locked vma containing @address, -ENOENT if no vma is found, or
- * -ENOMEM if anon_vma couldn't be allocated.
+ * Return: A locked vma containing @address, -ENOENT if no vma is found,
+ * -ENOMEM if anon_vma couldn't be allocated, or -EAGAIN if vma refcount
+ * overflow happened due to high number of readers and the caller should
+ * retry later.

I'm guessing you're fixing this up as part of the patch? But it feels a bit
random, I mean fine but you should mention this change in the commit message +
explain why.

I'm fixing that because Vlastimil asked about these inconsistencies in
his previous review :)
I can package these fixes as a separate patch or add a comment in the
changelog, smth like "While at it, fix the comments for related
functions". Would that work?

quoted

  */
 static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
                                     unsigned long address)
--
2.55.0.508.g3f0d502094-goog

--
Cheers, Lorenzo