On Mon Jun 29, 2026 at 3:25 PM EDT, Lorenzo Stoakes wrote:

The core do_mmap() function accepts a vm_flags_t parameter which it then
manipulates before passing to mmap_region() to do the heavy lifting of the
memory mapping.

Update do_mmap() to instead accept a vma_flags_t parameter, and adjust all
the logic within do_mmap() to manipulate this instead.

This is as part of the ongoing effort to convert VMA flags from a system
word size to a bitmap type which allows us to unrestrict the number of VMA
flags, as well as gain control over how VMA flag manipulation occurs.

We do not cascade these changes to all functions which accept vm_flags_t,
but rather use vma_flags_to_legacy() where necessary, specifically
deferring converting calc_vm_prot_bits(), calc_vm_flag_bits() and
__get_unmapped_area() to vma_flags_t.

Also utilise the new vma_flags_can_grow() predicate which correctly handles
the case of architectures without upward growing stacks.

As part of this change, introduce VMA_SHADOW_STACK so we can correctly
handle the case of the shadow stack not being defined.

No functional change intended.

Signed-off-by: Lorenzo Stoakes <ljs@kernel.org>
---
 arch/mips/kernel/vdso.c |  4 +--
 fs/aio.c                |  2 +-
 include/linux/memfd.h   |  6 ++--
 include/linux/mm.h      |  6 ++--
 ipc/shm.c               |  3 +-
 mm/memfd.c              | 15 ++++-----
 mm/mmap.c               | 67 ++++++++++++++++++++++++-----------------
 mm/nommu.c              |  3 +-
 mm/util.c               | 10 +++---
 mm/vma.c                |  7 ++---
 mm/vma.h                |  2 +-
 11 files changed, 69 insertions(+), 56 deletions(-)

<snip>

 
-static int check_write_seal(vm_flags_t *vm_flags_ptr)
+static int check_write_seal(vma_flags_t *vma_flags_ptr)
 {
-	vm_flags_t vm_flags = *vm_flags_ptr;
-	vm_flags_t mask = vm_flags & (VM_SHARED | VM_WRITE);
-
 	/* If a private mapping then writability is irrelevant. */
-	if (!(mask & VM_SHARED))
+	if (!vma_flags_test(vma_flags_ptr, VMA_SHARED_BIT))
 		return 0;
 
 	/*
 	 * New PROT_WRITE and MAP_SHARED mmaps are not allowed when
 	 * write seals are active.
 	 */
-	if (mask & VM_WRITE)
+	if (vma_flags_test(vma_flags_ptr, VMA_WRITE_BIT))
 		return -EPERM;
 
 	/*
 	 * This is a read-only mapping, disallow mprotect() from making a
 	 * write-sealed mapping writable in future.
 	 */
-	*vm_flags_ptr &= ~VM_MAYWRITE;
+	vma_flags_clear(vma_flags_ptr, VMA_MAYWRITE_BIT);
 
 	return 0;
 }

This function alone changed its original behavior, since vm_flags is a
snapshot of *vm_flags_ptr, but after the change this snapshot is gone.
But its only caller memfd_check_seals_mmap() gets vm_flags_ptr from the
input parameter of do_mmap(), so the overall behavior does not change.

<snip>

Lance pointed out the reordering of setting VMA_DROPPABLE and checking
of can_grow, but these flags are not overlapped and there is no parallel
writer to vma_flags. So it is still no functional change, just not
mechanical changes. :)

Otherwise, LGTM.

Reviewed-by: Zi Yan <ziy@nvidia.com>

-- 
Best Regards,
Yan, Zi