[PATCH for 7.4-rc2+ 0/7] Re-enable discarding lazyfree THPs at PMD level

For maintainers, Patch 6 requires first 5 patches to work correctly, so it
might be better to get ack from arch maintainers for first 5 patches and
merge the series via mm tree. The patch disabling lazyfree THP discard at
PMD level has been picked up [2] and would take some time to test, so this
patchset probably can be considered after the merge window closes. I am
sending it out for early feedback.

Motivation
===

A bug was found in the PMD-level discard of lazyfree THPs [1]. The
proposed fix keeps the PMD invalidated with pmdp_invalidate() instead of
clearing it, but that does not work on every architecture:

1. riscv, LoongArch and s390 let GUP-fast follow an invalidated PMD;
2. powerpc/book3s64 page table check counts an invalidated PMD as mapped;
3. sparc64's thp_pte_count becomes unbalanced around pmdp_invalidate().

So the PMD-level discard was disabled instead [2]. This series fixes the
arch code and re-enables it.

The discard decides from the folio refcount, so GUP-fast must not take a
reference through an invalidated PMD, and arch accounting must stay
balanced when an invalidated PMD is set again or cleared.

Details
===

Patches 1-5 fix the arch code, patch 6 from Kyle keeps the PMD invalidated
instead of cleared until the discard is committed, and patch 7 enables the
PMD-level discard again:

1. powerpc/book3s64: page table check counts an invalidated PMD as a live
   mapping, so setting it again or clearing it decrements the map count a
   second time and hits BUG_ON(). This is already reachable via MADV_COLD
   and mprotect() with CONFIG_PAGE_TABLE_CHECK.

2. riscv: an invalidated PMD still passes pmd_present() and the generic
   pmd_access_permitted(), so GUP-fast follows it. Add a
   pmd_access_permitted() that requires _PAGE_PRESENT.

3. LoongArch: same problem and fix as riscv.

4. s390: _SEGMENT_ENTRY_INVALID also marks an old PMD, so GUP-fast follows
   an invalidated PMD. Always clear _SEGMENT_ENTRY_READ in
   pmdp_invalidate(), which makes it pmd_protnone(), so GUP-fast rejects
   it.

5. sparc64: pmdp_invalidate() decrements thp_pte_count assuming a THP
   split follows, so setting the PMD again loses a count. Count only
   valid huge PMDs in __set_pmd_acct() instead, which keeps invalidate,
   restore, clear and split balanced.

6. mm/huge_memory: use pmdp_invalidate() in the discard, so munmap() and
   mremap() wait on the PMD lock instead of skipping a none PMD.

7. mm/rmap: discard lazyfree THPs at PMD level again.

Note
===

x86, arm64 and powerpc already make GUP-fast reject an invalidated PMD,
and sparc64 has no GUP-fast. 32-bit MIPS would not reject it, but it
cannot have THP: _PAGE_HUGE makes PFN_PTE_SHIFT exceed PAGE_SHIFT, so the
BUILD_BUG_ON() in arch_mm_preinit() fires, which is why Ingenic dropped
huge pages in commit 69b686ddd579 ("MIPS: ingenic: Remove
CPU_SUPPORTS_HUGEPAGES").

Link: https://lore.kernel.org/all/20261009165214.40212-2-kylebot@openai.com/ (local) [1]
Link: https://lore.kernel.org/all/20261012-lazyfree-stable-fix-v2-1-6387d6e30063@nvidia.com/ (local) [2]

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
Kyle Zeng (1):
      mm/huge_memory: avoid transient none PMDs during lazyfree reclaim

Zi Yan (6):
      powerpc/book3s64/mm: don't count invalidated PMDs in page table check
      riscv: mm: don't let GUP-fast follow invalidated THP PMDs
      LoongArch: mm: don't let GUP-fast follow invalidated THP PMDs
      s390/mm: always clear _SEGMENT_ENTRY_READ in pmdp_invalidate()
      sparc64: mm: fix thp_pte_count accounting around pmdp_invalidate()
      mm/rmap: re-enable discarding lazyfree THPs at PMD level

 arch/loongarch/include/asm/pgtable.h         |  6 ++++++
 arch/powerpc/include/asm/book3s/64/pgtable.h |  3 ++-
 arch/riscv/include/asm/pgtable.h             |  6 ++++++
 arch/s390/include/asm/pgtable.h              |  8 ++++++--
 arch/sparc/mm/tlb.c                          | 26 ++++++++++++--------------
 mm/huge_memory.c                             |  5 +++--
 mm/rmap.c                                    | 11 +++++++++++
 7 files changed, 46 insertions(+), 19 deletions(-)
---
base-commit: 71a9b024444eba184335b67867897714e4da143c
change-id: 20261007-anon_vma_uaf_fix-e7accd1d2ee1

Best regards,
--  
Yan, Zi