v1->v2:
- dropped debug_vm_pgtable patch which adds a testcase to simulate the
failure scenario. Since it belongs to linux-mm, I will send that out
separately.
- Modified Patch-2 in this series to also cover PMD device migration
entry (in addition to PMD THP migration entry). Hence dropped the
previous RB tag.
- Added a new Patch-3 to fix another selftests WARNING.
- Fixed commit subject of Patch-10.
- Changed subject pre-fix of few patches to be consistent with others
(powerpc/64s)
- Added RB tags
This patch series addresses selftests issues w.r.t warnings or
VM_BUG_ONs seen mainly on book3s64 powerpc kernel. This also carries
cleanups and refactoring changes which I identified while reviewing
other's patches and/or during code walkthrough.
Suggestions and feedback are welcome!
Ritesh Harjani (IBM) (10):
powerpc/pgtable-frag: Fix bad page state in pte_frag_destroy
powerpc/64s: Fix unmap race with PMD migration entries
powerpc/64s: Fix _HPAGE_CHG_MASK to include _PAGE_SPECIAL bit
powerpc/64s/tlbflush-radix: Remove unused radix__flush_tlb_pwc()
powerpc/64s: Move serialize_against_pte_lookup() to hash_pgtable.c
powerpc/64s: Kill the unused argument of exit_lazy_flush_tlb
powerpc/64s: Rename tlbie_va_lpid to tlbie_va_pid_lpid
powerpc/64s: Rename tlbie_lpid_va to tlbie_va_lpid
powerpc/64s: Make use of H_RPTI_TYPE_ALL macro
powerpc: Print MMU_FTRS_POSSIBLE & MMU_FTRS_ALWAYS at startup
arch/powerpc/include/asm/book3s/64/pgtable.h | 20 +++++-
.../include/asm/book3s/64/tlbflush-radix.h | 1 -
arch/powerpc/kernel/setup-common.c | 4 ++
arch/powerpc/mm/book3s64/hash_pgtable.c | 21 +++++++
arch/powerpc/mm/book3s64/internal.h | 2 -
arch/powerpc/mm/book3s64/pgtable.c | 40 +++---------
arch/powerpc/mm/book3s64/radix_tlb.c | 61 ++++++++-----------
arch/powerpc/mm/pgtable-frag.c | 1 +
8 files changed, 79 insertions(+), 71 deletions(-)
--
2.50.1 (Apple Git-155)
powerpc uses pt_frag_refcount as a reference counter for tracking it's
pte and pmd page table fragments. For PTE table, in case of Hash with
64K pagesize, we have 16 fragments of 4K size in one 64K page.
Patch series [1] "mm: free retracted page table by RCU"
added pte_free_defer() to defer the freeing of PTE tables when
retract_page_tables() is called for madvise MADV_COLLAPSE on shmem
range.
[1]: https://lore.kernel.org/all/7cd843a9-aa80-14f-5eb2-33427363c20@google.com/
pte_free_defer() sets the active flag on the corresponding fragment's
folio & calls pte_fragment_free(), which reduces the pt_frag_refcount.
When pt_frag_refcount reaches 0 (no active fragment using the folio), it
checks if the folio active flag is set, if set, it calls call_rcu to
free the folio, it the active flag is unset then it calls pte_free_now().
Now, this can lead to following problem in a corner case...
[ 265.351553][ T183] BUG: Bad page state in process a.out pfn:20d62
[ 265.353555][ T183] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x20d62
[ 265.355457][ T183] flags: 0x3ffff800000100(active|node=0|zone=0|lastcpupid=0x7ffff)
[ 265.358719][ T183] raw: 003ffff800000100 0000000000000000 5deadbeef0000122 0000000000000000
[ 265.360177][ T183] raw: 0000000000000000 c0000000119caf58 00000000ffffffff 0000000000000000
[ 265.361438][ T183] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 265.362572][ T183] Modules linked in:
[ 265.364622][ T183] CPU: 0 UID: 0 PID: 183 Comm: a.out Not tainted 6.18.0-rc3-00141-g1ddeaaace7ff-dirty #53 VOLUNTARY
[ 265.364785][ T183] Hardware name: IBM pSeries (emulated by qemu) POWER10 (architected) 0x801200 0xf000006 of:SLOF,git-ee03ae pSeries
[ 265.364908][ T183] Call Trace:
[ 265.364955][ T183] [c000000011e6f7c0] [c000000001cfaa18] dump_stack_lvl+0x130/0x148 (unreliable)
[ 265.365202][ T183] [c000000011e6f7f0] [c000000000794758] bad_page+0xb4/0x1c8
[ 265.365384][ T183] [c000000011e6f890] [c00000000079c020] __free_frozen_pages+0x838/0xd08
[ 265.365554][ T183] [c000000011e6f980] [c0000000000a70ac] pte_frag_destroy+0x298/0x310
[ 265.365729][ T183] [c000000011e6fa30] [c0000000000aa764] arch_exit_mmap+0x34/0x218
[ 265.365912][ T183] [c000000011e6fa80] [c000000000751698] exit_mmap+0xb8/0x820
[ 265.366080][ T183] [c000000011e6fc30] [c0000000001b1258] __mmput+0x98/0x300
[ 265.366244][ T183] [c000000011e6fc80] [c0000000001c81f8] do_exit+0x470/0x1508
[ 265.366421][ T183] [c000000011e6fd70] [c0000000001c95e4] do_group_exit+0x88/0x148
[ 265.366602][ T183] [c000000011e6fdc0] [c0000000001c96ec] pid_child_should_wake+0x0/0x178
[ 265.366780][ T183] [c000000011e6fdf0] [c00000000003a270] system_call_exception+0x1b0/0x4e0
[ 265.366958][ T183] [c000000011e6fe50] [c00000000000d05c] system_call_vectored_common+0x15c/0x2ec
The bad page state error occurs when such a folio gets freed (with
active flag set), from do_exit() path in parallel.
... this can happen when the pte fragment was allocated from this folio,
but when all the fragments get freed, the pte_frag_refcount still had some
unused fragments. Now, if this process exits, with such folio as it's cached
pte_frag in mm->context, then during pte_frag_destroy(), we simply call
pagetable_dtor() and pagetable_free(), meaning it doesn't clear the
active flag. This, can lead to the above bug. Since we are anyway in
do_exit() path, then if the refcount is 0, then I guess it should be
ok to simply clear the folio active flag before calling pagetable_dtor()
& pagetable_free().
Fixes: 32cc0b7c9d50 ("powerpc: add pte_free_defer() for pgtables sharing page")
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/mm/pgtable-frag.c | 1 +
1 file changed, 1 insertion(+)
commit af38538801c6a ("mm/memory: factor out common code from vm_normal_page_*()"),
added a VM_WARN_ON_ONCE for huge zero pfn.
This can lead to the following call stack.
------------[ cut here ]------------
WARNING: mm/memory.c:735 at vm_normal_page_pmd+0xf0/0x140, CPU#19: hmm-tests/3366
NIP [c00000000078d0c0] vm_normal_page_pmd+0xf0/0x140
LR [c00000000078d060] vm_normal_page_pmd+0x90/0x140
Call Trace:
[c00000016f56f850] [c00000000078d060] vm_normal_page_pmd+0x90/0x140 (unreliable)
[c00000016f56f8a0] [c0000000008a9e30] change_huge_pmd+0x7c0/0x870
[c00000016f56f930] [c0000000007b2bc4] change_protection+0x17a4/0x1e10
[c00000016f56fba0] [c0000000007b3440] mprotect_fixup+0x210/0x4c0
[c00000016f56fc30] [c0000000007b3c3c] do_mprotect_pkey+0x54c/0x780
[c00000016f56fdb0] [c0000000007b3ed8] sys_mprotect+0x68/0x90
[c00000016f56fdf0] [c00000000003ae40] system_call_exception+0x190/0x500
[c00000016f56fe50] [c00000000000d05c] system_call_vectored_common+0x15c/0x2ec
This happens when we call mprotect -> change_huge_pmd()
mprotect()
change_pmd_range()
pmd_modify(oldpmd, newprot) # this clears _PAGE_SPECIAL for zero huge pmd
pmdv = pmd_val(pmd);
pmdv &= _HPAGE_CHG_MASK; # -> gets cleared here
return pmd_set_protbits(__pmd(pmdv), newprot);
can_change_pmd_writable(vma, vmf->address, pmd)
vm_normal_page_pmd(vma, addr, pmd)
__vm_normal_page()
VM_WARN_ON(is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)); # this get hits as _PAGE_SPECIAL for zero huge pmd was cleared.
It can be easily reproduced with the following testcase:
p = mmap(NULL, 2 * hpage_pmd_size, PROT_READ, MAP_PRIVATE |
MAP_ANONYMOUS, -1, 0);
madvise((void *)p, 2 * hpage_pmd_size, MADV_HUGEPAGE);
aligned = (char*)(((unsigned long)p + hpage_pmd_size - 1) &
~(hpage_pmd_size - 1));
(void)(*(volatile char*)aligned); // read fault, installs huge zero PMD
mprotect((void *)aligned, hpage_pmd_size, PROT_READ | PROT_WRITE);
This patch adds _PAGE_SPECIAL to _HPAGE_CHG_MASK similar to
_PAGE_CHG_MASK, as we don't want to clear this bit when calling
pmd_modify() while changing protection bits.
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/include/asm/book3s/64/pgtable.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Originally,
commit fa4531f753f1 ("powerpc/mm: Don't send IPI to all cpus on THP updates")
introduced serialize_against_pte_lookup() call for both Radix and Hash.
However below commit fixed the race with Radix
commit 70cbc3cc78a9 ("mm: gup: fix the fast GUP race against THP collapse")
And therefore following commit removed the
serialize_against_pte_lookup() call from radix_pgtable.c
commit bedf03416913
("powerpc/64s/radix: don't need to broadcast IPI for radix pmd collapse flush")
Now since serialize_against_pte_lookup() only gets called from
hash__pmdp_collapse_flush(), thus move the related functions to
hash_pgtable.c
Hence this patch:
- moves serialize_against_pte_lookup() from radix_pgtable.c to hash_pgtable.c
- removes the radix specific calls from do_serialize()
- renames do_serialize() to do_nothing().
There should not be any functionality change in this patch.
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/include/asm/book3s/64/pgtable.h | 1 -
arch/powerpc/mm/book3s64/hash_pgtable.c | 21 ++++++++++++++++
arch/powerpc/mm/book3s64/pgtable.c | 25 --------------------
3 files changed, 21 insertions(+), 26 deletions(-)
@@ -150,31 +150,6 @@ void set_pud_at(struct mm_struct *mm, unsigned long addr,returnset_pte_at_unchecked(mm,addr,pudp_ptep(pudp),pud_pte(pud));}-staticvoiddo_serialize(void*arg)-{-/* We've taken the IPI, so try to trim the mask while here */-if(radix_enabled()){-structmm_struct*mm=arg;-exit_lazy_flush_tlb(mm,false);-}-}--/*-*Serializeagainst__find_linux_pte()whichdoeslock-less-*lookupinpagetableswithlocalinterruptsdisabled.Forhugepages-*itcastspmd_ttopte_t.Sinceformatofpte_tisdifferentfrom-*pmd_twewanttopreventtransitfrompmdpointingtopagetable-*topmdpointingtohugepage(andback)whileinterruptsaredisabled.-*Weclearpmdtopossiblyreplaceitwithpagetablepointerin-*differentcodepaths.Somakesurewewaitfortheparallel-*__find_linux_pte()tofinish.-*/-voidserialize_against_pte_lookup(structmm_struct*mm)-{-smp_mb();-smp_call_function_many(mm_cpumask(mm),do_serialize,mm,1);-}-/**Weusethistoinvalidateapmdpentrybeforeswitchingfroma*hugeptetoregularpmdentry.
In previous patch we removed the only caller of exit_lazy_flush_tlb()
which was passing always_flush = false in it's second argument.
With that gone, all the callers of exit_lazy_flush_tlb() are local to
radix_pgtable.c and there is no need of an additional argument.
This patch does the required cleanup. There should not be any
functionality change in this patch.
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/mm/book3s64/internal.h | 2 --
arch/powerpc/mm/book3s64/pgtable.c | 2 --
arch/powerpc/mm/book3s64/radix_tlb.c | 14 +++++---------
3 files changed, 5 insertions(+), 13 deletions(-)
It only make sense to rename these functions, so it's better reflect what
they are supposed to do. For e.g. __tlbie_va_pid_lpid name better reflect
that it is invalidating tlbie using VA, PID and LPID.
No functional change in this patch.
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/mm/book3s64/radix_tlb.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
In previous patch we renamed tlbie_va_lpid functions to
tlbie_va_pid_lpid() since those were working with PIDs as well.
This then allows us to rename tlbie_lpid_va to tlbie_va_lpid, which
finally makes all the tlbie function naming consistent.
No functional change in this patch.
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/mm/book3s64/radix_tlb.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
@@ -185,7 +185,7 @@ static __always_inline void __tlbie_va(unsigned long va, unsigned long pid,trace_tlbie(0,0,rb,rs,ric,prs,r);}-static__always_inlinevoid__tlbie_lpid_va(unsignedlongva,unsignedlonglpid,+static__always_inlinevoid__tlbie_va_lpid(unsignedlongva,unsignedlonglpid,unsignedlongap,unsignedlongric){unsignedlongrb,rs,prs,r;
@@ -249,17 +249,17 @@ static inline void fixup_tlbie_pid(unsigned long pid)}}-staticinlinevoidfixup_tlbie_lpid_va(unsignedlongva,unsignedlonglpid,+staticinlinevoidfixup_tlbie_va_lpid(unsignedlongva,unsignedlonglpid,unsignedlongap){if(cpu_has_feature(CPU_FTR_P9_TLBIE_ERAT_BUG)){asmvolatile("ptesync":::"memory");-__tlbie_lpid_va(va,0,ap,RIC_FLUSH_TLB);+__tlbie_va_lpid(va,0,ap,RIC_FLUSH_TLB);}if(cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)){asmvolatile("ptesync":::"memory");-__tlbie_lpid_va(va,lpid,ap,RIC_FLUSH_TLB);+__tlbie_va_lpid(va,lpid,ap,RIC_FLUSH_TLB);}}
@@ -278,7 +278,7 @@ static inline void fixup_tlbie_lpid(unsigned long lpid)if(cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)){asmvolatile("ptesync":::"memory");-__tlbie_lpid_va(va,lpid,mmu_get_ap(MMU_PAGE_64K),RIC_FLUSH_TLB);+__tlbie_va_lpid(va,lpid,mmu_get_ap(MMU_PAGE_64K),RIC_FLUSH_TLB);}}
Similar to CPU_FTRS_[POSSIBLE|ALWAYS], let's also print
MMU_FTRS_[POSSIBLE|ALWAYS]. This has some useful data to capture during
bootup.
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
arch/powerpc/kernel/setup-common.c | 4 ++++
1 file changed, 4 insertions(+)
v1->v2:
- dropped debug_vm_pgtable patch which adds a testcase to simulate the
failure scenario. Since it belongs to linux-mm, I will send that out
separately.
- Modified Patch-2 in this series to also cover PMD device migration
entry (in addition to PMD THP migration entry). Hence dropped the
previous RB tag.
- Added a new Patch-3 to fix another selftests WARNING.
- Fixed commit subject of Patch-10.
- Changed subject pre-fix of few patches to be consistent with others
(powerpc/64s)
- Added RB tags
This patch series addresses selftests issues w.r.t warnings or
VM_BUG_ONs seen mainly on book3s64 powerpc kernel. This also carries
cleanups and refactoring changes which I identified while reviewing
other's patches and/or during code walkthrough.
Suggestions and feedback are welcome!
Ritesh Harjani (IBM) (10):
powerpc/pgtable-frag: Fix bad page state in pte_frag_destroy
powerpc/64s: Fix unmap race with PMD migration entries
powerpc/64s: Fix _HPAGE_CHG_MASK to include _PAGE_SPECIAL bit
powerpc/64s/tlbflush-radix: Remove unused radix__flush_tlb_pwc()
powerpc/64s: Move serialize_against_pte_lookup() to hash_pgtable.c
powerpc/64s: Kill the unused argument of exit_lazy_flush_tlb
powerpc/64s: Rename tlbie_va_lpid to tlbie_va_pid_lpid
powerpc/64s: Rename tlbie_lpid_va to tlbie_va_lpid
powerpc/64s: Make use of H_RPTI_TYPE_ALL macro
powerpc: Print MMU_FTRS_POSSIBLE & MMU_FTRS_ALWAYS at startup
arch/powerpc/include/asm/book3s/64/pgtable.h | 20 +++++-
.../include/asm/book3s/64/tlbflush-radix.h | 1 -
arch/powerpc/kernel/setup-common.c | 4 ++
arch/powerpc/mm/book3s64/hash_pgtable.c | 21 +++++++
arch/powerpc/mm/book3s64/internal.h | 2 -
arch/powerpc/mm/book3s64/pgtable.c | 40 +++---------
arch/powerpc/mm/book3s64/radix_tlb.c | 61 ++++++++-----------
arch/powerpc/mm/pgtable-frag.c | 1 +
8 files changed, 79 insertions(+), 71 deletions(-)
Tested-by: Venkat Rao Bagalkote <redacted>
I applied the patch series on top of mainline and verified that the kernel
builds and boots successfully.
I also ran the following test suites on both RADIX (POWER11) and HASH
(POWER9)
MMU configurations:
- tools/testing/selftests/mm
- tools/testing/selftests/memory-hotplug
- tools/testing/selftests/powerpc/mm
- tools/testing/selftests/powerpc/cache_shape
- tools/testing/selftests/powerpc/copyloops
In addition, I executed basic sanity and stress tests, including:
stutter, eatmemory, hugepage_sanity, fork_mem, memory_api mprotect,
vatest, and several transparent-hugepage sanity checks.
All tests passed without regressions.
Regards,
Venkat
v1->v2:
- dropped debug_vm_pgtable patch which adds a testcase to simulate the
failure scenario. Since it belongs to linux-mm, I will send that out
separately.
- Modified Patch-2 in this series to also cover PMD device migration
entry (in addition to PMD THP migration entry). Hence dropped the
previous RB tag.
- Added a new Patch-3 to fix another selftests WARNING.
- Fixed commit subject of Patch-10.
- Changed subject pre-fix of few patches to be consistent with others
(powerpc/64s)
- Added RB tags
This patch series addresses selftests issues w.r.t warnings or
VM_BUG_ONs seen mainly on book3s64 powerpc kernel. This also carries
cleanups and refactoring changes which I identified while reviewing
other's patches and/or during code walkthrough.
Suggestions and feedback are welcome!
Ritesh Harjani (IBM) (10):
powerpc/pgtable-frag: Fix bad page state in pte_frag_destroy
powerpc/64s: Fix unmap race with PMD migration entries
powerpc/64s: Fix _HPAGE_CHG_MASK to include _PAGE_SPECIAL bit
powerpc/64s/tlbflush-radix: Remove unused radix__flush_tlb_pwc()
powerpc/64s: Move serialize_against_pte_lookup() to hash_pgtable.c
powerpc/64s: Kill the unused argument of exit_lazy_flush_tlb
powerpc/64s: Rename tlbie_va_lpid to tlbie_va_pid_lpid
powerpc/64s: Rename tlbie_lpid_va to tlbie_va_lpid
powerpc/64s: Make use of H_RPTI_TYPE_ALL macro
powerpc: Print MMU_FTRS_POSSIBLE & MMU_FTRS_ALWAYS at startup
arch/powerpc/include/asm/book3s/64/pgtable.h | 20 +++++-
.../include/asm/book3s/64/tlbflush-radix.h | 1 -
arch/powerpc/kernel/setup-common.c | 4 ++
arch/powerpc/mm/book3s64/hash_pgtable.c | 21 +++++++
arch/powerpc/mm/book3s64/internal.h | 2 -
arch/powerpc/mm/book3s64/pgtable.c | 40 +++---------
arch/powerpc/mm/book3s64/radix_tlb.c | 61 ++++++++-----------
arch/powerpc/mm/pgtable-frag.c | 1 +
8 files changed, 79 insertions(+), 71 deletions(-)
Tested-by: Venkat Rao Bagalkote <redacted>
Thanks a lot!
I applied the patch series on top of mainline and verified that the kernel
builds and boots successfully.
I also ran the following test suites on both RADIX (POWER11) and HASH
(POWER9)
MMU configurations:
- tools/testing/selftests/mm
- tools/testing/selftests/memory-hotplug
- tools/testing/selftests/powerpc/mm
- tools/testing/selftests/powerpc/cache_shape
- tools/testing/selftests/powerpc/copyloops
In addition, I executed basic sanity and stress tests, including:
stutter, eatmemory, hugepage_sanity, fork_mem, memory_api mprotect,
vatest, and several transparent-hugepage sanity checks.
Thanks Venkat for verifying this extensively.
So other than couple of hmm tests, there shouldn't be any other kernel
warnings or any VM_BUG_ONs() hitting after this patch series.
(we discussed this internally too!)
And as for those warnings with hmm tests fixes - will fix them in a
separate patch series later (as those looks to be non-powerpc fixes).
This should also enable Venkat and other CI systems to run mm selftests
on book3s64 PowerPC without any kernel issues.
On Mon, 09 Mar 2026 23:44:23 +0530, Ritesh Harjani (IBM) wrote:
v1->v2:
- dropped debug_vm_pgtable patch which adds a testcase to simulate the
failure scenario. Since it belongs to linux-mm, I will send that out
separately.
- Modified Patch-2 in this series to also cover PMD device migration
entry (in addition to PMD THP migration entry). Hence dropped the
previous RB tag.
- Added a new Patch-3 to fix another selftests WARNING.
- Fixed commit subject of Patch-10.
- Changed subject pre-fix of few patches to be consistent with others
(powerpc/64s)
- Added RB tags
[...]