[PATCH v2 11/12] mm/collapse: declare the collapse interface in collapse.h
From: Kiryl Shutsemau <hidden>
Date: 2026-09-10 12:03:37
Also in:
lkml
Subsystem:
memory management, memory management - thp (transparent huge page), the rest · Maintainers:
Andrew Morton, David Hildenbrand, Linus Torvalds
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org> A collapse takes four calls: - collapse_control_init() - set up the control a caller carries; - collapse_scan_pmd() - scan one PTE table, under mmap_lock; - collapse_run_pmd() - collapse what the scan found, no mmap_lock; - collapse_control_release() - done with the control. All four are static in khugepaged.c, as are collapse_possible_orders(), which says what a VMA allows, and the revalidate a caller needs once a collapse has given the mmap_lock up. No other file can ask for a collapse without them. Declare them in collapse.h, with a comment stating the order they are called in and who holds the lock over each step. hugepage_vma_revalidate() becomes collapse_vma_revalidate(): it is part of what a collapse offers now, not a helper of the daemon. Preparation for implementing MADV_COLLAPSE in madvise.c. No functional change. Assisted-by: LLM Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> --- mm/collapse.h | 44 ++++++++++++++++++++++++++++++++++++++++++++ mm/khugepaged.c | 18 +++++++++--------- 2 files changed, 53 insertions(+), 9 deletions(-)
diff --git a/mm/collapse.h b/mm/collapse.h
index 346859a2184f..1ebbbf63fb25 100644
--- a/mm/collapse.h
+++ b/mm/collapse.h@@ -106,4 +106,48 @@ struct collapse_control { bool scan_retract_only; }; +/* Which orders a VMA may collapse to, zero when it may not collapse at all */ +unsigned long collapse_possible_orders(struct vm_area_struct *vma, + vm_flags_t vm_flags, enum tva_type tva_flags); + +/* + * A caller states what it allows in cc->policy and then hands over one PTE + * table's worth of a VMA at a time: + * + * collapse_control_init(cc) once, before the first table + * collapse_scan_pmd(vma, addr, ...) per table + * collapse_run_pmd(mm, addr, cc) when a scan found work + * collapse_control_release(cc) once, when done with the control + * + * The caller holds mmap_lock for reading over the scan and passes an address + * within @vma, aligned to the PTE table the scan is to judge. + * + * The scan returns with that lock still held. It only reads, and almost every + * table it is offered has nothing to collapse, so a caller walks a whole VMA + * under the one lock it took to get there. SCAN_SUCCEED means there is + * something to collapse; anything else is why there is not. + * + * The run is called without the lock and returns without it, taking what it + * needs in between: what it does -- allocate, isolate, copy, flush -- is slow + * enough that a writer would wait behind it. The caller gives the lock up + * first, and with it @vma and anything derived under it, so a caller carrying + * on has to look up again with collapse_vma_revalidate(). The run revalidates + * for itself rather than trusting what the scan saw. + * + * A scan that found something has to be run: the file side takes a reference on + * the file while it still has the VMA to take it from, and the run is what + * gives it back. + */ +void collapse_control_init(struct collapse_control *cc); +void collapse_control_release(struct collapse_control *cc); +enum scan_result collapse_scan_pmd(struct vm_area_struct *vma, + unsigned long addr, struct collapse_control *cc, + unsigned long orders); +enum scan_result collapse_run_pmd(struct mm_struct *mm, unsigned long addr, + struct collapse_control *cc); +enum scan_result collapse_vma_revalidate(struct mm_struct *mm, + unsigned long address, bool expect_anon, + struct vm_area_struct **vmap, struct collapse_control *cc, + unsigned int order); + #endif /* __MM_COLLAPSE_H */
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 9e77f71a788d..77f34aedf0f2 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c@@ -498,7 +498,7 @@ void __khugepaged_enter(struct mm_struct *mm) * Check what orders are possible based on the vma and collapse type. * This is used to determine if mTHP collapse is a viable option. */ -static unsigned long collapse_possible_orders(struct vm_area_struct *vma, +unsigned long collapse_possible_orders(struct vm_area_struct *vma, vm_flags_t vm_flags, enum tva_type tva_flags) { unsigned long orders;
@@ -1016,7 +1016,7 @@ static int collapse_find_target_node(struct collapse_control *cc) * Returns enum scan_result value. */ -static enum scan_result hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, +enum scan_result collapse_vma_revalidate(struct mm_struct *mm, unsigned long address, bool expect_anon, struct vm_area_struct **vmap, struct collapse_control *cc, unsigned int order) {
@@ -1264,7 +1264,7 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s } mmap_read_lock(mm); - result = hugepage_vma_revalidate(mm, pmd_addr, /*expect_anon=*/ true, + result = collapse_vma_revalidate(mm, pmd_addr, /*expect_anon=*/ true, &vma, cc, order); if (result != SCAN_SUCCEED) { mmap_read_unlock(mm);
@@ -1299,7 +1299,7 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s * mmap_lock. */ mmap_write_lock(mm); - result = hugepage_vma_revalidate(mm, pmd_addr, /*expect_anon=*/ true, + result = collapse_vma_revalidate(mm, pmd_addr, /*expect_anon=*/ true, &vma, cc, order); if (result != SCAN_SUCCEED) goto out_up_write;
@@ -2743,13 +2743,13 @@ static enum scan_result collapse_scan_file(struct mm_struct *mm, return result; } -static void collapse_control_init(struct collapse_control *cc) +void collapse_control_init(struct collapse_control *cc) { cc->progress = 0; cc->scan_file = NULL; } -static void collapse_control_release(struct collapse_control *cc) +void collapse_control_release(struct collapse_control *cc) { /* A scan that took a file reference should have been run */ if (WARN_ON_ONCE(cc->scan_file)) {
@@ -2758,7 +2758,7 @@ static void collapse_control_release(struct collapse_control *cc) } } -static enum scan_result collapse_scan_pmd(struct vm_area_struct *vma, +enum scan_result collapse_scan_pmd(struct vm_area_struct *vma, unsigned long addr, struct collapse_control *cc, unsigned long orders) {
@@ -2802,7 +2802,7 @@ static enum scan_result collapse_scan_pmd(struct vm_area_struct *vma, return result; } -static enum scan_result collapse_run_pmd(struct mm_struct *mm, +enum scan_result collapse_run_pmd(struct mm_struct *mm, unsigned long addr, struct collapse_control *cc) { struct file *file = cc->scan_file;
@@ -3256,7 +3256,7 @@ int madvise_collapse(struct vm_area_struct *vma, unsigned long start, if (!vma) { cond_resched(); mmap_read_lock(mm); - result = hugepage_vma_revalidate(mm, addr, false, &found, + result = collapse_vma_revalidate(mm, addr, false, &found, cc, HPAGE_PMD_ORDER); if (result != SCAN_SUCCEED) { last_fail = result;
--
2.54.0