On Wed, Sep 22, 2021 at 04:04:44PM -0400, Peter Xu wrote:

On Wed, Sep 22, 2021 at 12:29:35PM -0700, Yang Shi wrote:

quoted

khugepqged does remove the pgtables. Please check out
retract_page_tables(). The pmd will be cleared and the ptes will be
freed otherwise the collapsed THP won't get PMD mapped by later
access.

Indeed.

I should probably still properly disable khugepaged for at least VM_SHARED &&
VM_UFFD_WP, then I'd keep the anonymous && minor mode behavior untouched.

The other problem is even if current mm/vma doesn't have UFFD_WP registered,
some other mm/vma could have UFFD_WP enabled there that mapped the same file.
Checking that up within retract_page_tables() on all VMAs seems to be a bit too
late.

Checking it early may not trivially work too - I can walk the vma interval tree
at the entry of khugepaged_scan_file(), making sure no vma has UFFD_WP set.
However I don't see how it'll stop some of the vma from having UFFD_WP
registered later after that point but before retract_page_tables().

I'll need to think about it, but thanks for the input, Yang.  That's a very
important point.

Perhaps I need something like this:

---8<---

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 045cc579f724..c63e957336d1 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c

@@ -1583,6 +1583,15 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)
                pmd = mm_find_pmd(mm, addr);
                if (!pmd)
                        continue;
+               /*
+                * When a vma is registered with uffd-wp, we can't recycle the
+                * pmd pgtable because there can be pte markers installed.
+                * Skip it only, so the rest mm/vma can still have the same
+                * file mapped hugely, however it'll always mapped in small
+                * page size for uffd-wp registered ranges.
+                */
+               if (userfaultfd_wp(vma))
+                       continue;
                /*
                 * We need exclusive mmap_lock to retract page table.
                 *
---8<---

I won't post a v2 because then that patch will be shmem-only and uffd-wp-only.
I'll keep it with the upcoming series I'm going to post to support shmem.

Thanks,

-- 
Peter Xu