[PATCH 15/23] hugetlb/userfaultfd: Hook page faults for uffd write protection
From: Peter Xu <peterx@redhat.com>
Date: 2021-03-23 00:51:20
Also in:
lkml
Subsystem:
hugetlb subsystem, memory management, the rest · Maintainers:
Muchun Song, Oscar Salvador, Andrew Morton, Linus Torvalds
Hook up hugetlbfs_fault() with the capability to handle userfaultfd-wp faults. We do this slightly earlier than hugetlb_cow() so that we can avoid taking some extra locks that we definitely don't need. Signed-off-by: Peter Xu <peterx@redhat.com> --- mm/hugetlb.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 56b78a206913..def2c7ddf3ae 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c@@ -4643,6 +4643,25 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, if (unlikely(!pte_same(entry, huge_ptep_get(ptep)))) goto out_ptl; + /* Handle userfault-wp first, before trying to lock more pages */ + if (userfaultfd_pte_wp(vma, huge_ptep_get(ptep)) && + (flags & FAULT_FLAG_WRITE) && !huge_pte_write(entry)) { + struct vm_fault vmf = { + .vma = vma, + .address = haddr, + .flags = flags, + }; + + spin_unlock(ptl); + if (pagecache_page) { + unlock_page(pagecache_page); + put_page(pagecache_page); + } + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + i_mmap_unlock_read(mapping); + return handle_userfault(&vmf, VM_UFFD_WP); + } + /* * hugetlb_cow() requires page locks of pte_page(entry) and * pagecache_page, so here we need take the former one
--
2.26.2