[PATCH v4 4/6] mm: hugetlb: retry dissolve page when hitting race
From: Muchun Song <hidden>
Date: 2021-01-13 05:27:12
Also in:
lkml
Subsystem:
hugetlb subsystem, memory management, the rest · Maintainers:
Muchun Song, Oscar Salvador, Andrew Morton, Linus Torvalds
There is a race between dissolve_free_huge_page() and put_page(). Theoretically, we should return -EBUSY when we encounter this race. In fact, we have a chance to successfully dissolve the page if we do a retry. Because the race window is quite small. If we seize this opportunity, it is an optimization for increasing the success rate of dissolving page. Signed-off-by: Muchun Song <redacted> --- mm/hugetlb.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 4a9011e12175..898e4ea43e13 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c@@ -1772,6 +1772,7 @@ int dissolve_free_huge_page(struct page *page) { int rc = -EBUSY; +retry: /* Not to disrupt normal path by vainly holding hugetlb_lock */ if (!PageHuge(page)) return 0;
@@ -1793,8 +1794,23 @@ int dissolve_free_huge_page(struct page *page) * We should make sure that the page is already on the free list * when it is dissolved. */ - if (unlikely(!PageHugeFreed(head))) - goto out; + if (unlikely(!PageHugeFreed(head))) { + spin_unlock(&hugetlb_lock); + + /* + * Theoretically, we should return -EBUSY when we + * encounter this race. In fact, we have a chance + * to successfully dissolve the page if we do a + * retry. Because the race window is quite small. + * If we seize this opportunity, it is an optimization + * for increasing the success rate of dissolving page. + */ + while (PageHeadHuge(head) && !PageHugeFreed(head)) { + cond_resched(); + cpu_relax(); + } + goto retry; + } /* * Move PageHWPoison flag from head page to the raw error page,
--
2.11.0