[PATCH v3 04/23] iommu/pages: Make iommu_put_pages_list() work with high order allocations
From: Jason Gunthorpe <jgg@nvidia.com>
Date: 2025-02-25 19:49:57
Also in:
asahi, linux-iommu, linux-patches, linux-riscv, linux-rockchip, linux-samsung-soc, linux-sunxi, linux-tegra
Subsystem:
iommu subsystem, the rest · Maintainers:
Joerg Roedel, Will Deacon, Linus Torvalds
alloc_pages_node(, order) needs to be paired with __free_pages(, order) to
free all the allocated pages. For order != 0 the return from
alloc_pages_node() is just a page list, it hasn't been formed into a
folio.
However iommu_put_pages_list() just calls put_page() on the head page of
an allocation, which will end up leaking the tail pages if order != 0.
Fix this by using __GFP_COMP to create a high order folio and then always
use put_page() to free the full high order folio.
__iommu_free_account() can get the order of the allocation via
folio_order(), which corrects the accounting of high order allocations in
iommu_put_pages_list(). This is the same technique slub uses.
As far as I can tell, none of the places using high order allocations are
also using the free list, so this not a current bug.
Fixes: 06c375053cef ("iommu/vt-d: add wrapper functions for page allocations")
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/iommu/iommu-pages.h | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h
index 0ca2437989a0e1..26b91940bdc146 100644
--- a/drivers/iommu/iommu-pages.h
+++ b/drivers/iommu/iommu-pages.h@@ -38,8 +38,9 @@ static inline void __iommu_alloc_account(struct page *page, int order) * @page: head struct page of the page. * @order: order of the page */ -static inline void __iommu_free_account(struct page *page, int order) +static inline void __iommu_free_account(struct page *page) { + unsigned int order = folio_order(page_folio(page)); const long pgcnt = 1l << order; mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt);
@@ -57,7 +58,8 @@ static inline void __iommu_free_account(struct page *page, int order) */ static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) { - struct page *page = alloc_pages_node(nid, gfp | __GFP_ZERO, order); + struct page *page = + alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); if (unlikely(!page)) return NULL;
@@ -115,8 +117,8 @@ static inline void iommu_free_pages(void *virt, int order) return; page = virt_to_page(virt); - __iommu_free_account(page, order); - __free_pages(page, order); + __iommu_free_account(page); + put_page(page); } /**
@@ -143,7 +145,7 @@ static inline void iommu_put_pages_list(struct list_head *page) struct page *p = list_entry(page->prev, struct page, lru); list_del(&p->lru); - __iommu_free_account(p, 0); + __iommu_free_account(p); put_page(p); } }
--
2.43.0