Re: [PATCH v3 0/3] A Solution to Re-enable hugetlb vmemmap optimize

On Thu, Jun 27, 2024 at 03:19:55PM -0600, Yu Zhao wrote:

On Wed, Feb 7, 2024 at 5:44 AM Catalin Marinas [off-list ref] wrote:

quoted

On Sat, Jan 27, 2024 at 01:04:15PM +0800, Nanyong Sun wrote:

quoted

On 2024/1/26 2:06, Catalin Marinas wrote:

quoted

On Sat, Jan 13, 2024 at 05:44:33PM +0800, Nanyong Sun wrote:

quoted

HVO was previously disabled on arm64 [1] due to the lack of necessary
BBM(break-before-make) logic when changing page tables.
This set of patches fix this by adding necessary BBM sequence when
changing page table, and supporting vmemmap page fault handling to
fixup kernel address translation fault if vmemmap is concurrently accessed.

[...]

quoted

quoted

How often is this code path called? I wonder whether a stop_machine()
approach would be simpler.

As long as allocating or releasing hugetlb is called.  We cannot
limit users to only allocate or release hugetlb when booting or
not running any workload on all other cpus, so if use
stop_machine(), it will be triggered 8 times every 2M and 4096
times every 1G, which is probably too expensive.

I'm hoping this can be batched somehow and not do a stop_machine() (or
8) for every 2MB huge page.

Theoretically, all hugeTLB vmemmap operations from a single user
request can be done in one batch. This would require the preallocation
of the new copy of vmemmap so that the old copy can be replaced with
one BBM.

Do we ever re-create pmd block entries back for the vmmemap range that
was split or do they remain pmd table + pte entries? If the latter, I
guess we could do a stop_machine() only for a pmd, it should be self
limiting after a while. I don't want user-space to DoS the system by
triggering stop_machine() when mapping/unmapping hugetlbfs pages.

If I did the maths right, for a 2MB hugetlb page, we have about 8
vmemmap pages (32K). Once we split a 2MB vmemap range, whatever else
needs to be touched in this range won't require a stop_machine().

quoted

Just to make sure I understand - is the goal to be able to free struct
pages corresponding to hugetlbfs pages?

Correct, if you are referring to the pages holding struct page[].

quoted

Can we not leave the vmemmap in
place and just release that memory to the page allocator?

We cannot, since the goal is to reuse those pages for something else,
i.e., reduce the metadata overhead for hugeTLB.

What I meant is that we can leave the vmemmap alias in place and just
reuse those pages via the linear map etc. The kernel should touch those
struct pages to corrupt the data. The only problem would be if we
physically unplug those pages but I don't think that's the case here.

-- 
Catalin