On Wed, Jul 8, 2026 at 1:41 AM Muchun Song [off-list ref] wrote:

quoted

On Jul 8, 2026, at 11:11, James Houghton [off-list ref] wrote:

Hi everyone,

This patch series uses a trick with the Access Flag on CPUs that support
hardware update of the AF to update vmemmap page table entries without
introducing a time window where CPUs accessing the vmemmap might fault.

By avoiding faults, the HugeTLB vmemmap optimization (HVO) can be
implemented correctly on arm64 in a much more straightforward way than
previously attempted, most recently here[1] (please see [1] for a
breakdown of the other approaches attempted before).

Hi,

Thanks for your ongoing efforts on supporting HVO on arm64. This is truly
one of the most useful features that arm64 has been missing for a while.

Hi Muchun,

Thanks for taking a look so quickly. :)

quoted

For large-memory systems that allocate most of their available memory to
HugeTLB, HVO saves a huge amount of memory (1.5% of system memory).

This series has four parts:
 1. Some preparatory changes (patches 1-3)
 1. Bare minimum HVO support (patches 4-10)

It looks like there is a typo in the number here.

Oh yes, so there is. I'll fix it in the next version of the cover letter.

quoted

 2. Drop BBML2_NOABORT requirement for HVO (patches 11-13)
 3. Drop the user-configurable Kconfig for HVO (patches 14-18)

Parts 3 and 4 are technically optional. More details below.

The main functional caveat with this series is that bootmem HugeTLB
pages are not "pre-HVOed". They will be HVOed, but because at pre-HVO
time SMP CPUs have not been enabled, we cannot query for full system
support.

Do you mean that the support for AF might vary across different CPUs?
I'm not that familiar with arm64, so it seems a bit strange to me that
such basic hardware features can differ so much from one CPU to another.

Yes, hardware updates of the Access Flag is a per-CPU feature. It is
available for a CPU to use if TCR_EL1.HA is set. TCR_EL1 is a system
register; each CPU has its own. (Linux will always enable HW AF for a
CPU when it is onlined[1] if support is advertised, so we simply need
to check if support is advertised to know that it is in fact enabled.)

These days it is not uncommon for a system to have two (or more?)
different core implementations, like with "fast" cores and "efficient"
cores.

[1] See the CONFIG_ARM64_HW_AFDBM bits in arch/arm64/mm/proc.S

quoted

This series is based on 7.2-rc2 (0e35b9b6ec0f).

This series almost 100% cleanly applies to mm-new, which has some of
Muchun's HVO patches, with one trivial conflict. I imagine this series
will conflict pretty heavily with some of Muchun's other patches[2].

You know, as we iterate on HVO, the codebase is becoming increasingly
complex and difficult to maintain. Aside from the work in [2] that you
mentioned, I actually have some local patches aimed at refactoring
hugetlb_vmemmap.c as well to reduce this maintenance complexity. I really
want to avoid piling more code on top of it right now, as that would
only make things worse.

Therefore, my personal suggestion is that if this approach is
theoretically validated on arm64, it it better to wait until the HVO
refactoring is complete, and then review your work based on the clean
codebase. For now, what we need to focus on is evaluating the feasibility
and soundness of the approach itself. What do you think?

I agree. The first order of business should be for the Arm folks to
tell me if they think this approach is unsound (though I've been
chatting with Will about this for quite some time :)).

I'm happy to rebase on top of your refactoring work when it is ready.

quoted

-- The AF trick --

The trick is that translations with the AF unset cannot be cached in the
TLB (see Rule R_DWZCQ in the Arm ARM), so they can be atomically updated
without needing a full break-before-make sequence.

So the PTE update sequence becomes:
 1. Atomically clear the AF on the existing PTE.
 2. Invalidate the TLB.
 3. cmpxchg the AF=0 PTE with the new PTE. If this fails, goto 1.

If there is a CPU on the system that does not support hardware access
flag updates, clearing the AF is problematic, as those CPUs might fault
on the vmemmap usage. Therefore, HVO compatbility checks all CPUs for HW
AF updates.

Sorry, I am not familiar with arm64, so I will leave it to the arm64
maintainers (or experts) to verify the feasibility here. Once the viability
of the approach is confirmed, I will look into the HVO-related implementation
details and consider how to support this feature on top of the refactored
code in the future.

Sounds good, thanks Muchun.

I'm hoping you can look at patches 1-3 anyway. I think they're the
right thing to do, even without the arm64 changes.

quoted

-- Application to HVO --

HVO relies on the following page table transitions:
 - When enabling HVO for a page, PMD block entries in the vmemmap are
   shattered into PMD table entries. The first PTE remains mapped
   normally (RW mapping to a real page of struct pages), but the
   remaining PTEs in the vmemmap are mapped read-only to a shared
   page of struct pages (that is, there is an OA change and a
   permissions change).

Just to clarify, does OA mean Output Address? Please spell it out for
readers who are less familiar with this context.

That's right, sorry about that. I'll spell it out more clearly in
future versions of the series.

Thanks, Muchun!