On Tue, Nov 23, 2021 at 02:23:23PM -0800, Mina Almasry wrote:

On Tue, Nov 23, 2021 at 2:03 PM Matthew Wilcox [off-list ref] wrote:

quoted

On Tue, Nov 23, 2021 at 01:47:33PM -0800, Mina Almasry wrote:

quoted

On Tue, Nov 23, 2021 at 1:30 PM Matthew Wilcox [off-list ref] wrote:

quoted

What I've been trying to communicate over the N reviews of this
patch series is that *the same thing is about to happen to THPs*.
Only more so.  THPs are going to be of arbitrary power-of-two size, not
necessarily sizes supported by the hardware.  That means that we need to
be extremely precise about what we mean by "is this a THP?"  Do we just
mean "This is a compound page?"  Do we mean "this is mapped by a PMD?"
Or do we mean something else?  And I feel like I haven't been able to
get that information out of you.

Yes, I'm very sorry for the trouble, but I'm also confused what the
disconnect is. To allocate hugepages I can do like so:

mount -t tmpfs -o huge=always tmpfs /mnt/mytmpfs

or

madvise(..., MADV_HUGEPAGE)

Note I don't ask the kernel for a specific size, or a specific mapping
mechanism (PMD/contig PTE/contig PMD/PUD), I just ask the kernel for
'huge' pages. I would like to know whether the kernel was successful
in allocating a hugepage or not. Today a THP hugepage AFAICT is PMD
mapped + is_transparent_hugepage(), which is the check I have here. In
the future, THP may become an arbitrary power of two size, and I think
I'll need to update this querying interface once/if that gets merged
to the kernel. I.e, if in the future I allocate pages by using:

mount -t tmpfs -o huge=2MB tmpfs /mnt/mytmpfs

I need the kernel to tell me whether the mapping is 2MB size or not.

If I allocate pages by using:

mount -t tmpfs -o huge=pmd tmpfs /mnt/mytmps,

Then I need the kernel to tell me whether the pages are PMD mapped or
not, as I'm doing here.

The current implementation is based on what the current THP
implementation is in the kernel, and depending on future changes to
THP I may need to update it in the future. Does that make sense?

Well, no.  You're adding (or changing, if you like) a userspace API.
We need to be precise about what that userspace API *means*, so that we
don't break it in the future when the implementation changes.  You're
still being fuzzy above.

I have no intention of adding an API like the ones you suggest above to
allow the user to specify what size pages to use.  That seems very strange
to me; how should the user (or sysadmin, or application) know what size is
best for the kernel to use to cache files?  Instead, the kernel observes
the usage pattern of the file (through the readahead mechanism) and grows
the allocation size to fit what the kernel thinks will be most effective.

I do honour some of the existing hints that userspace can provide; eg
VM_HUGEPAGE makes the pagefault path allocate PMD sized pages (if it can).

Right, so since VM_HUGEPAGE makes the kernel allocate PMD mapped THP
if it can, then I want to know if the page is actually a PMD mapped
THP or not. The implementation and documentation that I'm adding seem
consistent with that AFAICT, but sorry if I missed something.

So what userspace cares about is that the kernel is mapping the
memory with a PMD entry; it doesn't care whether the file is
being cached in 2MB (or larger) chunks.  So we can drop the 'THP'
from all of this, and just call the bit the PMD mapping bit?