Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

7 messages, 4 authors, 2021-01-04 · open the first message on its own page

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Dave Hansen <hidden>
Date: 2021-01-04 19:19:57

On 12/21/20 8:30 AM, Liang Li wrote:
quoted hunk
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_64BIT
 	PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+	PG_zero,
 #endif
 	__NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Matthew Wilcox <willy@infradead.org>
Date: 2021-01-04 19:30:50

On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote:
On 12/21/20 8:30 AM, Liang Li wrote:
quoted
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_64BIT
 	PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+	PG_zero,
 #endif
 	__NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
I was going to object to that too, but I think the entire approach is
flawed and needs to be thrown out.  It just nukes the caches in extremely
subtle and hard to measure ways, lowering overall system performance.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Dan Williams <hidden>
Date: 2021-01-04 19:45:19

On Mon, Jan 4, 2021 at 11:28 AM Matthew Wilcox [off-list ref] wrote:
On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote:
quoted
On 12/21/20 8:30 AM, Liang Li wrote:
quoted
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_64BIT
    PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+   PG_zero,
 #endif
    __NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
I was going to object to that too, but I think the entire approach is
flawed and needs to be thrown out.  It just nukes the caches in extremely
subtle and hard to measure ways, lowering overall system performance.
At a minimum the performance analysis should at least try to quantify
that externalized cost. Certainly that overhead went somewhere. Maybe
if this overhead was limited to run when the CPU would otherwise go
idle, but that might mean it never runs in practice...
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Dave Hansen <hidden>
Date: 2021-01-04 19:52:37

On 1/4/21 11:27 AM, Matthew Wilcox wrote:
On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote:
quoted
On 12/21/20 8:30 AM, Liang Li wrote:
quoted
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
 #endif
 #ifdef CONFIG_64BIT
 	PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+	PG_zero,
 #endif
 	__NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
I was going to object to that too, but I think the entire approach is
flawed and needs to be thrown out.  It just nukes the caches in extremely
subtle and hard to measure ways, lowering overall system performance.
Yeah, it certainly can't be the default, but it *is* useful for thing
where we know that there are no cache benefits to zeroing close to where
the memory is allocated.

The trick is opting into it somehow, either in a process or a VMA.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: David Hildenbrand <hidden>
Date: 2021-01-04 20:13:06

Am 04.01.2021 um 20:52 schrieb Dave Hansen [off-list ref]:

On 1/4/21 11:27 AM, Matthew Wilcox wrote:
quoted
quoted
On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote:
On 12/21/20 8:30 AM, Liang Li wrote:
quoted
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
#endif
#ifdef CONFIG_64BIT
   PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+    PG_zero,
#endif
   __NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
I was going to object to that too, but I think the entire approach is
flawed and needs to be thrown out.  It just nukes the caches in extremely
subtle and hard to measure ways, lowering overall system performance.
Yeah, it certainly can't be the default, but it *is* useful for thing
where we know that there are no cache benefits to zeroing close to where
the memory is allocated.

The trick is opting into it somehow, either in a process or a VMA.
The patch set is mostly trying to optimize starting a new process. So process/vma doesn‘t really work.

I still wonder if using tmpfs/shmem cannot somehow be used to cover the original use case of starting a new vm fast (or rebooting an existing one involving restarting the process).

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Dave Hansen <hidden>
Date: 2021-01-04 23:03:22

On 1/4/21 12:11 PM, David Hildenbrand wrote:
quoted
Yeah, it certainly can't be the default, but it *is* useful for
thing where we know that there are no cache benefits to zeroing
close to where the memory is allocated.

The trick is opting into it somehow, either in a process or a VMA.
The patch set is mostly trying to optimize starting a new process. So
process/vma doesn‘t really work.
Let's say you have a system-wide tunable that says: pre-zero pages and
keep 10GB of them around.  Then, you opt-in a process to being allowed
to dip into that pool with a process-wide flag or an madvise() call.
You could even have the flag be inherited across execve() if you wanted
to have helper apps be able to set the policy and access the pool like
how numactl works.

Dan makes a very good point about using filesystems for this, though.
It wouldn't be rocket science to set up a special tmpfs mount just for
VM memory and pre-zero it from userspace.  For qemu, you'd need to teach
the management layer to hand out zeroed files via mem-path=.  Heck, if
you taught MADV_FREE how to handle tmpfs, you could even pre-zero *and*
get the memory back quickly if those files ended up over-sized somehow.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

From: Dan Williams <hidden>
Date: 2021-01-04 23:26:44

On Mon, Jan 4, 2021 at 12:11 PM David Hildenbrand [off-list ref] wrote:
quoted
Am 04.01.2021 um 20:52 schrieb Dave Hansen [off-list ref]:

On 1/4/21 11:27 AM, Matthew Wilcox wrote:
quoted
quoted
On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote:
On 12/21/20 8:30 AM, Liang Li wrote:
quoted
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -137,6 +137,9 @@ enum pageflags {
#endif
#ifdef CONFIG_64BIT
   PG_arch_2,
+#endif
+#ifdef CONFIG_PREZERO_PAGE
+    PG_zero,
#endif
   __NR_PAGEFLAGS,
I don't think this is worth a generic page->flags bit.

There's a ton of space in 'struct page' for pages that are in the
allocator.  Can't we use some of that space?
I was going to object to that too, but I think the entire approach is
flawed and needs to be thrown out.  It just nukes the caches in extremely
subtle and hard to measure ways, lowering overall system performance.
Yeah, it certainly can't be the default, but it *is* useful for thing
where we know that there are no cache benefits to zeroing close to where
the memory is allocated.

The trick is opting into it somehow, either in a process or a VMA.
The patch set is mostly trying to optimize starting a new process. So process/vma doesn‘t really work.

I still wonder if using tmpfs/shmem cannot somehow be used to cover the original use case of starting a new vm fast (or rebooting an existing one involving restarting the process).
If it's rebooting a VM then file-backed should be able to skip the
zeroing because the stale data exposure is only to itself. If the
memory is being repurposed to a new VM then the file needs to be
reallocated / re-zeroed just like the anonymous case.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help