[PATCH RFC 0/2] mm: Enable page parallel initialisation for Power

From: Li Zhang <redacted>

Uptream has supported page parallel initialisation for X86 and the
boot time is improved greately. Some tests have been done for Power.

Here is the result I have done with different memory size.

* 4GB memory:
    boot time is as the following: 
    with patch vs without patch: 10.4s vs 24.5s
    boot time is improved 57%
* 200GB memory: 
   boot time looks the same with and without patches.
   boot time is about 38s
* 32TB memory: 
   boot time looks the same with and without patches 
   boot time is about 160s.
   The boot time is much shorter than X86 with 24TB memory.
   From community discussion, it costs about 694s for X86 24T system.

From code view, parallel initialisation improve the performance by

deferring memory initilisation to kswap with N kthreads, it should
improve the performance therotically. 

From the test result, On X86, performance is improved greatly with huge

memory. But on Power platform, it is improved greatly with less than 
100GB memory. For huge memory, it is not improved greatly. But it saves 
the time with several threads at least, as the following information 
shows(32TB system log):

[   22.648169] node 9 initialised, 16607461 pages in 280ms
[   22.783772] node 3 initialised, 23937243 pages in 410ms
[   22.858877] node 6 initialised, 29179347 pages in 490ms
[   22.863252] node 2 initialised, 29179347 pages in 490ms
[   22.907545] node 0 initialised, 32049614 pages in 540ms
[   22.920891] node 15 initialised, 32212280 pages in 550ms
[   22.923236] node 4 initialised, 32306127 pages in 550ms
[   22.923384] node 12 initialised, 32314319 pages in 550ms
[   22.924754] node 8 initialised, 32314319 pages in 550ms
[   22.940780] node 13 initialised, 33353677 pages in 570ms
[   22.940796] node 11 initialised, 33353677 pages in 570ms
[   22.941700] node 5 initialised, 33353677 pages in 570ms
[   22.941721] node 10 initialised, 33353677 pages in 570ms
[   22.941876] node 7 initialised, 33353677 pages in 570ms
[   22.944946] node 14 initialised, 33353677 pages in 570ms
[   22.946063] node 1 initialised, 33345485 pages in 580ms

It saves the time about 550*16 ms at least, although it can be ignore to compare 
the boot time about 160 seconds. What's more, the boot time is much shorter 
on Power even without patches than x86 for huge memory machine. 

So this patchset is still necessary to be enabled for Power. 

Li Zhang (2):
  mm: meminit: initialise more memory for inode/dentry hash tables in
    early boot
  Enable page parallel initialisation

 arch/powerpc/Kconfig |  1 +
 mm/page_alloc.c      | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

-- 
2.1.0

From: Li Zhang <redacted>

This patch is based on Mel Gorman's old patch in the mailing list,
https://lkml.org/lkml/2015/5/5/280 which is dicussed but it is
fixed with a completion to wait for all memory initialised in
page_alloc_init_late(). It is to fix the oom problem on X86
with 24TB memory which allocates memory in late initialisation.
But for Power platform with 32TB memory, it causes a call trace
in vfs_caches_init->inode_init() and inode hash table needs more
memory.
So this patch allocates 1GB for 0.25TB/node for large system
as it is mentioned in https://lkml.org/lkml/2015/5/1/627

This call trace is found on Power with 32TB memory, 1024CPUs, 16nodes.
The log from dmesg as the following:

[    0.091780] Dentry cache hash table entries: 2147483648 (order: 18,
17179869184 bytes)
[    2.891012] vmalloc: allocation failure, allocated 16021913600 of
17179934720 bytes
[    2.891034] swapper/0: page allocation failure: order:0,
mode:0x2080020
[    2.891038] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-0-ppc64
[    2.891041] Call Trace:
[    2.891046] [c0000000012bfa00] [c0000000007c4a50]
                .dump_stack+0xb4/0xb664 (unreliable)
[    2.891051] [c0000000012bfa80] [c0000000001f93d4]
                .warn_alloc_failed+0x114/0x160
[    2.891054] [c0000000012bfb30] [c00000000023c204]
                .__vmalloc_area_node+0x1a4/0x2b0
[    2.891058] [c0000000012bfbf0] [c00000000023c3f4]
                .__vmalloc_node_range+0xe4/0x110
[    2.891061] [c0000000012bfc90] [c00000000023c460]
                .__vmalloc_node+0x40/0x50
[    2.891065] [c0000000012bfd10] [c000000000b67d60]
                .alloc_large_system_hash+0x134/0x2a4
[    2.891068] [c0000000012bfdd0] [c000000000b70924]
                .inode_init+0xa4/0xf0
[    2.891071] [c0000000012bfe60] [c000000000b706a0]
                .vfs_caches_init+0x80/0x144
[    2.891074] [c0000000012bfef0] [c000000000b35208]
                .start_kernel+0x40c/0x4e0
[    2.891078] [c0000000012bff90] [c000000000008cfc]
                start_here_common+0x20/0x4a4
[    2.891080] Mem-Info:

Signed-off-by: Li Zhang <redacted>
---
 mm/page_alloc.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 838ca8bb..4847f25 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c

@@ -293,13 +293,20 @@ static inline bool update_defer_init(pg_data_t *pgdat,
 				unsigned long pfn, unsigned long zone_end,
 				unsigned long *nr_initialised)
 {
+	unsigned long max_initialise;
+
 	/* Always populate low zones for address-contrained allocations */
 	if (zone_end < pgdat_end_pfn(pgdat))
 		return true;
+	/*
+	* Initialise at least 2G of a node but also take into account that
+	* two large system hashes that can take up 1GB for 0.25TB/node.
+	*/
+	max_initialise = max(2UL << (30 - PAGE_SHIFT),
+		(pgdat->node_spanned_pages >> 8));
 
-	/* Initialise at least 2G of the highest zone */
 	(*nr_initialised)++;
-	if (*nr_initialised > (2UL << (30 - PAGE_SHIFT)) &&
+	if ((*nr_initialised > max_initialise) &&
 	    (pfn & (PAGES_PER_SECTION - 1)) == 0) {
 		pgdat->first_deferred_pfn = pfn;
 		return false;

-- 
2.1.0

From: Li Zhang <redacted>

Parallel initialisation has been enabled for X86,
boot time is improved greatly.
On Power8, for small memory, it is improved greatly.
Here is the result from my test on Power8 platform:

For 4GB memory: 57% is improved
For 50GB memory: 22% is improve

Signed-off-by: Li Zhang <redacted>
---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index e4824fd..83073c2 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig

@@ -158,6 +158,7 @@ config PPC
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select HAVE_ARCH_SECCOMP_FILTER
 	select ARCH_HAS_UBSAN_SANITIZE_ALL
+	select ARCH_SUPPORTS_DEFERRED_STRUCT_PAGE_INIT
 
 config GENERIC_CSUM
 	def_bool CPU_LITTLE_ENDIAN

-- 
2.1.0

On Thu, Mar 03, 2016 at 03:01:40PM +0800, Li Zhang wrote:

From: Li Zhang <redacted>

This patch is based on Mel Gorman's old patch in the mailing list,
https://lkml.org/lkml/2015/5/5/280 which is dicussed but it is
fixed with a completion to wait for all memory initialised in
page_alloc_init_late(). It is to fix the oom problem on X86
with 24TB memory which allocates memory in late initialisation.
But for Power platform with 32TB memory, it causes a call trace
in vfs_caches_init->inode_init() and inode hash table needs more
memory.
So this patch allocates 1GB for 0.25TB/node for large system
as it is mentioned in https://lkml.org/lkml/2015/5/1/627

Acked-by: Mel Gorman <redacted>

-- 
Mel Gorman
SUSE Labs

On Thu, Mar 03, 2016 at 03:01:41PM +0800, Li Zhang wrote:

From: Li Zhang <redacted>

Parallel initialisation has been enabled for X86,
boot time is improved greatly.
On Power8, for small memory, it is improved greatly.
Here is the result from my test on Power8 platform:

For 4GB memory: 57% is improved
For 50GB memory: 22% is improve

Signed-off-by: Li Zhang <redacted>

Acked-by: Mel Gorman <redacted>

-- 
Mel Gorman
SUSE Labs

On 03/03/2016 12:31 PM, Li Zhang wrote:

From: Li Zhang <redacted>

This patch is based on Mel Gorman's old patch in the mailing list,
https://lkml.org/lkml/2015/5/5/280 which is dicussed but it is

Typo here ....................................^^^^^^^^

fixed with a completion to wait for all memory initialised in
page_alloc_init_late(). It is to fix the oom problem on X86

You can just write *out of memory* instead of *oom* or put them in
capitals.

with 24TB memory which allocates memory in late initialisation.
But for Power platform with 32TB memory, it causes a call trace
in vfs_caches_init->inode_init() and inode hash table needs more
memory.
So this patch allocates 1GB for 0.25TB/node for large system
as it is mentioned in https://lkml.org/lkml/2015/5/1/627

I am wondering how its going to impact other architectures.

This call trace is found on Power with 32TB memory, 1024CPUs, 16nodes.
The log from dmesg as the following:

[    0.091780] Dentry cache hash table entries: 2147483648 (order: 18,
17179869184 bytes)
[    2.891012] vmalloc: allocation failure, allocated 16021913600 of
17179934720 bytes
[    2.891034] swapper/0: page allocation failure: order:0,
mode:0x2080020
[    2.891038] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-0-ppc64
[    2.891041] Call Trace:
[    2.891046] [c0000000012bfa00] [c0000000007c4a50]
                .dump_stack+0xb4/0xb664 (unreliable)
[    2.891051] [c0000000012bfa80] [c0000000001f93d4]
                .warn_alloc_failed+0x114/0x160
[    2.891054] [c0000000012bfb30] [c00000000023c204]
                .__vmalloc_area_node+0x1a4/0x2b0
[    2.891058] [c0000000012bfbf0] [c00000000023c3f4]
                .__vmalloc_node_range+0xe4/0x110
[    2.891061] [c0000000012bfc90] [c00000000023c460]
                .__vmalloc_node+0x40/0x50
[    2.891065] [c0000000012bfd10] [c000000000b67d60]
                .alloc_large_system_hash+0x134/0x2a4
[    2.891068] [c0000000012bfdd0] [c000000000b70924]
                .inode_init+0xa4/0xf0
[    2.891071] [c0000000012bfe60] [c000000000b706a0]
                .vfs_caches_init+0x80/0x144
[    2.891074] [c0000000012bfef0] [c000000000b35208]
                .start_kernel+0x40c/0x4e0
[    2.891078] [c0000000012bff90] [c000000000008cfc]
                start_here_common+0x20/0x4a4
[    2.891080] Mem-Info:

The dmesg output here needs some formatting.

Does this change need to be tested on all architectures ? 

On Thu, Mar 3, 2016 at 5:41 PM, Anshuman Khandual
[off-list ref] wrote:

On 03/03/2016 12:31 PM, Li Zhang wrote:

quoted

From: Li Zhang <redacted>

This patch is based on Mel Gorman's old patch in the mailing list,
https://lkml.org/lkml/2015/5/5/280 which is dicussed but it is

Typo here ....................................^^^^^^^^

Sorry, I will correct it.

quoted

fixed with a completion to wait for all memory initialised in
page_alloc_init_late(). It is to fix the oom problem on X86

You can just write *out of memory* instead of *oom* or put them in
capitals.

OK

quoted

with 24TB memory which allocates memory in late initialisation.
But for Power platform with 32TB memory, it causes a call trace
in vfs_caches_init->inode_init() and inode hash table needs more
memory.
So this patch allocates 1GB for 0.25TB/node for large system
as it is mentioned in https://lkml.org/lkml/2015/5/1/627

I am wondering how its going to impact other architectures.

I just saw some discussion for X86 in https://lkml.org/lkml/2015/5/1/627.
And some tests are done for X86 with huge memory.

From code view, this allocate more memory in early initialisation, it may

cause boot time a little bit.

quoted

This call trace is found on Power with 32TB memory, 1024CPUs, 16nodes.
The log from dmesg as the following:

[    0.091780] Dentry cache hash table entries: 2147483648 (order: 18,
17179869184 bytes)
[    2.891012] vmalloc: allocation failure, allocated 16021913600 of
17179934720 bytes
[    2.891034] swapper/0: page allocation failure: order:0,
mode:0x2080020
[    2.891038] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-0-ppc64
[    2.891041] Call Trace:
[    2.891046] [c0000000012bfa00] [c0000000007c4a50]
                .dump_stack+0xb4/0xb664 (unreliable)
[    2.891051] [c0000000012bfa80] [c0000000001f93d4]
                .warn_alloc_failed+0x114/0x160
[    2.891054] [c0000000012bfb30] [c00000000023c204]
                .__vmalloc_area_node+0x1a4/0x2b0
[    2.891058] [c0000000012bfbf0] [c00000000023c3f4]
                .__vmalloc_node_range+0xe4/0x110
[    2.891061] [c0000000012bfc90] [c00000000023c460]
                .__vmalloc_node+0x40/0x50
[    2.891065] [c0000000012bfd10] [c000000000b67d60]
                .alloc_large_system_hash+0x134/0x2a4
[    2.891068] [c0000000012bfdd0] [c000000000b70924]
                .inode_init+0xa4/0xf0
[    2.891071] [c0000000012bfe60] [c000000000b706a0]
                .vfs_caches_init+0x80/0x144
[    2.891074] [c0000000012bfef0] [c000000000b35208]
                .start_kernel+0x40c/0x4e0
[    2.891078] [c0000000012bff90] [c000000000008cfc]
                start_here_common+0x20/0x4a4
[    2.891080] Mem-Info:

The dmesg output here needs some formatting.

quoted

Signed-off-by: Li Zhang <redacted>
---
 mm/page_alloc.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 838ca8bb..4847f25 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c

@@ -293,13 +293,20 @@ static inline bool update_defer_init(pg_data_t *pgdat,
                              unsigned long pfn, unsigned long zone_end,
                              unsigned long *nr_initialised)
 {
+     unsigned long max_initialise;
+
      /* Always populate low zones for address-contrained allocations */
      if (zone_end < pgdat_end_pfn(pgdat))
              return true;
+     /*
+     * Initialise at least 2G of a node but also take into account that
+     * two large system hashes that can take up 1GB for 0.25TB/node.
+     */
+     max_initialise = max(2UL << (30 - PAGE_SHIFT),
+             (pgdat->node_spanned_pages >> 8));

-     /* Initialise at least 2G of the highest zone */
      (*nr_initialised)++;
-     if (*nr_initialised > (2UL << (30 - PAGE_SHIFT)) &&
+     if ((*nr_initialised > max_initialise) &&

Does this change need to be tested on all architectures ?

Currently, this feature is only supported for X86 in upstream.
It can be tested more in the future if some architectures need to enable it.

-- 

Best Regards
-Li

On 03/03/2016 08:01 AM, Li Zhang wrote:

The indentation is wrong here.

+	max_initialise = max(2UL << (30 - PAGE_SHIFT),
+		(pgdat->node_spanned_pages >> 8));
 
-	/* Initialise at least 2G of the highest zone */
 	(*nr_initialised)++;
-	if (*nr_initialised > (2UL << (30 - PAGE_SHIFT)) &&
+	if ((*nr_initialised > max_initialise) &&
 	    (pfn & (PAGES_PER_SECTION - 1)) == 0) {
 		pgdat->first_deferred_pfn = pfn;
 		return false;

On Fri, Mar 4, 2016 at 4:48 PM, Vlastimil Babka [off-list ref] wrote:

On 03/03/2016 08:01 AM, Li Zhang wrote:

quoted

--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c

@@ -293,13 +293,20 @@ static inline bool update_defer_init(pg_data_t *pgdat,
                              unsigned long pfn, unsigned long zone_end,
                              unsigned long *nr_initialised)
 {
+     unsigned long max_initialise;
+
      /* Always populate low zones for address-contrained allocations */
      if (zone_end < pgdat_end_pfn(pgdat))
              return true;
+     /*
+     * Initialise at least 2G of a node but also take into account that
+     * two large system hashes that can take up 1GB for 0.25TB/node.
+     */

The indentation is wrong here.

Thanks for reviewing, I will fix it in next version.

quoted

+     max_initialise = max(2UL << (30 - PAGE_SHIFT),
+             (pgdat->node_spanned_pages >> 8));

-     /* Initialise at least 2G of the highest zone */
      (*nr_initialised)++;
-     if (*nr_initialised > (2UL << (30 - PAGE_SHIFT)) &&
+     if ((*nr_initialised > max_initialise) &&
          (pfn & (PAGES_PER_SECTION - 1)) == 0) {
              pgdat->first_deferred_pfn = pfn;
              return false;

-- 

Best Regards
-Li