[PATCH RFC] powerpc/mm: Support bootmem for PPC64 again

CONFIG_NO_BOOTMEM is enabled on PPC platforms by default. However,
user might disable that to have bootmem for other purpose. So it's
resonable to support it.

This implements setup_bootmem() which is called by initmem_init()
to setup bootmem node and its free bitmap. In the meantime, the
free pages frames tracked by memblock are released to bootmem.
In theory, we also need similar thing for PPC32 platforms. However,
I don't have machine for testing. So that would be something later
when I have hardware to test.

Signed-off-by: Gavin Shan <redacted>
---
 arch/powerpc/mm/numa.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index 669a15e..288b1b5 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c

@@ -951,6 +951,41 @@ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn)
 	NODE_DATA(nid)->node_spanned_pages = spanned_pages;
 }
 
+#ifndef CONFIG_NO_BOOTMEM
+static void __init setup_bootmem(struct pglist_data *pgdat,
+				 unsigned long start_pfn,
+				 unsigned long end_pfn)
+{
+	unsigned long map_size, addr;
+	phys_addr_t cur[2], range[2];
+	u64 i;
+
+	/* Memory required for bootmem bitmap */
+	map_size = bootmem_bootmap_pages(end_pfn - start_pfn);
+	map_size <<= PAGE_SHIFT;
+	addr = memblock_alloc_try_nid(map_size,
+				      SMP_CACHE_BYTES, pgdat->node_id);
+
+	/* Initialize bootmem allocator */
+	pgdat->bdata = &bootmem_node_data[pgdat->node_id];
+	init_bootmem_node(pgdat, PHYS_PFN(addr), start_pfn, end_pfn);
+
+	/* Release free pages in memblock */
+	range[0] = PFN_PHYS(start_pfn);
+	range[1] = PFN_PHYS(end_pfn);
+	for_each_free_mem_range(i, pgdat->node_id, MEMBLOCK_NONE,
+				&cur[0], &cur[1], NULL) {
+		cur[0] = clamp(cur[0], range[0], range[1]);
+		cur[1] = clamp(cur[1], range[0], range[1]);
+		if (cur[0] >= cur[1])
+			continue;
+
+		memblock_free_early_nid(cur[0], cur[1] - cur[0],
+					pgdat->node_id);
+	}
+}
+#endif /* !CONFIG_NO_BOOTMEM */
+
 void __init initmem_init(void)
 {
 	int nid, cpu;

@@ -977,6 +1012,9 @@ void __init initmem_init(void)
 
 		get_pfn_range_for_nid(nid, &start_pfn, &end_pfn);
 		setup_node_data(nid, start_pfn, end_pfn);
+#ifndef CONFIG_NO_BOOTMEM
+		setup_bootmem(NODE_DATA(nid), start_pfn, end_pfn);
+#endif
 		sparse_memory_present_with_active_regions(nid);
 	}
 

-- 
2.1.0

On Sat, Mar 26, 2016 at 3:03 AM, Gavin Shan [off-list ref] wrote:

CONFIG_NO_BOOTMEM is enabled on PPC platforms by default. However,
user might disable that to have bootmem for other purpose. So it's
resonable to support it.

The changelog could use some rewriting. What other purpose would
bootmem would be used for?

This implements setup_bootmem() which is called by initmem_init()
to setup bootmem node and its free bitmap. In the meantime, the
free pages frames tracked by memblock are released to bootmem.
In theory, we also need similar thing for PPC32 platforms. However,
I don't have machine for testing. So that would be something later
when I have hardware to test.

Balbir Singh

On Fri, 2016-25-03 at 16:03:41 UTC, Gavin Shan wrote:

CONFIG_NO_BOOTMEM is enabled on PPC platforms by default. However,
user might disable that to have bootmem for other purpose. So it's
resonable to support it.

Hi Gavin,

Sorry but I don't understand. We deliberately removed bootmem support, and don't
want it back, it doesn't add anything other than complexity. See 10239733ee86
("powerpc: Remove bootmem allocator").

I wasn't aware there was any way to enable it? If there is we should fix that.

cheers

On Sat, Mar 26, 2016 at 08:23:54PM +1100, Michael Ellerman wrote:

On Fri, 2016-25-03 at 16:03:41 UTC, Gavin Shan wrote:

quoted

CONFIG_NO_BOOTMEM is enabled on PPC platforms by default. However,
user might disable that to have bootmem for other purpose. So it's
resonable to support it.

Hi Gavin,

Sorry but I don't understand. We deliberately removed bootmem support, and don't
want it back, it doesn't add anything other than complexity. See 10239733ee86
("powerpc: Remove bootmem allocator").

I wasn't aware there was any way to enable it? If there is we should fix that.

[Cc Balbir so that I needn't reply separately]

Michael, here's the history: Li, who is in the cc list, backports deferred
page initialization feature from upstream kernel to RHEL 7.2 (3.10.0-357.el7,
or RHEL7.3?). Eventually, she runs into system kernel crash that should be caused
by memory corruption. In RHEL kernel, CONFIG_BOOTMEM and CONFIG_DEFERRED_STRUCT_PAGE_INIT
are enabled at same time. Li observed the issue on pSeries platform.

I jumped in to help understanding what was going on. I tried to reproduce the issue
on PowerNV platform with upstream kernel (v4.5-rc7) and found bootmem is never supported.
In order to trigger the system crash with exact scenario that Li had. I have to support
bootmem first of all.

I guess it's fine to remove bootmem which allocates memory in page granularity. From
the perspective, it's not different from memblock. But user might still need bootmem
supported in code level. The story I told might be not enough to get bootmem to be
supported in code level. If that's the case, please ignore the patch. At least, we
won't support bootmem in long term :-)

Thanks,
Gavin