Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

9 messages, 6 authors, 2018-07-12 · open the first message on its own page

Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2018-07-11 12:49:58

akpm@linux-foundation.org writes:
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
  $ qemu-system-ppc64 -nographic -vga none -M pseries -m 2G -kernel vmlinux 
  ...
  vmemmap_populate f000000000000000..f000000000004000, node 0
        * f000000000000000..f000000001000000 allocated at c00000007e000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x7e000000
  vmemmap_populate f000000000000000..f000000000008000, node 0
        * f000000000000000..f000000001000000 allocated at c00000007d000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x7d000000
  vmemmap_populate f000000000000000..f00000000000c000, node 0
        * f000000000000000..f000000001000000 allocated at c00000007c000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x7c000000
  vmemmap_populate f000000000000000..f000000000010000, node 0
        * f000000000000000..f000000001000000 allocated at c00000007b000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x7b000000
  vmemmap_populate f000000000000000..f000000000014000, node 0
        * f000000000000000..f000000001000000 allocated at c00000007a000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x7a000000
  vmemmap_populate f000000000000000..f000000000018000, node 0
        * f000000000000000..f000000001000000 allocated at c000000079000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x79000000
  vmemmap_populate f000000000000000..f00000000001c000, node 0
        * f000000000000000..f000000001000000 allocated at c000000078000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x78000000
  vmemmap_populate f000000000000000..f000000000020000, node 0
        * f000000000000000..f000000001000000 allocated at c000000077000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x77000000
  vmemmap_populate f000000000000000..f000000000024000, node 0
        * f000000000000000..f000000001000000 allocated at c000000076000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x76000000
  hash__vmemmap_create_mapping: failed -1

  <repeated many times>

Then there's lots of other warnings about bad page states and eventually
a NULL deref and we panic().


The problem seems to be that we're calling down into
hash__vmemmap_create_mapping() for every call to vmemmap_populate(),
whereas previously we would only call hash__vmemmap_create_mapping()
once because our vmemmap_populated() would return true.

There's actually a comment in sparse_init() that says:

	 * powerpc need to call sparse_init_one_section right after each
	 * sparse_early_mem_map_alloc, so allocate usemap_map at first.

So changing that behaviour does seem to be the problem.

I assume that comment is talking about the fact that we use pfn_valid()
in vmemmap_populated().

I'm not clear on how to fix it though.

Any ideas?

cheers

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Baoquan He <hidden>
Date: 2018-07-11 13:12:25

Hi Michael,

On 07/11/18 at 10:49pm, Michael Ellerman wrote:
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
  $ qemu-system-ppc64 -nographic -vga none -M pseries -m 2G -kernel vmlinux 
  vmemmap_populate f000000000000000..f000000000024000, node 0
        * f000000000000000..f000000001000000 allocated at c000000076000000
  hash__vmemmap_create_mapping: start 0xf000000000000000 size 0x1000000 phys 0x76000000
  hash__vmemmap_create_mapping: failed -1

  <repeated many times>

Then there's lots of other warnings about bad page states and eventually
a NULL deref and we panic().


The problem seems to be that we're calling down into
hash__vmemmap_create_mapping() for every call to vmemmap_populate(),
whereas previously we would only call hash__vmemmap_create_mapping()
once because our vmemmap_populated() would return true.

There's actually a comment in sparse_init() that says:

	 * powerpc need to call sparse_init_one_section right after each
	 * sparse_early_mem_map_alloc, so allocate usemap_map at first.

So changing that behaviour does seem to be the problem.

I assume that comment is talking about the fact that we use pfn_valid()
in vmemmap_populated().

I'm not clear on how to fix it though.
Have you tried reverting that patch and building kernel to test again?
Does it work?

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Oscar Salvador <hidden>
Date: 2018-07-11 13:37:37

On Wed, Jul 11, 2018 at 10:49:58PM +1000, Michael Ellerman wrote:
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
Could you please share the config?
I was not able to find such config in the kernel tree.
-- 
Oscar Salvador
SUSE L3

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Pavel Tatashin <hidden>
Date: 2018-07-11 13:55:59

I am OK, if this patch is removed from Baoquan's series. But, I would
still like to get rid of CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER, I
can work on this in my sparse_init re-write series. ppc64 should
really fallback safely to small chunks allocs, and if it does not
there is some existing bug. Michael please send the config that you
used.

Thank you,
Pavel
On Wed, Jul 11, 2018 at 9:37 AM Oscar Salvador
[off-list ref] wrote:
On Wed, Jul 11, 2018 at 10:49:58PM +1000, Michael Ellerman wrote:
quoted
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
Could you please share the config?
I was not able to find such config in the kernel tree.
--
Oscar Salvador
SUSE L3

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Andrew Morton <akpm@linux-foundation.org>
Date: 2018-07-11 21:13:44

On Wed, 11 Jul 2018 09:55:59 -0400 Pavel Tatashin [off-list ref] wrote:
On Wed, Jul 11, 2018 at 9:37 AM Oscar Salvador
[off-list ref] wrote:
quoted
On Wed, Jul 11, 2018 at 10:49:58PM +1000, Michael Ellerman wrote:
quoted
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
Could you please share the config?
I was not able to find such config in the kernel tree.
(top-posting repaired so I can reply to your email, add other people
and not confuse the heck out of them.  Please don't)
I am OK, if this patch is removed from Baoquan's series. But, I would
still like to get rid of CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER, I
can work on this in my sparse_init re-write series. ppc64 should
really fallback safely to small chunks allocs, and if it does not
there is some existing bug. Michael please send the config that you
used.
OK, I shall drop
mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch for now.

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Stephen Rothwell <hidden>
Date: 2018-07-11 23:47:29

Hi Andrew,

On Wed, 11 Jul 2018 14:13:44 -0700 Andrew Morton [off-list ref] wrote:
OK, I shall drop
mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch for now.
I have dropped it from linux-next today (in case you don't get time).

-- 
Cheers,
Stephen Rothwell

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2018-07-12 04:29:05

Baoquan He [off-list ref] writes:
On 07/11/18 at 10:49pm, Michael Ellerman wrote:
quoted
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.
...
Have you tried reverting that patch and building kernel to test again?
Does it work?
Yes.

Reverting that patch on top of 98be45067040799a801e6ce52d8bf4659a153893
works as before.

cheers

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2018-07-12 04:36:10

Oscar Salvador [off-list ref] writes:
On Wed, Jul 11, 2018 at 10:49:58PM +1000, Michael Ellerman wrote:
quoted
akpm@linux-foundation.org writes:
quoted
The mm-of-the-moment snapshot 2018-07-10-16-50 has been uploaded to

   http://www.ozlabs.org/~akpm/mmotm/
...
quoted
* mm-sparse-add-a-static-variable-nr_present_sections.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing.patch
* mm-sparsemem-defer-the-ms-section_mem_map-clearing-fix.patch
* mm-sparse-add-a-new-parameter-data_unit_size-for-alloc_usemap_and_memmap.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init.patch
* mm-sparse-optimize-memmap-allocation-during-sparse_init-checkpatch-fixes.patch
quoted
* mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch
This seems to be breaking my powerpc pseries qemu boots.

The boot log with some extra debug shows eg:

  $ make pseries_le_defconfig
Could you please share the config?
I was not able to find such config in the kernel tree.
It's a generated config, see:

  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/Makefile?commit=021c91791a5e7e85c567452f1be3e4c2c6cb6063#n315


It's based on pseries_defconfig, which is an actual file in the tree,
but with the endian flipped.

You can see the generated result from a recent build here:

  http://kisskb.ellerman.id.au/kisskb/buildresult/13428803/config/


cheers

Re: Boot failures with "mm/sparse: Remove CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER" on powerpc (was Re: mmotm 2018-07-10-16-50 uploaded)

From: Stephen Rothwell <hidden>
Date: 2018-07-12 06:55:05

Hi all,

On Thu, 12 Jul 2018 09:47:29 +1000 Stephen Rothwell [off-list ref] wrote:
On Wed, 11 Jul 2018 14:13:44 -0700 Andrew Morton [off-list ref] wrote:
quoted
OK, I shall drop
mm-sparse-remove-config_sparsemem_alloc_mem_map_together.patch for now.  
I have dropped it from linux-next today (in case you don't get time).
I am certain I did drop it, but some how it is still in there :-(

I will drop it for tomorrow.
-- 
Cheers,
Stephen Rothwell
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help