On 21.05.2014 [14:58:12 -0400], Tejun Heo wrote:

Hello,

On Wed, May 21, 2014 at 09:16:27AM -0500, Christoph Lameter wrote:

quoted

On Mon, 19 May 2014, Nishanth Aravamudan wrote:

quoted

I'm seeing a panic at boot with this change on an LPAR which actually
has no Node 0. Here's what I think is happening:

start_kernel
    ...
    -> setup_per_cpu_areas
        -> pcpu_embed_first_chunk
            -> pcpu_fc_alloc
                -> ___alloc_bootmem_node(NODE_DATA(cpu_to_node(cpu), ...
    -> smp_prepare_boot_cpu
        -> set_numa_node(boot_cpuid)

So we panic on the NODE_DATA call. It seems that ia64, at least, uses
pcpu_alloc_first_chunk rather than embed. x86 has some code to handle
early calls of cpu_to_node (early_cpu_to_node) and sets the mapping for
all CPUs in setup_per_cpu_areas().

Maybe we can switch ia64 too embed? Tejun: Why are there these
dependencies?

quoted

Thoughts? Does that mean we need something similar to x86 for powerpc?

I'm missing context to properly understand what's going on but the
specific allocator in use shouldn't matter.  e.g. x86 can use both
embed and page allocators.  If the problem is that the arch is
accessing percpu memory before percpu allocator is initialized and the
problem was masked before somehow, the right thing to do would be
removing those premature percpu accesses.  If early percpu variables
are really necessary, doing similar early_percpu thing as in x86 would
be necessary.

The early access is in the arch's pcpu_alloc_bootmem. On x86, rather
than using NODE_DATA(cpu_to_node), it uses (in pcpu_alloc_bootmem),
early_cpu_to_node(cpu) with their custom logic.

The issue is that cpu_to_node, if USE_PERCPU_NUMA_NODE_ID is defined
(which it is for NUMA powerpc, x86, ia64), is that cpu_to_node uses the
percpu area, which data isn't initialized yet.

So I guess powerpc needs the same treatment as x86.

Thanks,
Nish