powerpc,numa: Memory hotplug to memory-less nodes ?

Hi,

While developing memory hotplug support in QEMU for PoweKVM, I
realized that guest kernel has specific checks to prevent hot addition
of memory to a memory-less node.

I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
has explicit checks to ensure that it returns a nid that has some some
memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
hotplug to a node that currently has zero memory.

Is this limitation by design ?

Regards,
Bharata.
-- 
http://raobharata.wordpress.com/

So will it be correct to say that memory hotplug to memory-less node
isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
for PowerKVM ?

On Mon, Jun 22, 2015 at 10:18 AM, Bharata B Rao [off-list ref] wrote:

Hi,

While developing memory hotplug support in QEMU for PoweKVM, I
realized that guest kernel has specific checks to prevent hot addition
of memory to a memory-less node.

I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
has explicit checks to ensure that it returns a nid that has some some
memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
hotplug to a node that currently has zero memory.

Is this limitation by design ?

Regards,
Bharata.
--
http://raobharata.wordpress.com/

-- 
http://raobharata.wordpress.com/

On 06/23/2015 11:01 PM, Bharata B Rao wrote:

So will it be correct to say that memory hotplug to memory-less node
isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
for PowerKVM ?

I'm not sure if that is correct. It appears that we initialize all online
nodes, even those without spanned_pages, at boot time. This occurs
in setup_node_data() called from initmem_init().

Looking at this I would think that we could add memory to any online node
even if it does not have any spanned_pages. I think an interesting test
we be to check for the node being online instead of checking to see if
it has any memory.

-Nathan

On Mon, Jun 22, 2015 at 10:18 AM, Bharata B Rao [off-list ref] wrote:

quoted

Hi,

While developing memory hotplug support in QEMU for PoweKVM, I
realized that guest kernel has specific checks to prevent hot addition
of memory to a memory-less node.

I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
has explicit checks to ensure that it returns a nid that has some some
memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
hotplug to a node that currently has zero memory.

Is this limitation by design ?

Regards,
Bharata.
--
http://raobharata.wordpress.com/

On 24.06.2015 [07:13:36 -0500], Nathan Fontenot wrote:

On 06/23/2015 11:01 PM, Bharata B Rao wrote:

quoted

So will it be correct to say that memory hotplug to memory-less node
isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
for PowerKVM ?

I'm not sure if that is correct. It appears that we initialize all online
nodes, even those without spanned_pages, at boot time. This occurs
in setup_node_data() called from initmem_init().

Looking at this I would think that we could add memory to any online node
even if it does not have any spanned_pages. I think an interesting test
we be to check for the node being online instead of checking to see if
it has any memory.

I see no *technical* reason we should't be able to hotplug to an
initially memoryless node. I'm not sure it happens in practice under
PowerVM (where we have far less control of the topology anyways). One
aspect of this that I have on my todo list is seeing what SLUB does when
a node goes from memoryless to populated -- as during boot memoryless
nodes get a 'useless' per node structure (early_kmem_cache_node_alloc).

I can look at this a bit under KVM maybe later this week myself to see
what happens in a guest.

-Nish

On Mon, Jun 22, 2015 at 10:18:43AM +0530, Bharata B Rao wrote:

While developing memory hotplug support in QEMU for PoweKVM, I
realized that guest kernel has specific checks to prevent hot addition
of memory to a memory-less node.

I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
has explicit checks to ensure that it returns a nid that has some some
memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
hotplug to a node that currently has zero memory.

Is this limitation by design ?

Sorry to resurrect such an old thread, but did anything come of this?

I've recently bumped into the same limitation. Could we consider 
removing it?

(I couldn't find the original distribution list for this thread. Please 
add whomever is appropriate to the Cc:)

-- 
Reza Arbab