Here's a dumb simple implementation of fake NUMA nodes for PowerPC. Fake
NUMA nodes can be specified using the following command line option
numa=fake=<node range>
node range is of the format <range1>,<range2>,...<rangeN>
Each of the rangeX parameters is passed using memparse(). I find the patch
useful for fake NUMA emulation on my simple PowerPC machine. I've tested it
on a non-numa box with the following arguments
numa=fake=1G
numa=fake=1G,2G
name=fake=1G,512M,2G
numa=fake=1500M,2800M mem=3500M
numa=fake=1G mem=512M
numa=fake=1G mem=1G
So this doesn't appear to allow one to assign cpus to fake nodes? Do
all cpus just get assigned to node 0 with numa=fake?
Yes, they all appear on node 0. We could have tweaks to distribute CPU's
as well.
A different approach that occurs to me is to use kexec with a doctored
device tree (i.e. with the ibm,associativity properties modified to
reflect your desired topology). Perhaps a little bit obscure, but it
seems more flexible.
That would be interesting, but it always means that we need to run
kexec, which might involve two boots.
--
Warm Regards,
Balbir Singh
Linux Technology Center
IBM, ISTL
From: David Rientjes <rientjes@google.com> Date: 2007-12-07 23:13:14
On Sat, 8 Dec 2007, Balbir Singh wrote:
Yes, they all appear on node 0. We could have tweaks to distribute CPU's
as well.
You're going to want to distribute the cpu's based on how they match up
physically with the actual platform that you're running on. x86_64 does
this already and it makes fake NUMA more useful because it matches the
real-life case more often.
Yes, they all appear on node 0. We could have tweaks to distribute CPU's
as well.
You're going to want to distribute the cpu's based on how they match up
physically with the actual platform that you're running on. x86_64 does
Could you explain this better, how does it match up CPU's with fake NUMA
memory? Is there some smartness there? I'll try and look at the code and
also see what I can do for PowerPC
this already and it makes fake NUMA more useful because it matches the
real-life case more often.
Yes, I agree, but I don't want that to be the first step for fake NUMA
nodes on PowerPC. I think we can incrementally add features.
--
Warm Regards,
Balbir Singh
Linux Technology Center
IBM, ISTL
From: David Rientjes <rientjes@google.com> Date: 2007-12-08 04:46:05
On Sat, 8 Dec 2007, Balbir Singh wrote:
quoted
You're going to want to distribute the cpu's based on how they match up
physically with the actual platform that you're running on. x86_64 does
Could you explain this better, how does it match up CPU's with fake NUMA
memory? Is there some smartness there? I'll try and look at the code and
also see what I can do for PowerPC
numa_cpumask_lookup_table[] would return the correct cpumask for the fake
node index. Then all the code that uses node_to_cpumask() in generic
kernel code like the scheduler and VM still preserve their true NUMA
affinity that matches the underlying hardware. I tried to make x86_64
fake NUMA as close to the real thing as possible.
You also probably want to make all you changes dependent on
CONFIG_NUMA_EMU like the x86_64 case. That'll probably be helpful as you
extend this tool more and more.
David