Re: [BUG] rebuild_sched_domains considered dangerous

On Mon, 2011-05-09 at 16:26 -0500, Jesse Larrew wrote:

According the the Power firmware folks, updating the home node of a
virtual cpu happens rather infrequently. The VPHN code currently
checks for topology updates every 60 seconds, but we can poll less
frequently if it helps. I chose 60 second intervals simply because
that's how often they check the topology on s390. ;-)

This just makes me shudder, so you poll the state? Meaning that the vcpu
can actually run 99% of the time on another node?

What's the point of this if the vcpu scheduler can move the vcpu around
much faster?

As for updating the memory topology, there are cases where changing
the home node of a virtual cpu doesn't affect the memory topology. If
it does, there is a separate notification system for memory topology
updates that is independent from the cpu updates. I plan to start
working on a patch set to enable memory topology updates in the kernel
in the coming weeks, but I wanted to get the cpu patches out on the
list so we could start having these debates. :) 

Well, they weren't put out on a list (well maybe on the ppc list but
that's the same as not posting them from my pov), they were merged (and
thus declared done) that's not how you normally start a debate.

I would really like to see both patch-sets together. Also, I'm not at
all convinced its a sane thing to do. Pretty much all NUMA aware
software I know of assumes that CPU<->NODE relations are static,
breaking that in kernel renders all existing software broken.