Re: [PATCH] cpusets: Make cpus_allowed and mems_allowed masks hotplug invariant

Hi Raghu,

On 10/08/2014 08:24 PM, Raghavendra KT wrote:

On Wed, Oct 8, 2014 at 12:37 PM, Preeti U Murthy
[off-list ref] wrote:

quoted

There are two masks associated with cpusets. The cpus/mems_allowed
and effective_cpus/mems. On the legacy hierarchy both these masks
are consistent with each other. This is the intersection of their
value and the currently active cpus. This means that we destroy the
original values set in these masks on each cpu/mem hot unplug operation.
        As a consequence when we hot plug back the cpus/mems, the tasks
no longer run on them and performance degrades, inspite of having
resources to run on.

This effect is not seen in the default hierarchy since the
allowed and effective masks are distinctly maintained.
allowed masks are never touched once configured and effective masks
alone are hotplug variant.

This patch replicates the above design even for the legacy hierarchy,
so that:

1. Tasks always run on the cpus/memory nodes that they are allowed to run on
as long as they are online. The allowed masks are hotplug invariant.

2. When all cpus/memory nodes in a cpuset are hot unplugged out, the tasks
are moved to their nearest ancestor which has resources to run on.

Hi Preeti,

I may be missing some thing here could you please explain when do we get
tasks move out of a cpuset after this patch and why it is even necessary?

On the legacy hierarchy the tasks are moved to their parents cpusets if
the cpuset to which they were initially bound becomes empty. What the
patch does has nothing to do with moving tasks when the cpuset to which
they are bound becomes empty.The point 2 above was mentioned to merely
state that this part of the behavior is not really changed with the
patch. The patch merely ensures that the original cpuset configuration
is not messed with during hotplug operations.

IIUC, with default hierarchy we should never hit a case where we have empty
effective cpuset and hence remove_tasks_in_empty_cpuset should never happen. no?

if my assumption is correct then we should remove
remove_tasks_in_empty_cpuset itself...

remove_tasks_in_empty_cpuset() is called on the legacy hierarchy when
the cpuset becomes empty, hence we require it. But you are right its not
called on the default hierarchy.

Regards
Preeti U Murthy