Re: [PATCH] ext4: slab caches set to SLAB_MEM_SPREAD flags.

2011/11/17 Amit Sahrawat [off-list ref]:

Hi All,

To share with you all, this was picked as part of some review process.
We looked for the details regarding the introduction of
SLAB_MEM_SHARED(http://linux.derkeiler.com/Mailing-Lists/Kernel/2006-02/msg01409.html
- provide the slab cache infrastructure to support cpuset memory
spreading), and further the changes were introduced across all
filesystems - http://lwn.net/Articles/173654/.
Since, we do not have direct access to NUMA architecture machine, so
could not think of verifying the changes - but looking at the
respective changes - the changes in this patch does seems relevant
keeping in sync with all changes.
And, Yes David - these changes will make sense in case of CONFIG_SLAB.

Please share your opinion on the above introductions irrespective of
the code changes done in this patch.

Thanks & Regards,
Amit Sahrawat

On Thu, Nov 17, 2011 at 3:22 AM, David Rientjes [off-list ref] wrote:

quoted

On Wed, 16 Nov 2011, Theodore Tso wrote:

quoted

On Nov 16, 2011, at 10:04 AM, Namjae Jeon wrote:

quoted

If slab caches set to SLAB_MEM_SPREAD flags, The allocation is spread
evenly over all the memory nodes instead of favoring allocation on the
node local to current cpu.

And why do you think this is a good thing?   For mballoc in particular,
the data structures are used immediately and then freed immediately ---
on the local node, so using a non-local memory just makes things worse
in a NUMA system.

I don't think this has the effect that Namjae thinks it does: this is only
useful for CONFIG_SLAB and when you have cpusets enabled with
cpuset.memory_spread_slab set.

To test how useful it is, you should enable CONFIG_SLAB and then mount
cpusets, set cpuset.memory_spread_slab, and create an MPOL_INTERLEAVE
mempolicy over all online nodes.  This will have the same effect as adding
SLAB_MEM_SPREAD to these slab caches (it just doesn't require the
mempolicy) and will be able to quantify the effects without any changes to
the kernel at all.

Regarding ted's question,
We prevent one node being concentrating workload from redundant
alloc/free by balancing workload to other nodes in performance side.
and we can avoid reclaim probability a little by allocation from only
local node also.
slab caches for extents io of btrfs is set to SLAB_MEM_SPREAD flags
with recliam flags.

quoted

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html