Re: [PATCH 4/6] fs: Introduce a per_cpu nr_inodes

On Thu, 2008-11-27 at 10:33 +0100, Peter Zijlstra wrote:

On Thu, 2008-11-27 at 00:32 +0100, Eric Dumazet wrote:

quoted

Avoids cache line ping pongs between cpus and prepare next patch,
because updates of nr_inodes metric dont need inode_lock anymore.

(socket8 bench result : 25s to 20.5s)

Signed-off-by: Eric Dumazet <dada1-fPLkHRcR87vqlBn2x/YWAg@public.gmane.org>
---

quoted

@@ -96,9 +96,40 @@ static DEFINE_MUTEX(iprune_mutex);
  * Statistics gathering..
  */
 struct inodes_stat_t inodes_stat;
+static DEFINE_PER_CPU(int, nr_inodes);
 
 static struct kmem_cache * inode_cachep __read_mostly;
 
+int get_nr_inodes(void)
+{
+	int cpu;
+	int counter = 0;
+
+	for_each_possible_cpu(cpu)
+	    counter += per_cpu(nr_inodes, cpu);
+	if (counter < 0)
+		counter = 0;
+	return counter;
+}

It would be good to get a cpu hotplug handler here and move to
for_each_online_cpu(). People are wanting distro's to be build with
NR_CPUS=4096.

Also, this trade-off between global vs per_cpu only works if
get_nr_inodes() is called significantly less than nr_inodes is changed.

With it being called from writeback that might not be true for all
workloads. One thing you can do about it is use the regular per-cpu
counter stuff, which allows you to do an approximation of the global
number (it also does all the hotplug stuff for you already).