From: Jesper Dangaard Brouer <redacted>
Date: Mon, 03 Dec 2012 15:02:41 +0100

On Thu, 2012-11-29 at 09:06 -0800, Eric Dumazet wrote:

quoted

On Thu, 2012-11-29 at 17:13 +0100, Jesper Dangaard Brouer wrote:

quoted

The major performance bottleneck on NUMA systems, is the mem limit
counter which is based an atomic counter.  This patch removes the
cache-bouncing of the atomic counter, by moving this accounting to be
bound to each CPU.  The LRU list also need to be done per CPU,
in-order to keep the accounting straight.

If fragments belonging together is "sprayed" across CPUs, performance
will still suffer, but due to NIC rxhashing this is not very common.
Correct accounting in this situation is maintained by recording and
"assigning" a CPU to a frag queue when its allocated (caused by the
first packet associated packet).

[...]

quoted

quoted

+/* Need to maintain these resource limits per CPU, else we will kill
+ * performance due to cache-line bouncing
+ */
+struct frag_cpu_limit {
+	atomic_t                mem;
+	struct list_head        lru_list;
+	spinlock_t              lru_lock;
+} ____cacheline_aligned_in_smp;
+

This looks like a big patch introducing a specific infrastructure, while
we already have lib/percpu_counter.c

For the record, I cannot use the lib/percpu_counter, because this
accounting is not kept strictly per CPU, if the fragments are "sprayed"
across CPUs (as described in the commit message above).

The percpu infrastructure allows precise counts and comparisons even
in that case.  It uses the cheap test when possible, and defers to a
more expensive test when necessary.