On Tue, 24 Nov 2009, Peter P Waskiewicz Jr wrote:

On Tue, 2009-11-24 at 13:56 -0800, Thomas Gleixner wrote:

quoted

On Tue, 24 Nov 2009, David Miller wrote:

quoted

From: Thomas Gleixner <redacted>
Date: Tue, 24 Nov 2009 12:07:35 +0100 (CET)

quoted

And what does the kernel do with this information and why are we not
using the existing device/numa_node information ?

It's a different problem space Thomas.

If the device lives on NUMA node X, we still end up wanting to
allocate memory resources (RX ring buffers) on other NUMA nodes on a
per-queue basis.

Otherwise a network card's forwarding performance is limited by the
memory bandwidth of a single NUMA node, and on a multiqueue cards we
therefore fare much better by allocating each device RX queue's memory
resources on a different NUMA node.

It is this NUMA usage that PJ is trying to export somehow to userspace
so that irqbalanced and friends can choose the IRQ cpu masks more
intelligently.

So you need a preferred irq mask information on a per IRQ basis and
that mask is not restricted to the CPUs of a single NUMA node, right ?

Just to clarify, I need a preferred CPU mask on a per IRQ basis.  And
yes, that mask may not be restricted to the CPUs of a single NUMA node.
But in the normal case, the mask will be restricted to CPUs of a single
node.

Right, but the normal case does not help much if we need to consider
the special case of multiple nodes affected which requires another
cpumask in irq_desc. That's what I really want to avoid.

I at least understand the exact problem you guys want to solve. Will
think more about it.

Thanks,

	tglx