On Wed, Aug 16, 2006 at 01:00:31PM +0400, Evgeniy Polyakov wrote:

On Wed, Aug 16, 2006 at 09:48:08AM +0100, Christoph Hellwig (hch@infradead.org) wrote:

quoted

quoted

Doesn't alloc_pages() automatically switches to alloc_pages_node() or
alloc_pages_current()?

That's not what's wanted.  If you have a slow interconnect you always want
to allocate memory on the node the network device is attached to.

There is drawback here - if data was allocated on CPU wheere NIC is
"closer" and then processed on different CPU it will cost more than 
in case where buffer was allocated on CPU where it will be processed.

But from other point of view, most of the adapters preallocate set of
skbs, and with msi-x help there will be a possibility to bind irq and
processing to the CPU where data was origianlly allocated.

The case we've benchmarked (spidernet) is the common preallocated case.
For allocate on demand I'd expect the slab allocator to get things right.
We do have the irq on the right node, not through MSI but due to the odd
interreupt architecture of the Cell blades.

So I would like to know how to determine which node should be used for
allocation. Changes of __get_user_pages() to alloc_pages_node() are
trivial.

The patches I have add the node field to struct net_device and use it.
It's set via alloc_netdev_node, a function I add and for the normal case
of PCI adapters the node arguments comes from pcibus_to_node().  It's
arguable we should add a alloc_foodeve_pdev variant that hids that detail,
but I'm not entirely sure about whether it's worth the effort.