David> Don't touch interrupts until both RX and TX queue work is
David> fullydepleted. You seem to have this notion that RX and TX
David> interrupts are seperate. They aren't, even if your device
David> can generate those events individually. Whatever interrupt
David> you get, you shut down all interrupt sources and schedule
David> the ->poll(). Then ->poll() does something like:
This is a digression from spidernet, but what if a device is able to
generate separate MSIs for TX and RX? Some people from IBM have
suggested that it is beneficial for throughput to handle TX work and
RX work for IP-over-InfiniBand in parallel on separate CPUs, and
handling everything through the ->poll() method would defeat this.
- R.
From: Roland Dreier <redacted>
Date: Mon, 21 Aug 2006 17:29:05 -0700
This is a digression from spidernet, but what if a device is able to
generate separate MSIs for TX and RX? Some people from IBM have
suggested that it is beneficial for throughput to handle TX work and
RX work for IP-over-InfiniBand in parallel on separate CPUs, and
handling everything through the ->poll() method would defeat this.
The TX work is so incredibly cheap, relatively speaking, compared
to the full input packet processing path that the RX side runs
that I see no real benefit.
In fact, you might even get better locality due to the way the
locking can be performed if TX reclaim runs inside of ->poll()
quoted
This is a digression from spidernet, but what if a device is able to
generate separate MSIs for TX and RX? Some people from IBM have
suggested that it is beneficial for throughput to handle TX work and
RX work for IP-over-InfiniBand in parallel on separate CPUs, and
handling everything through the ->poll() method would defeat this.
The TX work is so incredibly cheap, relatively speaking, compared
to the full input packet processing path that the RX side runs
that I see no real benefit.
In fact, you might even get better locality due to the way the
locking can be performed if TX reclaim runs inside of ->poll()
We are still comparing these two approaches IPoIB performance under
different configurations. Hopefully the performance work can be done soon.
So we can have some data to share.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638