Re: [PATCH net-next 3/3] virtio-net: clean tx descriptors from rx napi

On Fri, Apr 7, 2017 at 5:10 PM, Michael S. Tsirkin [off-list ref] wrote:

On Fri, Apr 07, 2017 at 04:59:58PM -0400, Willem de Bruijn wrote:

quoted

On Fri, Apr 7, 2017 at 3:28 PM, Michael S. Tsirkin [off-list ref] wrote:

quoted

On Mon, Apr 03, 2017 at 01:02:13AM -0400, Willem de Bruijn wrote:

quoted

On Sun, Apr 2, 2017 at 10:47 PM, Michael S. Tsirkin [off-list ref] wrote:

quoted

On Sun, Apr 02, 2017 at 04:10:12PM -0400, Willem de Bruijn wrote:

quoted

From: Willem de Bruijn <willemb@google.com>

Amortize the cost of virtual interrupts by doing both rx and tx work
on reception of a receive interrupt if tx napi is enabled. With
VIRTIO_F_EVENT_IDX, this suppresses most explicit tx completion
interrupts for bidirectional workloads.

Signed-off-by: Willem de Bruijn <willemb@google.com>

This is a popular approach, but I think this will only work well if tx
and rx interrupts are processed on the same CPU and if tx queue is per
cpu.  If they target different CPUs or if tx queue is used from multiple
CPUs they will conflict on the shared locks.

Yes. As a result of this discussion I started running a few vcpu affinity tests.

quoted

This can even change dynamically as CPUs/queues are reconfigured.
How about adding a flag and skipping the tx poll if there's no match?

I suspect that even with the cache invalidations this optimization
will be an improvement over handling all tx interrupts in the tx napi
handler. I will get the datapoint for that.

That said, we can make this conditional. What flag exactly do you
propose? Compare raw_smp_processor_id() in the rx softint with one
previously stored in the napi tx callback?

I'm not sure. Another idea is to check vi->affinity_hint_set.
If set we know rq and sq are on the same CPU.

I was not aware of that flag, thanks. Yes, that looks like it should work.