On 11/25/2011 10:58 AM, Krishna Kumar2 wrote:

jasowang[off-list ref]  wrote on 11/24/2011 06:30:52 PM:

quoted

quoted

quoted

On Thu, Nov 24, 2011 at 01:47:14PM +0530, Krishna Kumar wrote:

quoted

It was reported that the macvtap device selects a
different vhost (when used with multiqueue feature)
for incoming packets of a single connection. Use
packet hash first. Patch tested on MQ virtio_net.

So this is sure to address the problem, why exactly does this happen?
Does your device spread a single flow across multiple RX queues? Would
not that cause trouble in the TCP layer?
It would seem that using the recorded queue should be faster with
less cache misses. Before we give up on that, I'd
like to understand why it's wrong. Do you know?

I am using ixgbe. From what I briefly saw, ixgbe_alloc_rx_buffers
calls skb_record_rx_queue when a skb is allocated. When a packet
is received (ixgbe_alloc_rx_buffers), it sets rxhash. The
recorded value is different for most skbs when I ran a single
stream TCP stream test (does skbs move between the rx_rings?).

Yes, it moves. It depends on last processor or tx queue who transmits
the packets of this stream. Because ixgbe select tx queue based on the
processor id, so if vhost thread transmits skbs on different processors,
the skb of a single stream may comes from different rx ring.

But I don't see transmit going on different queues,
only incoming.

- KK

Maybe I'm not clear enough, I mean the processor of host and tx queue of 
ixgbe.  So you would see, for a single vhost thread, as it moves among 
host cpus, it would use different tx queues of ixgbe. I think if you pin 
the vhost thread on host cpu, you may get consistent rx queue no.