"Michael S. Tsirkin" [off-list ref] wrote on 11/24/2011 09:44:31 PM:

quoted

As far as I can see, ixgbe binds queues to physical cpu, so let

consider:

quoted

vhost thread transmits packets of flow A on processor M
during packet transmission, ixgbe driver programs the card to
deliver the packet of flow A to queue/cpu M through flow director
(see ixgbe_atr())
vhost thread then receives packet of flow A with from M
...
vhost thread transmits packets of flow A on processor N
ixgbe driver programs the flow director to change the delivery of
flow A to queue N ( cpu N )
vhost thread then receives packet of flow A with from N
...

So, for a single flow A, we may get different queue mappings. Using
rxhash instead may solve this issue.

Or better, transmit a single flow from a single vhost thread.

If packets of a single flow get spread over different CPUs,
they will get reordered and things are not going to work well.

My testing so far shows that guest sends on (e.g.) TXQ#2
only, which is handled by vhost#2; and this doesn't change
for the entire duration of the test. Incoming keeps
changing for different packets but become same with
this patch. To iterate, I have not seen the following:

"
vhost thread transmits packets of flow A on processor M
...
vhost thread transmits packets of flow A on processor N
"

- KK