Re: [PATCH v2] net/virtio: fix an incorrect behavior of device stop/start

-----Original Message-----
From: Bie, Tiwei
Sent: Monday, December 4, 2017 7:20 AM
To: Fischetti, Antonio <redacted>
Cc: dev@dpdk.org; yliu@fridaylinux.org; maxime.coquelin@redhat.com;
jfreimann@redhat.com; stable@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v2] net/virtio: fix an incorrect behavior
of device stop/start

On Sat, Dec 02, 2017 at 12:30:33PM +0800, Tiwei Bie wrote:

quoted

Hi Antonio,

On Sat, Dec 02, 2017 at 01:17:58AM +0800, Fischetti, Antonio wrote:

quoted

Hi All,
I've got an update on this.
I could replicate the same issue by using testpmd + a VM (= Virtual

Machine).

quoted

quoted

The test topology I'm using is:


[Traffic gen]----[NIC port #0]----[testpmd]----[vhost port #2]----+
                                                                  |
                                                                  |

[testpmd in

quoted

quoted

                                                                the

VM]

quoted

quoted

                                                                  |
                                                                  |
[Traffic gen]----[NIC port #1]----[testpmd]----[vhost port #3]----+


So there's no OvS now in the picture: one testpmd running on the

host

quoted

quoted

and one testpmd running on the VM.

The issue is that no packet goes through testpmd in the VM.
It seems this is happening after this patch was upstreamed.

Please note
-----------
To replicate this issue both the next 2 conditions must be met:
 - the traffic is already being sent before launching testpmd in the

VM

quoted

quoted

 - there are at least 2 forwarding lcores.

[...]

quoted

Do you see anything I missed? Or can you reproduce the issue with the
setup I'm using?

Hi Antonio,

Are you using vector Rx in your test? After some further

[Antonio] Hi Tiwei, yes I suppose so.

Below some more details on my testbench to explain why I'm
using this configuration.

With this topology I can replicate as much as possible
my initial OvS-DPDK testbench. In the ovs case I 
had 1 single process on the host (OvS) which spawns 2 
dequeuing threads running on 2 different lcores (please 
note: if they run on the same lcore there's no issue).

So in place of Ovs I'm using 1 testpmd only on the host like:

sudo ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 \
    --socket-mem=1024,1024 \
    --vdev 'eth_vhost0,iface=/tmp/sock0,queues=1' \
    --vdev 'eth_vhost1,iface=/tmp/sock1,queues=1' -- -i  

so the testpmd on the host is seeing 4 phy ports, 2 are real
phy ports and 2 are vdev.

Then I set the forwarding cores

 testpmd> set corelist 2,3

and the port order
 testpmd> set portlist 0,2,1,3
 testpmd> set fwd mac retry
 testpmd> start

I then launch the VM and inside of it I launch testpmd:

./testpmd -c 0x3 -n 4 --socket-mem 512 -- --burst=64 -i \
    --txqflags=0xf00 --disable-hw-vlan

on testpmd console:
 testpmd> set fwd mac retry
 testpmd> start

As the traffic is already running at the line-rate for 
some seconds, for sure all the descs have been consumed.

I'm currently using 1 GB hugepage sizes, but I was
seeing the same issue with OvS when I was using 2 MB 
hugepage sizes.

investigations, I found that the vector Rx could be broken
if backend has consumed all the avail descs before the
device is started. Because in current implementation, the
vector Rx will return immediately without refilling the
avail ring if the used ring is empty. So we have to refill
the avail ring after flushing the elements in the used ring.

Best regards,
Tiwei Bie