Re: How to debug packet sends to virtual functions

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mats Liljegren
Sent: Tuesday, February 04, 2014 10:45 AM
To: jigsaw
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] How to debug packet sends to virtual functions

Hi Qinglai,

Thanks for the response!

My previous attempt was with ixgbe loaded in the host. I also needed to load
ixgbevf, but this seems to be because of a short-coming in libvirt. Maybe
loading ixgbevf and then unbind it when running the guest is what causes my
problems. I could receive packets with this setup, but not transmit. I used
extra debug to syslog, and it showed that packets was placed in the transmit
queue, but these packets was never sent.

I'll see if I can get this working without loading ixgbevf in the host.

What instructions did you follow to get this working? Did you do any
deviation from the instructions?

Regards
Mats

On Tue, Feb 4, 2014 at 11:26 AM, jigsaw [off-list ref] wrote:

quoted

Hi Mats,

I've tried vf with 82599EB and it works fine. As long as the VF is
visible in guest, DPDK's VF driver should work just as ixgbevf, which
shares more or less the same code.

I don't understand why you would expect DPDK@guest to work as VF,
while the host has no ixgbe loaded.
To make further debug I'd suggest compile ixgbe driver with your own
syslogs. At least you will be able to see the signalling between vf
and ixgbe drivers.

-Qinglai


On Tue, Feb 4, 2014 at 12:08 PM, Mats Liljegren
[off-list ref] wrote:

quoted

This is my fourth mail in my desperate attempt to get DPDK running in
KVM and no comments so far, not even any "it works for me". Am I the
only one crazy enough to believe that this can be done?

Anyway, out of desperation I tried to get it running without having
ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed
the instructions in the Programmer's Guide, chapter "Setting Up a KVM
Virtual Machine Monitor", using the PMD version of the instructions.

I get as far as being able to see my four virtual functions in the
guest using "lspci". But starting the DPDK application gives me the
following error:

PMD:    The MAC address is not valid.
        The most likely cause of this error is that the VM host
        has not assigned a valid MAC address to this VF device.
        Please consult the DPDK Release Notes (FAQ section) for
        a possible solution to this problem.

This may be true, but without any kernel modules loaded, how am I
supposed to change any MAC addresses? Can this be done from within
DPDK?

As a side-note, I did try to load ixgbevf in the guest, but it
produced no interfaces. There was no error messages in the syslog
though.

Is it possible to get X540 working in a guest or should I switch hardware?

Since the instructions assumes I know the command line to KVM to
start my guest (which I do not), I cannot followed them precisely. I
use virsh and XML file, and maybe I've misunderstood how to translate
the pci-assign parameter to XML code. I currently use a <hostdev>
entry, but I've also tried <interface type='hostdev'>. Neither has
been working for me so far, though the <interface> version got me as
far as being able to receive packets at least, but not transmitting.

Regards
Mats


On Mon, Feb 3, 2014 at 12:13 PM, Mats Liljegren
[off-list ref] wrote:

quoted

Never mind, I was hit by the infamous MAC spoofing... I got it
working on both the host and the guest using ixgbevf driver, so
apparently the cables are correctly attached.

Using DPDK is still no-go. It can receive packets, but when sending
the packets the function returns success, but the driver reports
nothing (i.e. no errors, no sent packets, no nothing, except for
received packets of course).

What could cause this behavior?

Regards
Mats

On Fri, Jan 31, 2014 at 7:30 PM, Mats Liljegren
[off-list ref] wrote:

quoted

I have a follow-up on this:

ixgbe version 3.13.10-k
ixgbevf version 2.7.12-k

(These are what was provided by Ubuntu 13.10)

I tried the following sequence on the host, before starting the guest:
1) sudo rmmod ixgbe
2) sudo modprobe ixgbe max_vfs=2
3) sudo ifconfig em1 up  # This is the physical function
4) sudo ifconfig em1_0 192.168.2.2  # This is the virtual function
5) ping 192.168.2.1

I can see that the ping request reaches its target, and a reply is
sent back. But this reply is not received by the ping shell command.

Why?

Regards,
Mats

On Wed, Jan 29, 2014 at 6:56 PM, Mats Liljegren
[off-list ref] wrote:

quoted

I'm trying to get a modified version of the l2fwd example running,
and have problems with packets being silently thrown away. I can
receive packets, and my printf's indicates that the packets are
being sent to the correct port, using correct MAC address as
source address. And still, the packets are lost.

Since the port is a virtual function, it seems like I cannot use
tcpdump on it to see the network traffic. There is nothing coming
out of the cable (activity light not flashing, the receiving end
running tcpdump does not see any traffic).

I'm using two X540 with two virtual functions each. The
application runs in a KVM/Qemu environmen.

Any suggestions how to debug this?

Regards,
Mats

Hi Mats

Basically, there is no way to change MAC address of a VF device from within the VM, you have to do it on the host. So, whatever the FAQ section describes (i.e. ip link ...) has to be done on the host, not inside the VM. That is, you have to load ixgbe kernel module on the host with max_vfs= set to whatever number of VF devices you need, and then go through each VF device and assign a MAC address via the "ip link" command, as described in the FAQ section of the DPDK Release Notes. You can then load either ixgbevf or DPDK VF drivers on the guest, and they will initialize correctly.

Best regards,
Anatoly Burakov
DPDK SW Engineer
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.