On Thu, Feb 15, 2018 at 2:28 AM, Burakov, Anatoly <anatoly.burakov@intel.com

wrote:

On 14-Feb-18 8:00 PM, Ravi Kerur wrote:

quoted

Earlier I was focusing only on DMAR errors and I might have said 'it
worked' when I didn't notice them on host when dpdk was started on guest.
When trying to send packets out of that interface from guest I did see DMAR
errors. I am attaching information you requested.  I have enabled
log-level=8 and files contain dpdk EAL/PMD logs as well.

Great, now we're on the same page.

quoted

Snippets below

on host, DMAR fault address from dmesg

[351576.998109] DMAR: DRHD: handling fault status reg 702
[351576.998113] DMAR: [DMA Read] Request device [04:10.0] fault addr
257617000 [fault reason 06] PTE Read access is not set

on guest (dump phys_mem_layout)

Segment 235: phys:0x257600000, len:2097152, virt:0x7fce87e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0
...
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce87e0f4c0
sw_sc_ring=0x7fce87e07380 hw_ring=0x7fce87e17600 dma_addr=0x257617600
PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce89c67d40
sw_sc_ring=0x7fce89c5fc00 hw_ring=0x7fce89c6fe80 dma_addr=0x25406fe80
...

To me this looks like host (i.e. either QEMU or the PF driver) is trying
to do DMA using guest-physical (and not host-physical). I'm not too
well-versed in how QEMU works, but i'm pretty sure that's not supposed to
happen.

Is PF also bound to DPDK, or are you using native Linux ixgbe driver?

Thanks for your help. I cannot use PF with DPDK (vfio-pci), VF interfaces
disappear after it is bound to DPDK. If there is a way to use PF and VF
with DPDK let me know I can try it out. I am not sure how to move forward
on this, Is CPU/IXGBE PF driver playing a role? Following are the versions
I have

lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                56
On-line CPU(s) list:   0-27
Off-line CPU(s) list:  28-55
Thread(s) per core:    1
Core(s) per socket:    14
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU E5-2683 v3 @ 2.00GHz
Stepping:              2
CPU MHz:               2500.610
CPU max MHz:           3000.0000
CPU min MHz:           1200.0000
BogoMIPS:              4000.74
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              35840K
NUMA node0 CPU(s):     0-13
NUMA node1 CPU(s):     14-27

# ethtool -i enp4s0f0
driver: ixgbe
version: 5.3.3
firmware-version: 0x800007b8, 1.1018.0
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Thanks,
Ravi

--
Thanks,
Anatoly