On Thu, Jun 07, 2018 at 09:17:42AM -0700, Stephen Hemminger wrote:
On Thu, 7 Jun 2018 18:41:31 +0300
"Michael S. Tsirkin" [off-list ref] wrote:
quoted
quoted
quoted
Why would DPDK care what we do in the kernel? Isn't it just slapping
vfio-pci on the netdevs it sees?
Alex, you are correct for Intel devices; but DPDK on Azure is not Intel based.,.
The DPDK support uses:
* Mellanox MLX5 which uses the Infinband hooks to do DMA directly to
userspace. This means VF netdev device must exist and be visible.
* Slow path using kernel netvsc device, TAP and BPF to get exception
path packets to userspace.
* A autodiscovery mechanism that to set all this up that relies on
2 device model and sysfs.
Could you describe what does it look for exactly? What will break if
instead of MLX5 being a child of the PV, it's a child of the failover
device?
So in DPDK there is an internal four device model:
1. failsafe is like failover in your model
2. TAP is used like netvsc in kernel
3. MLX5 is the VF
4. vdev_netvsc is a pseudo device whose only reason to exist
is to glue everything together.
Digging deeper inside...
Vdev_netvsc does:
* driver is started in a convuluted way off device arguments
* probe routine for driver runs
- scans list of kernel interfaces in sysfs
- matches those using VMBUS
Could you tell a bit more what does this step entail?
- skip netvsc devices that have an IPV4 route
* scan for PCI devices that have same MAC address as kernel netvsc
devices discovered in previous step
* add these interfaces to arguments to failsafe
Then failsafe configures based on arguments on device
The code works but is specific to the Azure hardware model, and exposes lots
of things to application that it should not have to care about.
If you try and walk through this code in DPDK, you will see why I have developed
a dislike for high levels of indirection.
Thanks that was helpful! I'll try to poke at it next week. Just from
the description it seems the kernel is merely used to locate the MAC
address through sysfs and that for this DPDK code to keep working the
hidden device must be hidden from it in sysfs - is that a fair summary?
--
MST