On 17/10/2023 16:42, Yishai Hadas wrote:

This series introduce a vfio driver over virtio devices to support the
legacy interface functionality for VFs.

Background, from the virtio spec [1].
--------------------------------------------------------------------
In some systems, there is a need to support a virtio legacy driver with
a device that does not directly support the legacy interface. In such
scenarios, a group owner device can provide the legacy interface
functionality for the group member devices. The driver of the owner
device can then access the legacy interface of a member device on behalf
of the legacy member device driver.

For example, with the SR-IOV group type, group members (VFs) can not
present the legacy interface in an I/O BAR in BAR0 as expected by the
legacy pci driver. If the legacy driver is running inside a virtual
machine, the hypervisor executing the virtual machine can present a
virtual device with an I/O BAR in BAR0. The hypervisor intercepts the
legacy driver accesses to this I/O BAR and forwards them to the group
owner device (PF) using group administration commands.
--------------------------------------------------------------------

The first 6 patches are in the virtio area and handle the below:
- Fix common config map for modern device as was reported by Michael Tsirkin.
- Introduce the admin virtqueue infrastcture.
- Expose the layout of the commands that should be used for
   supporting the legacy access.
- Expose APIs to enable upper layers as of vfio, net, etc
   to execute admin commands.

The above follows the virtio spec that was lastly accepted in that area
[1].

The last 3 patches are in the vfio area and handle the below:
- Expose some APIs from vfio/pci to be used by the vfio/virtio driver.
- Introduce a vfio driver over virtio devices to support the legacy
   interface functionality for VFs.

The series was tested successfully over virtio-net VFs in the host,
while running in the guest both modern and legacy drivers.

[1]
https://github.com/oasis-tcs/virtio-spec/commit/03c2d32e5093ca9f2a17797242fbef88efe94b8c

Changes from V0: https://www.spinics.net/lists/linux-virtualization/msg63802.html

Virtio:
- Fix the common config map size issue that was reported by Michael
   Tsirkin.
- Do not use vp_dev->vqs[] array upon vp_del_vqs() as was asked by
   Michael, instead skip the AQ specifically.
- Move admin vq implementation into virtio_pci_modern.c as was asked by
   Michael.
- Rename structure virtio_avq to virtio_pci_admin_vq and some extra
   corresponding renames.
- Remove exported symbols virtio_pci_vf_get_pf_dev(),
   virtio_admin_cmd_exec() as now callers are local to the module.
- Handle inflight commands as part of the device reset flow.
- Introduce APIs per admin command in virtio-pci as was asked by Michael.

Vfio:
- Change to use EXPORT_SYMBOL_GPL instead of EXPORT_SYMBOL for
   vfio_pci_core_setup_barmap() and vfio_pci_iowrite#xxx() as pointed by
   Alex.
- Drop the intermediate patch which prepares the commands and calls the
   generic virtio admin command API (i.e. virtio_admin_cmd_exec()).
- Instead, call directly to the new APIs per admin command that are
   exported from Virtio - based on Michael's request.
- Enable only virtio-net as part of the pci_device_id table to enforce
   upon binding only what is supported as suggested by Alex.
- Add support for byte-wise access (read/write) over the device config
   region as was asked by Alex.
- Consider whether MSIX is practically enabled/disabled to choose the
   right opcode upon issuing read/write admin command, as mentioned
   by Michael.
- Move to use VIRTIO_PCI_CONFIG_OFF instead of adding some new defines
   as was suggested by Michael.
- Set the '.close_device' op to vfio_pci_core_close_device() as was
   pointed by Alex.
- Adapt to Vfio multi-line comment style in a few places.
- Add virtualization@lists.linux-foundation.org in the MAINTAINERS file
   to be CCed for the new driver as was suggested by Jason.

Yishai

Feng Liu (5):
   virtio-pci: Fix common config map for modern device
   virtio: Define feature bit for administration virtqueue
   virtio-pci: Introduce admin virtqueue
   virtio-pci: Introduce admin command sending function
   virtio-pci: Introduce admin commands

Yishai Hadas (4):
   virtio-pci: Introduce APIs to execute legacy IO admin commands
   vfio/pci: Expose vfio_pci_core_setup_barmap()
   vfio/pci: Expose vfio_pci_iowrite/read##size()
   vfio/virtio: Introduce a vfio driver over virtio devices

  MAINTAINERS                            |   7 +
  drivers/vfio/pci/Kconfig               |   2 +
  drivers/vfio/pci/Makefile              |   2 +
  drivers/vfio/pci/vfio_pci_core.c       |  25 ++
  drivers/vfio/pci/vfio_pci_rdwr.c       |  38 +-
  drivers/vfio/pci/virtio/Kconfig        |  15 +
  drivers/vfio/pci/virtio/Makefile       |   4 +
  drivers/vfio/pci/virtio/main.c         | 577 +++++++++++++++++++++++++
  drivers/virtio/virtio.c                |  37 +-
  drivers/virtio/virtio_pci_common.c     |  14 +
  drivers/virtio/virtio_pci_common.h     |  20 +-
  drivers/virtio/virtio_pci_modern.c     | 441 ++++++++++++++++++-
  drivers/virtio/virtio_pci_modern_dev.c |  24 +-
  include/linux/vfio_pci_core.h          |  20 +
  include/linux/virtio.h                 |   8 +
  include/linux/virtio_config.h          |   4 +
  include/linux/virtio_pci_admin.h       |  18 +
  include/linux/virtio_pci_modern.h      |   5 +
  include/uapi/linux/virtio_config.h     |   8 +-
  include/uapi/linux/virtio_pci.h        |  66 +++
  20 files changed, 1295 insertions(+), 40 deletions(-)
  create mode 100644 drivers/vfio/pci/virtio/Kconfig
  create mode 100644 drivers/vfio/pci/virtio/Makefile
  create mode 100644 drivers/vfio/pci/virtio/main.c
  create mode 100644 include/linux/virtio_pci_admin.h

Hi Michael,

Did you have the chance to review the virtio part of that series ?

IMO, we addressed all your notes on V0, I would be happy to get your 
feedback on V1 before sending V2.

In my TO-DO list for V2, have for now the below minor items.
Virtio:
Patch #6: Fix a krobot note where it needs to include the H file as part 
of the export symbols C file.
Vfio:
#patch #9: Rename the 'ops' variable to drop the 'acc' and potentially 
some rename in the description of the module with regards to 'family'.

Alex,
Are you fine to leave the provisioning of the VF including the control 
of its transitional capability in the device hands as was suggested by 
Jason ?
Any specific recommendation following the discussion in the ML, for the 
'family' note ?

Once I'll have the above feedback I may prepare and send V2.

Yishai

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization