Re: [PATCH 0/5] virtio_ring: per virtqueue DMA device

On Wed, Jan 11, 2023 at 9:33 PM Eli Cohen [off-list ref] wrote:

quoted

From: Jason Wang <redacted>
Sent: Wednesday, 11 January 2023 8:28
To: mst@redhat.com; jasowang@redhat.com
Cc: Eli Cohen <redacted>; gdawar@amd.com;
virtualization@lists.linux-foundation.org; linux-kernel@vger.kernel.org;
tanuj.kamde@amd.com
Subject: [PATCH 0/5] virtio_ring: per virtqueue DMA device

Hi All:

In some cases, the virtqueue could be backed by different devices. One
example is that in the case of vDPA some parent may emualte virtqueue
via vringh. In this case, it would be wrong if we stick with the
physical DMA device for software emulated device, since there's no
easy way for vringh to know about the hardware IOMMU mappings.

So this series tries to introduce per virtqueue DMA device, then
software virtqueues can utilize the transport specific method to
assign appropirate DMA device.

This fixes the crash of mlx5_vdpa + virtio_vdpa when platform IOMMU is
enabled but not in the passthrough mode. The reason for the crash is
that the virito_ring tries to map the control virtqueue into platform
IOMMU but the vringh assumes a direct mapping (PA as IOVA). This is
fixed by advetise the vDPA device that doesnt do DMA (without a DMA
ops). So DMA API can go with the direct mapping then the vringh will
be happy since mlx5_vdpa assuems a direct/identical mapping by
default.

Could you provide instructions how to reproduce the crash you were seeing?

It should be something like:

1) boot host kernel with iommu enabled but not in passthrough mode: I
use intel_iommu=on
2) create vdpa device on top of mlx5_vdpa VF
3) bind the vdpa device to virtio_vdpa

Then I can see the crash.

Thanks

quoted

Please review.

Thanks

Jason Wang (5):
  virtio_ring: per virtqueue dma device
  vdpa: introduce get_vq_dma_device()
  virtio-vdpa: support per vq dma device
  vdpa: set dma mask for vDPA device
  vdpa: mlx5: support per virtqueue dma device

 drivers/vdpa/mlx5/net/mlx5_vnet.c |  11 +++
 drivers/vdpa/vdpa.c               |   5 ++
 drivers/virtio/virtio_ring.c      | 133 +++++++++++++++++++++---------
 drivers/virtio/virtio_vdpa.c      |  13 ++-
 include/linux/vdpa.h              |   6 ++
 include/linux/virtio_ring.h       |  16 ++++
 6 files changed, 141 insertions(+), 43 deletions(-)

--
2.25.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization