Re: [PATCH v5 2/3] virtio_pci: Use the DMA API for virtqueues when possible

On Mon, Sep 29, 2014 at 01:55:11PM -0700, Andy Lutomirski wrote:

On Mon, Sep 29, 2014 at 1:49 PM, Benjamin Herrenschmidt
[off-list ref] wrote:

quoted

On Mon, 2014-09-29 at 11:55 -0700, Andy Lutomirski wrote:

quoted

Rusty and Michael, what's the status of this?

The status is that I still think we need *a* way to actually inform the
guest whether the virtio implementation will or will not bypass the
IOMMU. I don't know Xen enough to figure out how to do that and we could
maybe just make it something qemu puts in the device-tree on powerpc
only.

However I dislike making it global or per-bus, we could have a
combination of qemu and HW virtio on the same guest, so I really think
this needs to be a capability of the virtio device.

Or a capability of the PCI slot, somehow.  I don't understand PCI
topology very well.

quoted

I don't completely understand what games Xen is playing here, but from
what I can tell, it's pretty clear that today's qemu implementation
always bypasses any iommu and so should always be exported as such on
all platforms, at least all kvm and pure qemu ones.

Except that I think that PPC is the only platform on which QEMU's code
actually bypasses any IOMMU.  Unless we've all missed something, there
is no QEMU release that will put a virtio device behind an IOMMU on
any platform other than PPC.

I think that is true but it seems that this will be true for x86 for
QEMU 2.2 unless we make some changes there.
Which we might not have the time for since 2.2 is feature frozen
from tomorrow.
Maybe we should disable the IOMMU in 2.2, this is worth considering.

If the eventual solution is to say that virtio 1.0 PCI devices always
respect an IOMMU unless they set a magic flag saying "I'm not real
hardware and I bypass the IOMMU", then I don't really object to that,
except that it'll be a mess if the guest is running Xen.  But even Xen
would (I think) be okay if it actually worked by having a new DMA API
operation that says "this device is magically identity mapped" and
then just teaching Xen to implement that.

But I'm not an OASIS member, so I can't really do this.  I agree that
this issue needs to be addressed somehow, but I don't think it needs
to block these patches.

--Andy

I thought hard about this, I think we are better off waiting till the
next release: there's a chance QEMU will have IOMMU support for KVM x86
then, and this will make it easier to judge which way does the wind
blow.

It seems that we lose nothing substantial keeping the status quo a bit longer,
but if we make an incompatible change in guests now we might
create nasty compatibility headaches going forward.

quoted

quoted

I think that (aside from the trivial DMI/DMA typo) the only real issue
here is that the situation on PPC is ugly.  We're failing to enable
physical virtio hardware on PPC with these patches, but that never
worked anyway.  I don't think that there are any regressions other
than ugliness.

My preference would be to apply the patches as is (or with "DMA"
spelled correctly), and then to:

 - Make sure that all virtio-mmio systems have working DMA ops so that
virtio-mmio can the DMA API

 - Fix the DMA API on s390 (probably easy) and on PPC (not necessarily so easy)

 - Remove the non-DMA-API code, which would be a very small change on
top of these patches.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC