With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <redacted>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
-----Original Message-----
From: iommu-bounces at lists.linux-foundation.org [mailto:iommu-
bounces at lists.linux-foundation.org] On Behalf Of Antonios Motakis
Sent: Thursday, June 05, 2014 10:33 PM
To: alex.williamson at redhat.com; kvmarm at lists.cs.columbia.edu;
iommu at lists.linux-foundation.org
Cc: kvm at vger.kernel.org; eric.auger at linaro.org; open list;
will.deacon at arm.com; a.rigo at virtualopensystems.com; Yoder Stuart-B08248;
moderated list:ARM SMMU DRIVER; Antonios Motakis;
tech at virtualopensystems.com; christoffer.dall at linaro.org
Subject: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <redacted>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index
Why mention this as an IOMMU capability when IOMMU doesn't support it? Also, wouldn't this depend on the GIC IP version? What are we gaining by adding this IOMMU capability list?
-Varun
From: Christoffer Dall <hidden> Date: 2014-06-08 10:31:36
On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
quoted hunk
With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <redacted>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,if(smmu_domain->root_cfg.smmu->features&ARM_SMMU_FEAT_COHERENT_WALK)caps|=IOMMU_CAP_CACHE_COHERENCY;-caps|=IOMMU_CAP_NOEXEC;+caps|=IOMMU_CAP_NOEXEC|IOMMU_CAP_INTR_REMAP;return!!(cap&caps);}
--
1.8.3.2
What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
somewhere I can look at? (A quick scan of the Linux souce code doesn't
reveal much, and I'm not sure if this is purely MSI related or what...)
Thanks,
-Christoffer
On Sun, Jun 08, 2014 at 12:31:29PM +0200, Christoffer Dall wrote:
On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
quoted
With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <redacted>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,if(smmu_domain->root_cfg.smmu->features&ARM_SMMU_FEAT_COHERENT_WALK)caps|=IOMMU_CAP_CACHE_COHERENCY;-caps|=IOMMU_CAP_NOEXEC;+caps|=IOMMU_CAP_NOEXEC|IOMMU_CAP_INTR_REMAP;return!!(cap&caps);}
--
1.8.3.2
What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
somewhere I can look at? (A quick scan of the Linux souce code doesn't
reveal much, and I'm not sure if this is purely MSI related or what...)
The flag was introduced for x86 IOMMUs to detect whether an IOMMU in the
system has and enables interrupt remapping to allow safe device
assignment to KVM guests. Without interrupt remapping a malicious guest
could attack the host with MSIs from the attached device.
How are PCI MSIs implemented on ARM with SMMU and GIC enabled. MSIs are
only memory dma transactions in the end, is it guaranteed on ARM that a
device only sends MSI transactions it is allowed to?
Joerg
From: Will Deacon <hidden> Date: 2014-06-16 15:13:35
On Mon, Jun 16, 2014 at 03:53:44PM +0100, Joerg Roedel wrote:
On Sun, Jun 08, 2014 at 12:31:29PM +0200, Christoffer Dall wrote:
quoted
On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
quoted
With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <redacted>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,if(smmu_domain->root_cfg.smmu->features&ARM_SMMU_FEAT_COHERENT_WALK)caps|=IOMMU_CAP_CACHE_COHERENCY;-caps|=IOMMU_CAP_NOEXEC;+caps|=IOMMU_CAP_NOEXEC|IOMMU_CAP_INTR_REMAP;return!!(cap&caps);}
--
1.8.3.2
What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
somewhere I can look at? (A quick scan of the Linux souce code doesn't
reveal much, and I'm not sure if this is purely MSI related or what...)
The flag was introduced for x86 IOMMUs to detect whether an IOMMU in the
system has and enables interrupt remapping to allow safe device
assignment to KVM guests. Without interrupt remapping a malicious guest
could attack the host with MSIs from the attached device.
How are PCI MSIs implemented on ARM with SMMU and GIC enabled. MSIs are
only memory dma transactions in the end, is it guaranteed on ARM that a
device only sends MSI transactions it is allowed to?
MSIs look just like memory accesses made by the device, so the SMMU will
translate them to point at the GIC ITS (doorbell). The ITS then has tables
to work out how to route the MSI.
So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the SMMU can
translate MSIs to point somewhere else, then the ARM SMMU can always do it.
If it's supposed to indicate that the actual MSI payload can be
filtered/routed, then that requires the GIC ITS.
The part I'm unsure of is how VFIO knows where to map the MSIs to. That
requires knowledge of the physical and virtual doorbell pages -- is that
discoverable in the API?
Will
On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
MSIs look just like memory accesses made by the device, so the SMMU
will translate them to point at the GIC ITS (doorbell). The ITS then
has tables to work out how to route the MSI.
So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
always do it. If it's supposed to indicate that the actual MSI
payload can be filtered/routed, then that requires the GIC ITS.
The part I'm unsure of is how VFIO knows where to map the MSIs to.
That requires knowledge of the physical and virtual doorbell pages --
is that discoverable in the API?
VFIO does not care about the actual routing, it only cares that the
device can not send interrupts it is not allowed to send (e.g.
interrupts to vectors used by other devices or, on x86, exception vectors).
If that is guaranteed by the SMMU or the GIC ITS hardware and driver
then it is fine to set this flag.
Joerg
From: Will Deacon <hidden> Date: 2014-06-16 15:25:32
On Mon, Jun 16, 2014 at 04:21:58PM +0100, Joerg Roedel wrote:
On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
quoted
MSIs look just like memory accesses made by the device, so the SMMU
will translate them to point at the GIC ITS (doorbell). The ITS then
has tables to work out how to route the MSI.
So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
always do it. If it's supposed to indicate that the actual MSI
payload can be filtered/routed, then that requires the GIC ITS.
The part I'm unsure of is how VFIO knows where to map the MSIs to.
That requires knowledge of the physical and virtual doorbell pages --
is that discoverable in the API?
VFIO does not care about the actual routing, it only cares that the
device can not send interrupts it is not allowed to send (e.g.
interrupts to vectors used by other devices or, on x86, exception vectors).
If that is guaranteed by the SMMU or the GIC ITS hardware and driver
then it is fine to set this flag.
Ok, thanks. In which case, I think this is really a combined property of
the SMMU and the interrupt controller, so we might need some extra code
so that the SMMU can check that the interrupt controller for the device
is also capable of interrupt remapping.
Will
From: Alex Williamson <hidden> Date: 2014-06-16 15:31:12
On Mon, 2014-06-16 at 17:21 +0200, Joerg Roedel wrote:
On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
quoted
MSIs look just like memory accesses made by the device, so the SMMU
will translate them to point at the GIC ITS (doorbell). The ITS then
has tables to work out how to route the MSI.
So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
always do it. If it's supposed to indicate that the actual MSI
payload can be filtered/routed, then that requires the GIC ITS.
The part I'm unsure of is how VFIO knows where to map the MSIs to.
That requires knowledge of the physical and virtual doorbell pages --
is that discoverable in the API?
VFIO does not care about the actual routing, it only cares that the
device can not send interrupts it is not allowed to send (e.g.
interrupts to vectors used by other devices or, on x86, exception vectors).
If that is guaranteed by the SMMU or the GIC ITS hardware and driver
then it is fine to set this flag.
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
Ok, thanks. In which case, I think this is really a combined property of
the SMMU and the interrupt controller, so we might need some extra code
so that the SMMU can check that the interrupt controller for the device
is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons
on x86. Interrupt remapping is purely implemented in the IOMMU there, so
on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X,
Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces@lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list; iommu at lists.linux-foundation.org; alex.williamson at redhat.com; moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
Ok, thanks. In which case, I think this is really a combined property
of the SMMU and the interrupt controller, so we might need some extra
code so that the SMMU can check that the interrupt controller for the
device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces@lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces@lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list; iommu at lists.linux-foundation.org; alex.williamson at redhat.com; moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
Ok, thanks. In which case, I think this is really a combined property
of the SMMU and the interrupt controller, so we might need some extra
code so that the SMMU can check that the interrupt controller for the
device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Sorry there was a type,
The question is:
How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
-----Original Message-----
From: Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:16 AM
To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces@lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces@lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list; iommu at lists.linux-foundation.org; alex.williamson at redhat.com; moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
Ok, thanks. In which case, I think this is really a combined property
of the SMMU and the interrupt controller, so we might need some extra
code so that the SMMU can check that the interrupt controller for the
device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
From: Alex Williamson <hidden> Date: 2014-06-26 19:02:08
On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
Sorry there was a type,
The question is:
How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
All interrupts are configured via ioctl, not MSI config space or the
MSI-X vector table in MMIO space. VFIO protects the MSI config area by
virtualizing it (you can't actually write the physical enable bit or
address/data through VFIO). The MSI-X vector table is protected by
preventing read, write, or mmap access to it. QEMU provides further
virtualization above the basics provided by VFIO. We really can't
guarantee that devices don't have backdoors to configure these though.
See the realtek quirk in QEMU for an example of a device that has such a
backdoor. That's why we require interrupt remapping, so that a device
that does this can only hurt the guest, and require the user to opt-out
if they feel they have a sufficiently trusted guest. Thanks,
Alex
-----Original Message-----
From: Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:16 AM
To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu [mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list; iommu at lists.linux-foundation.org; alex.williamson at redhat.com; moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
quoted
Ok, thanks. In which case, I think this is really a combined property
of the SMMU and the interrupt controller, so we might need some extra
code so that the SMMU can check that the interrupt controller for the
device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Thanks for the clarification Alex, That?s exactly my point, why are we relying on QEMU or something else to emulate the MSI space when we can directly give access to devices using ITS (of course with a small emulation code).
This way we are also benefited from all ITS services like VCPU migration etc.
What about non QEMU VFIO users, for example, if I wanted to use VFIO to assign a device to a user process I don't need to depend on QEMU. I thought this is one of the main goals of vfio to make it independent of hypervisors.
Thanks,
Tirumalesh.
-----Original Message-----
From: Alex Williamson [mailto:alex.williamson at redhat.com]
Sent: Thursday, June 26, 2014 12:00 PM
To: Chalamarla, Tirumalesh
Cc: Joerg Roedel; Will Deacon; kvm at vger.kernel.org; open list; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
Sorry there was a type,
The question is:
How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
All interrupts are configured via ioctl, not MSI config space or the MSI-X vector table in MMIO space. VFIO protects the MSI config area by virtualizing it (you can't actually write the physical enable bit or address/data through VFIO). The MSI-X vector table is protected by preventing read, write, or mmap access to it. QEMU provides further virtualization above the basics provided by VFIO. We really can't guarantee that devices don't have backdoors to configure these though.
See the realtek quirk in QEMU for an example of a device that has such a backdoor. That's why we require interrupt remapping, so that a device that does this can only hurt the guest, and require the user to opt-out if they feel they have a sufficiently trusted guest. Thanks,
Alex
-----Original Message-----
From: Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:16 AM
To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com;
stuart.yoder at freescale.com; iommu at lists.linux-foundation.org;
tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated
list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu
[mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Chalamarla,
Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com;
stuart.yoder at freescale.com; iommu at lists.linux-foundation.org;
tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated
list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu
[mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg
Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list;
iommu at lists.linux-foundation.org; alex.williamson at redhat.com;
moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com;
kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
quoted
Ok, thanks. In which case, I think this is really a combined
property of the SMMU and the interrupt controller, so we might need
some extra code so that the SMMU can check that the interrupt
controller for the device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
From: Alex Williamson <hidden> Date: 2014-06-26 19:38:05
On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
Thanks for the clarification Alex, That?s exactly my point, why are we relying on QEMU or something else to emulate the MSI space when we can directly give access to devices using ITS (of course with a small emulation code).
This way we are also benefited from all ITS services like VCPU migration etc.
I have no idea what ITS is.
What about non QEMU VFIO users, for example, if I wanted to use VFIO to assign a device to a user process I don't need to depend on QEMU. I thought this is one of the main goals of vfio to make it independent of hypervisors.
Where did QEMU become a requirement? Maybe I'm missing something in the
ARM part of the conversation that got chopped off, but this is exactly
why we have the VFIO/QEMU split that we do. VFIO provides basic
virtualization for config space and restricts access to other areas that
users shouldn't be allowed to change. QEMU is just one example of a
userspace VFIO driver. QEMU takes the decomposed device exposed through
the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no
dependency on QEMU. Thanks,
Alex
-----Original Message-----
From: Alex Williamson [mailto:alex.williamson at redhat.com]
Sent: Thursday, June 26, 2014 12:00 PM
To: Chalamarla, Tirumalesh
Cc: Joerg Roedel; Will Deacon; kvm at vger.kernel.org; open list; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
quoted
Sorry there was a type,
The question is:
How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
All interrupts are configured via ioctl, not MSI config space or the MSI-X vector table in MMIO space. VFIO protects the MSI config area by virtualizing it (you can't actually write the physical enable bit or address/data through VFIO). The MSI-X vector table is protected by preventing read, write, or mmap access to it. QEMU provides further virtualization above the basics provided by VFIO. We really can't guarantee that devices don't have backdoors to configure these though.
See the realtek quirk in QEMU for an example of a device that has such a backdoor. That's why we require interrupt remapping, so that a device that does this can only hurt the guest, and require the user to opt-out if they feel they have a sufficiently trusted guest. Thanks,
Alex
quoted
-----Original Message-----
From: Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:16 AM
To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com;
stuart.yoder at freescale.com; iommu at lists.linux-foundation.org;
tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated
list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu
[mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Chalamarla,
Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: kvm at vger.kernel.org; open list; alex.williamson at redhat.com;
stuart.yoder at freescale.com; iommu at lists.linux-foundation.org;
tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated
list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: kvmarm-bounces at lists.cs.columbia.edu
[mailto:kvmarm-bounces at lists.cs.columbia.edu] On Behalf Of Joerg
Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: stuart.yoder at freescale.com; kvm at vger.kernel.org; open list;
iommu at lists.linux-foundation.org; alex.williamson at redhat.com;
moderated list:ARM SMMU DRIVER; tech at virtualopensystems.com;
kvmarm at lists.cs.columbia.edu; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
quoted
Ok, thanks. In which case, I think this is really a combined
property of the SMMU and the interrupt controller, so we might need
some extra code so that the SMMU can check that the interrupt
controller for the device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
From: Will Deacon <hidden> Date: 2014-06-27 08:50:26
On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote:
On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
quoted
Thanks for the clarification Alex, That?s exactly my point, why are we
relying on QEMU or something else to emulate the MSI space when we can
directly give access to devices using ITS (of course with a small
emulation code). This way we are also benefited from all ITS services
like VCPU migration etc.
I have no idea what ITS is.
ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller).
I agree that we will need an ITS emulation if we want to use MSIs in the
guest, and I believe that Marc (CC'd) had already started thinking about
that.
quoted
What about non QEMU VFIO users, for example, if I wanted to use VFIO to
assign a device to a user process I don't need to depend on QEMU. I
thought this is one of the main goals of vfio to make it independent of
hypervisors.
Where did QEMU become a requirement? Maybe I'm missing something in the
ARM part of the conversation that got chopped off, but this is exactly
why we have the VFIO/QEMU split that we do. VFIO provides basic
virtualization for config space and restricts access to other areas that
users shouldn't be allowed to change. QEMU is just one example of a
userspace VFIO driver. QEMU takes the decomposed device exposed through
the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no
dependency on QEMU. Thanks,
I also don't understand the QEMU part here. The MSI emulation would be in
the kernel, just like the GICv2 emulation that we already have. For
userspace drivers, wouldn't you just use eventfd rather than bother with
emulating MSIs?
Finally, the interrupt remapping part is about the SMMU preventing MSI
writes to arbitrary portions of the host address space. The ITS is about
routing interrupts to CPUs.
Will
Marc,
What is your opinion on ITS emulation . is it should be part of KVM or VFIO.
Also this code needs to depend on ITS host driver a lot, Host ITS driver needs to have an interface for this code to use.
Thanks,
Tirumalesh
-----Original Message-----
From: Will Deacon [mailto:will.deacon at arm.com]
Sent: Friday, June 27, 2014 1:47 AM
To: Alex Williamson
Cc: Chalamarla, Tirumalesh; Joerg Roedel; kvm at vger.kernel.org; open list; stuart.yoder at freescale.com; iommu at lists.linux-foundation.org; tech at virtualopensystems.com; kvmarm at lists.cs.columbia.edu; moderated list:ARM SMMU DRIVER; marc.zyngier at arm.com
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote:
On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
quoted
Thanks for the clarification Alex, That?s exactly my point, why are
we relying on QEMU or something else to emulate the MSI space when
we can directly give access to devices using ITS (of course with a
small emulation code). This way we are also benefited from all ITS
services like VCPU migration etc.
I have no idea what ITS is.
ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller).
I agree that we will need an ITS emulation if we want to use MSIs in the guest, and I believe that Marc (CC'd) had already started thinking about that.
quoted
What about non QEMU VFIO users, for example, if I wanted to use VFIO to
assign a device to a user process I don't need to depend on QEMU. I
thought this is one of the main goals of vfio to make it independent of
hypervisors.
Where did QEMU become a requirement? Maybe I'm missing something in
the ARM part of the conversation that got chopped off, but this is
exactly why we have the VFIO/QEMU split that we do. VFIO provides
basic virtualization for config space and restricts access to other
areas that users shouldn't be allowed to change. QEMU is just one
example of a userspace VFIO driver. QEMU takes the decomposed device
exposed through the VFIO ABI and re-creates a PCI device out of it.
VFIO itself has no dependency on QEMU. Thanks,
I also don't understand the QEMU part here. The MSI emulation would be in the kernel, just like the GICv2 emulation that we already have. For userspace drivers, wouldn't you just use eventfd rather than bother with emulating MSIs?
Finally, the interrupt remapping part is about the SMMU preventing MSI writes to arbitrary portions of the host address space. The ITS is about routing interrupts to CPUs.
Will