RE: [RFC][PATCH v2 00/13] iommu/arm-smmu-v3: Add NVIDIA implementation

From: Jason Gunthorpe <jgg@nvidia.com>
Sent: Thursday, September 2, 2021 10:45 PM

On Wed, Sep 01, 2021 at 06:55:55AM +0000, Tian, Kevin wrote:

quoted

quoted

From: Alex Williamson
Sent: Wednesday, September 1, 2021 12:16 AM

On Mon, 30 Aug 2021 19:59:10 -0700
Nicolin Chen [off-list ref] wrote:

quoted

The SMMUv3 devices implemented in the Grace SoC support NVIDIA's

custom

quoted

CMDQ-Virtualization (CMDQV) hardware. Like the new ECMDQ feature

first

quoted

quoted

quoted

introduced in the ARM SMMUv3.3 specification, CMDQV adds multiple

VCMDQ

quoted

interfaces to supplement the single architected SMMU_CMDQ in an

effort

quoted

quoted

quoted

to reduce contention.

This series of patches add CMDQV support with its preparational

changes:

quoted

quoted

quoted

* PATCH-1 to PATCH-8 are related to shared VMID feature: they are

used

quoted

quoted

quoted

  first to improve TLB utilization, second to bind a shared VMID with a
  VCMDQ interface for hardware configuring requirement.

The vfio changes would need to be implemented in alignment with the
/dev/iommu proposals[1].  AIUI, the VMID is essentially binding
multiple containers together for TLB invalidation, which I expect in
the proposal below is largely already taken care of in that a single
iommu-fd can support multiple I/O address spaces and it's largely
expected that a hypervisor would use a single iommu-fd so this explicit
connection by userspace across containers wouldn't be necessary.

Agree. VMID is equivalent to DID (domain id) in other vendor iommus.
with /dev/iommu multiple I/O address spaces can share the same VMID
via nesting. No need of exposing VMID to userspace to build the
connection.

Indeed, this looks like a flavour of the accelerated invalidation
stuff we've talked about already.

I would see it probably exposed as some HW specific IOCTL on the iommu
fd to get access to the accelerated invalidation for IOASID's in the
FD.

Indeed, this seems like a further example of why /dev/iommu is looking
like a good idea as this RFC is very complicated to do something
fairly simple.

Where are thing on the /dev/iommu work these days?

We are actively working on the basic skeleton. Our original plan is to send
out the 1st draft before LPC, with support of vfio type1 semantics and
and pci dev only (single-device group). But later we realized that adding
multi-devices group support is also necessary even in the 1st draft to avoid
some dirty hacks and build the complete picture. This will add some time
though. If things go well, we'll still try hit the original plan. If not, it will be
soon after LPC.

Thanks
Kevin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel