Hi Robin,


On Wed, Mar 24, 2021, at 16:29, Robin Murphy wrote:

On 2021-03-20 15:19, Sven Peter wrote:

quoted

I have just noticed today though that at least the USB DWC3 controller in host
mode uses *two* darts at the same time. I'm not sure yet which parts seem to
require which DART instance.

This means that we might need to support devices attached to two iommus
simultaneously and just create the same iova mappings. Currently this only
seems to be required for USB according to Apple's Device Tree.

I see two options for this and would like to get feedback before
I implement either one:

     1) Change #iommu-cells = <1>; to #iommu-cells = <2>; and use the first cell
        to identify the DART and the second one to identify the master.
        The DART DT node would then also take two register ranges that would
        correspond to the two DARTs. Both instances use the same IRQ and the
        same clocks according to Apple's device tree and my experiments.
        This would keep a single device node and the DART driver would then
        simply map iovas in both DARTs if required.

This is broadly similar to the approach used by rockchip-iommu and the 
special arm-smmu-nvidia implementation, where there are multiple 
instances which require programming identically, that are abstracted 
behind a single "device". Your case is a little different since you're 
not programming both *entirely* identically, although maybe that's a 
possibility if each respective ID isn't used by anything else on the 
"other" DART?

That would be possible. The only difference is that I need to
program ID 0 of the first DART and ID 1 of the second one. Both
of these IDs are only connected to the same USB controller.

Overall I tend to view this approach as a bit of a hack because it's not 
really describing the hardware truthfully - just because two distinct 
functional blocks have their IRQ lines wired together doesn't suddenly 
make them a single monolithic block with multiple interfaces - and tends 
to be done for the sake of making the driver implementation easier in 
terms of the Linux IOMMU API (which, note, hasn't evolved all that far 
from its PCI-centric origins and isn't exactly great for arbitrary SoC 
topologies).

Yes, the easier driver implementation was my reason to favour this option.

quoted

     2) Keep #iommu-cells as-is but support
             iommus = <&usb_dart1a 1>, <&usb_dart1b 0>;
        instead.
        This would then require two devices nodes for the two DART instances and
        some housekeeping in the DART driver to support mapping iovas in both
        DARTs.
        I believe omap-iommu.c supports this setup but I will have to read
        more code to understand the details there and figure out how to implement
        this in a sane way.

This approach is arguably the most honest, and more robust in terms of 
making fewer assumptions, and is used by at least exynos-iommu and 
omap-iommu. In Linux it currently takes a little bit more housekeeping 
to keep track of linked instances within the driver since the IOMMU API 
holds the notion that any given client device is associated with "an 
IOMMU", but that's always free to change at any time, unlike the design 
of a DT binding.

Sounds good. I'll read those drivers and give it a try for v2.


Thanks,


Sven

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel