[PATCH 0/5] Implement SMMU passthrough using the default domain

Hi Will, Rob,

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Will Deacon
Sent: Thursday, February 02, 2017 9:41 PM
To: Sricharan <redacted>
Cc: linux-arm-kernel at lists.infradead.org; 'Rob Clark' <redacted>; 'Joerg Roedel' <joro@8bytes.org>; iommu at lists.linux-
foundation.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

On Thu, Feb 02, 2017 at 09:15:19PM +0530, Sricharan wrote:

quoted

Hi Rob,

quoted

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Rob Clark
Sent: Thursday, February 02, 2017 8:33 PM
To: Joerg Roedel <joro@8bytes.org>
Cc: Will Deacon <redacted>; iommu at lists.linux-foundation.org; Sricharan <redacted>; linux-arm-
kernel at lists.infradead.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

On Thu, Jan 26, 2017 at 12:18 PM, Joerg Roedel [off-list ref] wrote:

quoted

On Tue, Jan 24, 2017 at 08:42:23PM +0530, Sricharan wrote:

quoted

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between
process pagetables.  GPU driver was not using DMA domain and
the GPU's firmware was always expecting to run out  of contextbank
 '0' (although not correct) , which was not the case after the DMA domain
was made default  as '0' was getting allocated for DMA domain and
there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Note that iommu-drivers can request direct-mapping for any given device
on its initializtion. This is used on x86 for devices that need a 1-1
mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to
a device, which allows them to manage the dma address space on their
own.

Part of the problem is that dev->archdata.dma_ops gets wired up to
iommu_dma_ops.  Which isn't so bad on it's own, except that cache ops
are not exposed to drivers, forcing us to use dma-mapping API
(dma_map_sg, etc) for cache operations.

Possibly we should just expose cache op's to drivers bypass this abuse
of dma-mapping.

[1], with this, when the default domain in not DOMAIN_DMA, then
dev->archdata.dma_ops is not set to iommu_dma_ops , instead remains
to be swiotlb_ops. Is that not correct for gpu's unmanaged domain case ?

https://www.spinics.net/lists/arm-kernel/msg556209.html

quoted

btw, Will, we definitely want this to *not* rely on kcmdline for the
gpu with it's own private iommu case..

Ya, that changes behavior for all devices and some might want
DMA_DOMAIN and some UNMANAGED.

My patch changes the *default* domain. When would this ever be UNMANAGED?

If you're using an UNMANAGED domain, I think you need to take care of the
DMA ops yourself. There are likely missing functions to do that, but they
should be added in a separate series.

Sorry, i should have said DOMAIN_DMA or DOMAIN_IDENTITY is set as
*default* domains from your patchset. So i have not tested this series
yet. But the requirement that i thought for the gpu was
    * a context bank (specially '0') should not be reserved for DOMAIN_DMA
   * dma_ops should not be set to iommu_dma_ops for the gpu device.

I was seeing that both of that are satisfied in this series, if we choose
IDENTITY_DOMAIN as default and later gpu attaches it own
*UNMANAGED* domain.

The only thing i was thinking was, since we are changing the default domain
for all device from command line, some device which require DOMAIN_DMA
and some which require DOMAIN_IDENTITY as default was not possible.

Regards,
 Sricharan