On 10/19/2016 6:51 AM, Dan Williams wrote:

On Tue, Oct 18, 2016 at 2:42 PM, Stephen Bates [off-list ref] wrote:

quoted

1. Address Translation. Suggestions have been made that in certain
architectures and topologies the dma_addr_t passed to the DMA master
in a peer-2-peer transfer will not correctly route to the IO memory
intended. However in our testing to date we have not seen this to be
an issue, even in systems with IOMMUs and PCIe switches. It is our
understanding that an IOMMU only maps system memory and would not
interfere with device memory regions.

I'm not sure that's the case. I think it works because with ZONE_DEVICE,
the iommu driver will simply treat a dma_map_page call as any other PFN,
and create a mapping as it does for any memory page.

quoted

(It certainly has no opportunity
to do so if the transfer gets routed through a switch).

It can still go through the IOMMU if you enable ACS upstream forwarding.

There may still be platforms where peer-to-peer cycles are routed up
through the root bridge and then back down to target device, but we
can address that when / if it happens.

I agree.

I wonder if we could (ab)use a
software-defined 'pasid' as the requester id for a peer-to-peer
mapping that needs address translation.

Why would you need that? Isn't it enough to map the peer-to-peer
addresses correctly in the iommu driver?

Haggai