On Tue, Jun 22, 2021 at 6:31 PM Christian König
[off-list ref] wrote:

Am 22.06.21 um 17:28 schrieb Jason Gunthorpe:

quoted

On Tue, Jun 22, 2021 at 05:24:08PM +0200, Christian König wrote:

quoted

quoted

quoted

I will take two GAUDI devices and use one as an exporter and one as an
importer. I want to see that the solution works end-to-end, with real
device DMA from importer to exporter.

I can tell you it doesn't. Stuffing physical addresses directly into
the sg list doesn't involve any of the IOMMU code so any configuration
that requires IOMMU page table setup will not work.

Sure it does. See amdgpu_vram_mgr_alloc_sgt:

         amdgpu_res_first(res, offset, length, &cursor);

          ^^^^^^^^^^

I'm not talking about the AMD driver, I'm talking about this patch.

+             bar_address = hdev->dram_pci_bar_start +
+                             (pages[cur_page] - prop->dram_base_address);
+             sg_dma_address(sg) = bar_address;

Yeah, that is indeed not working.

Oded you need to use dma_map_resource() for this.

Christian.

Yes, of course.
But will it be enough ?
Jason said that supporting IOMMU isn't nice when we don't have struct pages.
I fail to understand the connection, I need to dig into this.

Oded

quoted

Jason