On Tue, Jan 05, 2021 at 11:33:20AM -0800, David Christensen wrote:

Hey Nithin,

quoted

quoted

+static int
+test_memory_vfio_dma_map(void)
+{
+	uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
+	uint64_t unmap1, unmap2;
+	uint8_t *alloc_mem;
+	uint8_t *mem;
+	int ret;
+
+	/* Allocate twice size of requirement from heap to align later */
+	alloc_mem = malloc(sz * 2);
+	if (!alloc_mem) {
+		printf("Skipping test as unable to alloc %"PRIx64"B from heap\n",
+		       sz * 2);
+		return 1;
+	}
+
+	/* Force page allocation */
+	memset(alloc_mem, 0, sz * 2);
+
+	mem = RTE_PTR_ALIGN(alloc_mem, rte_mem_page_size());
+
+	/* map the whole region */
+	ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
+					 (uintptr_t)mem, (rte_iova_t)mem, sz);

I'm not sure how to resolve this patch for POWER systems.  The patch
currently fails with the error:

EAL:   cannot map vaddr for IOMMU, error 22 (Invalid argument)

The problem is that the size argument (page size of 64KB * 2) is smaller
than the page size set when the DMA window is created (2MB or 1GB depending
on system configuration for hugepages), resulting in the EINVAL error.  When
I tried bumping the sz value up to 2 * 1GB the test also failed because the
VA address was well outside the DMA window set when scanning memseg lists.

Allocating heap memory dynamically through the EAL works since it's
allocated in hugepage size segments and the EAL attempts to keep VA memory
addresses contiguous, therefore within the defined DMA window. But the
downside is that the memory is DMA mapped behind the scenes in
vfio_mem_event_callback().

Not sure how to get around this without duplicating a lot of the heap
management code in your test.  Maybe others have a suggestion.

David, Anatoly Burakov,

Given that both malloc'ed memory and mmap'd memory is not working for POWER9
setup, I can either drop this test patch(3/4) alone or restrict it to non-POWER9
system's. Since main fix is already acked, I think it shouldn't be a problem to
drop test case which was only added to demostrate the problem.

Any thoughts ?

Dave