On Sat, Sep 26, 2020 at 4:14 PM Christoph Hellwig [off-list ref] wrote:

On Fri, Sep 25, 2020 at 06:46:22PM +0000, Tomasz Figa wrote:

quoted

quoted

+static void *iommu_dma_alloc_noncoherent(struct device *dev, size_t size,
+           dma_addr_t *handle, enum dma_data_direction dir, gfp_t gfp)
+{
+   if (!gfpflags_allow_blocking(gfp)) {
+           struct page *page;
+
+           page = dma_common_alloc_pages(dev, size, handle, dir, gfp);
+           if (!page)
+                   return NULL;
+           return page_address(page);
+   }
+
+   return iommu_dma_alloc_remap(dev, size, handle, gfp | __GFP_ZERO,
+                                PAGE_KERNEL, 0);

iommu_dma_alloc_remap() makes use of the DMA_ATTR_ALLOC_SINGLE_PAGES attribute
to optimize the allocations for devices which don't care about how contiguous
the backing memory is. Do you think we could add an attrs argument to this
function and pass it there?

As ARM is being moved to the common iommu-dma layer as well, we'll probably
make use of the argument to support the DMA_ATTR_NO_KERNEL_MAPPING attribute to
conserve the vmalloc area.

We could probably at it.  However I wonder why this is something the
drivers should care about.  Isn't this really something that should
be a kernel-wide policy for a given system?

There are IOMMUs out there which support huge pages and those can
benefit *some* hardware depending on what kind of accesses they
perform, possibly on a per-buffer basis. At the same time, order > 0
allocations can be expensive, significantly affecting allocation
latency, so for devices which don't care about huge pages anyone would
prefer simple single-page allocations. Currently the drivers know the
best on whether the hardware they drive would care. There are some
decision factors listed in the documentation [1].

I can imagine cases where drivers could not be the best to decide
about this - for example, the workload could vary depending on the
userspace or a product decision regarding the performance vs
allocation latency, but we haven't seen such cases in practice yet.

[1] https://www.kernel.org/doc/html/latest/core-api/dma-attributes.html?highlight=dma_attr_alloc_single_pages#dma-attr-alloc-single-pages

Best regards,
Tomasz