-----Original Message-----
From: Mike Kravetz [mailto:mike.kravetz@oracle.com]
Sent: Saturday, August 22, 2020 5:53 AM
To: Song Bao Hua (Barry Song) <redacted>; hch@lst.de;
m.szyprowski@samsung.com; robin.murphy@arm.com; will@kernel.org;
ganapatrao.kulkarni@cavium.com; catalin.marinas@arm.com;
akpm@linux-foundation.org
Cc: iommu@lists.linux-foundation.org; linux-arm-kernel@lists.infradead.org;
linux-kernel@vger.kernel.org; Zengtao (B) [off-list ref];
huangdaode [off-list ref]; Linuxarm [off-list ref]
Subject: Re: [PATCH v7 0/3] make dma_alloc_coherent NUMA-aware by
per-NUMA CMA

Hi Barry,
Sorry for jumping in so late.

On 8/21/20 4:33 AM, Barry Song wrote:

quoted

with per-numa CMA, smmu will get memory from local numa node to save

command

quoted

queues and page tables. that means dma_unmap latency will be shrunk

much.

Since per-node CMA areas for hugetlb was introduced, I have been thinking
about the limited number of CMA areas.  In most configurations, I believe
it is limited to 7.  And, IIRC it is not something that can be changed at
runtime, you need to reconfig and rebuild to increase the number.  In contrast
some configs have NODES_SHIFT set to 10.  I wasn't too worried because of
the limited hugetlb use case.  However, this series is adding another user
of per-node CMA areas.

With more users, should try to sync up number of CMA areas and number of
nodes?  Or, perhaps I am worrying about nothing?

Hi Mike,
The current limitation is 8. If the server has 4 nodes and we enable both pernuma
CMA and hugetlb, the last node will fail to get one cma area as the default
global cma area will take 1 of 8. So users need to change menuconfig.
If the server has 8 nodes, we enable one of pernuma cma and hugetlb, one node
will fail to get cma.

We may set the default number of CMA areas as 8+MAX_NODES(if hugetlb enabled) +
MAX_NODES(if pernuma cma enabled) if we don't expect users to change config, but
right now hugetlb has not an option in Kconfig to enable or disable like pernuma cma
has DMA_PERNUMA_CMA.

--
Mike Kravetz

Thanks
Barry
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel