Re: [LSF/MM TOPIC] Memory hotplug, ZONE_DEVICE, and the future of struct page
From: Dan Williams <hidden>
Date: 2017-01-12 23:59:00
Also in:
linux-fsdevel, linux-mm, nvdimm
On Thu, Jan 12, 2017 at 3:14 PM, Jerome Glisse [off-list ref] wrote:
On Thu, Jan 12, 2017 at 02:43:03PM -0800, Dan Williams wrote:quoted
Back when we were first attempting to support DMA for DAX mappings of persistent memory the plan was to forgo 'struct page' completely and develop a pfn-to-scatterlist capability for the dma-mapping-api. That effort died in this thread: https://lkml.org/lkml/2015/8/14/3 ...where we learned that the dependencies on struct page for dma mapping are deeper than a PFN_PHYS() conversion for some architectures. That was the moment we pivoted to ZONE_DEVICE and arranged for a 'struct page' to be available for any persistent memory range that needs to be the target of DMA. ZONE_DEVICE enables any device-driver that can target "System RAM" to also be able to target persistent memory through a DAX mapping. Since that time the "page-less" DAX path has continued to mature [1] without growing new dependencies on struct page, but at the same time continuing to rely on ZONE_DEVICE to satisfy get_user_pages(). Peer-to-peer DMA appears to be evolving from a niche embedded use case to something general purpose platforms will need to comprehend. The "map_peer_resource" [2] approach looks to be headed to the same destination as the pfn-to-scatterlist effort. It's difficult to avoid 'struct page' for describing DMA operations without custom driver code. With that background, a statement and a question to discuss at LSF/MM: General purpose DMA, i.e. any DMA setup through the dma-mapping-api, requires pfn_to_page() support across the entire physical address range mapped.Note that in my case it is even worse. The pfn of the page does not correspond to anything so it need to go through a special function to find if a page can be mapped for another device and to provide a valid pfn at which the page can be access by other device.
I still haven't quite wrapped my head about how these pfn ranges are created. Would this be a use case for a new pfn_t flag? It doesn't sound like something we'd want to risk describing with raw 'unsigned long' pfns.