Re: [RFC PATCH 1/7] x86, mm: ZONE_DEVICE for "device memory"

On Fri, Aug 21, 2015 at 08:02:51AM -0700, Dan Williams wrote:

[ Adding David Woodhouse ]

On Sat, Aug 15, 2015 at 1:59 AM, Christoph Hellwig [off-list ref] wrote:

quoted

On Fri, Aug 14, 2015 at 02:52:15PM -0700, Dan Williams wrote:

quoted

The idea is that this memory is not meant to be available to the page
allocator and should not count as new memory capacity.  We're only
hotplugging it to get struct page coverage.

This might need a bigger audit of the max_pfn usages.  I remember
architectures using it as a decisions for using IOMMUs or similar.

We chatted about this at LPC yesterday.  The takeaway was that the
max_pfn checks that the IOMMU code does is for checking whether a
device needs an io-virtual mapping to reach addresses above its DMA
limit (if it can't do 64-bit DMA).  Given the capacities of persistent
memory it's likely that a device with this limitation already can't
address all of RAM let alone PMEM.   So it seems to me that updating
max_pfn for PMEM hotplug does not buy us anything except a few more
opportunities to confuse PMEM as typical RAM.

I think it is wrong do not update max_pfn for 3 reasons :
  - In some case your PMEM memory will end up below current max_pfn
    value so device doing DMA can confuse your PMEM for regular RAM.
  - Given the above, not updating PMEM means you are not consistant,
    ie on some computer PMEM will be DMA addressable by device and
    on other computer it would not. Because different RAM and PMEM
    configuration, different bios, ... that cause max_pfn to be below
    range where PMEM get hotpluged.
  - Last why would we want to block device to access PMEM directly ?
    Wouldn't it make sense for some device like say network to read
    PMEM directly and stream it over the network ? All this happening
    through IOMMU (i am assuming PCIE network card using IOMMU). Which
    imply having the IOMMU consider this like regular mapping (ignoring
    Will Davis recent patchset here that might affect the IOMMU max_pfn
    test).

Cheers,
Jerome

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>