Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory

On Wed, Feb 28, 2018 at 7:56 PM, Benjamin Herrenschmidt
[off-list ref] wrote:

On Thu, 2018-03-01 at 14:54 +1100, Benjamin Herrenschmidt wrote:

quoted

On Wed, 2018-02-28 at 16:39 -0700, Logan Gunthorpe wrote:

quoted

Hi Everyone,

So Oliver (CC) was having issues getting any of that to work for us.

The problem is that acccording to him (I didn't double check the latest
patches) you effectively hotplug the PCIe memory into the system when
creating struct pages.

This cannot possibly work for us. First we cannot map PCIe memory as
cachable. (Note that doing so is a bad idea if you are behind a PLX
switch anyway since you'd ahve to manage cache coherency in SW).

Note: I think the above means it won't work behind a switch on x86
either, will it ?

The devm_memremap_pages() infrastructure allows placing the memmap in
"System-RAM" even if the hotplugged range is in PCI space. So, even if
it is an issue on some configurations, it's just a simple adjustment
to where the memmap is placed.