Re: lpar issue for ZONE_DEVICE p2pmem in 4.14-rc

On Wed, Oct 18, 2017 at 7:38 AM, Stephen  Bates [off-list ref] wrote=
:

Hi All

I am hoping someone can help shed some light on an issue I am seeing with=

 my attempt to add p2pmem [1] to the ppc64el kernel. p2pmem is a (currently=
) out-of-tree patchset that allows one to add device memory with struct pag=
e backing into the Linux kernel. It leverages MEMORY_HOTPLUG and ZONE_DEVIC=
E which were added to ppc64 in 4.14 so I thought it would be fun to try it =
out.

We constructed a patchset based off 4.14-rc1 [1] and build a kernel based=

 off the pseries defconfig and ran this on upstream qemu-system-ppc64. The =
exact command to run QEMU was:

qemu-system-ppc64 \
    -machine pseries \
    -cpu power8 \
    -smp 1 -m 2048 \
    -kernel ~/kernel/linux-ppc64el/vmlinux \
    -append "nvme.use_cmb=3D24 console=3Dhvc root=3D/dev/sda rootwait rw"=

 \

    -serial mon:stdio -drive file=3Dimage-ppc64el.img,if=3Dscsi,format=3D=

raw,index=3D0 \

    -nographic \
    -drive file=3D../images/nvme.qcow2,if=3Dnone,id=3Dnvme1,snapshot=3Don=

 \

    -device nvme,drive=3Dnvme1,serial=3Dnvme1 \
    -drive file=3D../images/nvme2.qcow2,if=3Dnone,id=3Dnvme2,snapshot=3Do=

n \

    -device nvme,drive=3Dnvme2,serial=3Dnvme2,cmb_size_mb=3D64

This resulted in the following extract from dmesg when registering the p2=

pmem associated with one of the NVMe SSDs.

[    3.508497] nvme 0000:00:03.0: enabling device (0100 -> 0102)
[    3.510743] nvme 0000:00:03.0: Using 64-bit direct DMA at offset 80000=

0000000000

[    3.535706] p2pmem p2pmem0: registered
[    3.537780] lpar: Attempting to resize HPT to shift 21
[    3.539251] Unable to resize hash page table to target order 21: -1

I am guessing that the hotplug of ZONE_DEVICE memory was done
incorrectly as it lead to HPT resizing (the system thinking this is
normal memory). Ideally one would expect that the driver would online
ZONE_DEVICE memory and not go through the HOTPLUG path. Are you using
devm_memremap_pages() path to add these pages?

Balbir Singh.