Re: lpar issue for ZONE_DEVICE p2pmem in 4.14-rc

On Thu, Oct 26, 2017 at 1:34 AM, Oliver [off-list ref] wrote:

On Tue, Oct 24, 2017 at 7:17 AM, Stephen  Bates [off-list ref] wro=

te:

quoted

quoted

[    3.537780] lpar: Attempting to resize HPT to shift 21
[    3.539251] Unable to resize hash page table to target order 21: -1
[    3.541079] Unable to create mapping for hot added memory 0xc0002100=

00000000..0xc000210004000000: -2

quoted

quoted

For #1 above please check if your qemu supports H_RESIZE_HPT_* hcalls?

Balbir do you have any suggestions as to how to test for this support? N=

ote I am running this on my x86_64 host so there is no virtualization hardw=
are in my QEMU. My qemu is very recent (QEMU emulator version 2.10.50 (v2.1=
0.0-1026-gd8f932c-dirty)).

Honestly I'd just ignore the resize error. The hash table stores PTE
entries so it should be sized based on the amount of memory in the
system. If it's drastically under sized there'll be a performance hit,
but  everything should still work.

quoted

quoted

For create mapping failures, the rc is -ENOENT. Can you help debug this=

 further? We could do hcall tracing or enable debugging.

quoted

Sure I can help debug. My original email also had all you needed to recr=

eate this issue so that=E2=80=99s an option too?

I'm not too sure what's happening there. My hunch is that the
hypervisor (qemu in this case) is rejecting the attempt to map the PCI
device MMIO space as cachable memory. On bare metal systems this can
result in cache paradoxes which will kill the system so the hypervisor
has an incentive to prevent that situation.

So I had a deeper look and found the hypervisor interface spec (PAPR)
says the hypervisor should reject attempts to map memory with
inappropriate attributes for the type of memory being mapped. The
pseries model in qemu interprets this by only allowing cacheable
mappings on memory ranges that it considers as RAM. While KVM will
allow any mappings provided they have the same cachable attribute as
the hypervisor's mapping. Either way trying to use
devm_memremap_pages() like this on pseries is fundementally broken.
The alternative approach you mentioned that uses ioremap() should work
fine though.

Also, Alexy (+cc) said he was interested in trying this on some real
hardware. Is there a test suite for p2pmem floating around that he can
use?

Thanks,
Oliver