Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
The error occurs because of the following constraint (from
include/linux/mmzone.h) being violated:
MAX_ORDER -1 + PAGESHIFT <= SECTION_SIZE_BITS.
Expanding this out, we get:
FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT,
which requires, for a 64K page, FORCE_MAX_ZONEBITS <= 9. Thus set max
value of FORCE_MAX_ZONEORDER for 64K pages to 9, and 4K pages to 13.
Also, check the minimum value:
In include/linux/huge_mm.h, we have the constraint HPAGE_PMD_ORDER <
MAX_ORDER which expands out to:
PTE_INDEX_SIZE < FORCE_MAX_ZONEORDER.
PTE_INDEX_SIZE is:
9 (4k hash or no hash 4K pgtable) or
8 (64K hash or no hash 64K pgtable).
Thus a min value of 8 for 64K pages and 9 for 4K pages is reasonable.
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
---
v2: Changed the range for 4K pages and minimum for 64K pages as suggested
by Balbir Singh.
arch/powerpc/Kconfig | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
The error occurs because of the following constraint (from
include/linux/mmzone.h) being violated:
MAX_ORDER -1 + PAGESHIFT <= SECTION_SIZE_BITS.
Expanding this out, we get:
FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT,
which requires, for a 64K page, FORCE_MAX_ZONEBITS <= 9. Thus set max
value of FORCE_MAX_ZONEORDER for 64K pages to 9, and 4K pages to 13.
Also, check the minimum value:
In include/linux/huge_mm.h, we have the constraint HPAGE_PMD_ORDER <
MAX_ORDER which expands out to:
PTE_INDEX_SIZE < FORCE_MAX_ZONEORDER.
PTE_INDEX_SIZE is:
9 (4k hash or no hash 4K pgtable) or
8 (64K hash or no hash 64K pgtable).
Thus a min value of 8 for 64K pages and 9 for 4K pages is reasonable.
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
---
v2: Changed the range for 4K pages and minimum for 64K pages as suggested
by Balbir Singh.
arch/powerpc/Kconfig | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-04-11 12:35:17
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
quoted
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
quoted
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
Did we test hugetlbfs 4k config with this patch ? Will it work if we
start marking hugepage as gigantic page ?
-aneesh
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
quoted
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
quoted
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
The build will fail for MAX_ZONEORDER beyond the specified limits.
MAX_ORDER > 12 for what page size?
My understanding is this
1. gigantic refers to the fact the regular allocators cannot allocate
this page
2. Use alloc_contig_range() with CONFIG_CMA for gigantic pages
I could be wrong
Did we test hugetlbfs 4k config with this patch ? Will it work if we
start marking hugepage as gigantic page ?
Nope.. I did not
Thanks for the review!
Balbir Singh
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
quoted
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
quoted
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
Did we test hugetlbfs 4k config with this patch ? Will it work if we
start marking hugepage as gigantic page ?
-aneesh
Hello Rashmica,
With upstream linux kernel 4.8.0-rc1-00006-gbae9cc6 compiled with linux
4k page size we are not able set hugepages, Aneesh had a look at the
problem and he mentioned this commit is causing the issue.
*Details:*
We are using pkvm ubuntu 16.04 guest with upstream kernel
[4.8.0-rc1-00006-gbae9cc6] compiled with 4k page size
o/p from guest:
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 16384 kB
Page sizes from device-tree: [dmesg]
[ 0.000000] base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000,
tlbiel=1, penc=0
[ 0.000000] base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000,
tlbiel=1, penc=56
[ 0.000000] base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001,
tlbiel=0, penc=0
while trying to configure the hugepages inside the guest it throws the
below error:
echo 100 > /proc/sys/vm/nr_hugepages
-bash: echo: write error: Invalid argument
*Note*: we do not see the problem when the linux page is 64k
Thanks,
Santhosh G
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
quoted
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
quoted
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
9 to 13 was done based on calculations you can find the commit
quoted
Did we test hugetlbfs 4k config with this patch ? Will it work if we
start marking hugepage as gigantic page ?
-aneesh
Hello Rashmica,
With upstream linux kernel 4.8.0-rc1-00006-gbae9cc6 compiled with linux 4k page size we are not able set hugepages, Aneesh had a look at the problem and he mentioned this commit is causing the issue.
*Details:*
We are using pkvm ubuntu 16.04 guest with upstream kernel [4.8.0-rc1-00006-gbae9cc6] compiled with 4k page size
o/p from guest:
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 16384 kB
Page sizes from device-tree: [dmesg]
[ 0.000000] base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0
[ 0.000000] base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56
[ 0.000000] base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0
while trying to configure the hugepages inside the guest it throws the below error:
echo 100 > /proc/sys/vm/nr_hugepages
-bash: echo: write error: Invalid argument
*Note*: we do not see the problem when the linux page is 64k
Just to reiterate you are seeing this problem using 4k page size and 16M as the hugepage size.
With FORCE_MAX_ZONEORDER set to 9 to 13 for 4k pages, you can do upto 32M if FORCE_MAX_ZONEORDER
is 13 and same for 64K with FORCE_MAX_ZONEORDER set to 9.
Basically the constraint is
FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT
What is your value of FORCE_MAX_ZONEORDER in the .config?
Balbir Singh.
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote:
quoted
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves
FORCE_MAX_ZONEORDER set to 13 - which produces a compile error.
...
quoted
So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages
and from 13-64 to 9-13 for 4K pages.
Signed-off-by: Rashmica Gupta <redacted>
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page
size. We do have the below constraint w.r.t hugetlb pages
static inline bool hstate_is_gigantic(struct hstate *h)
{
return huge_page_order(h) >= MAX_ORDER;
}
That require MAX_ORDER to be greater than 12.
9 to 13 was done based on calculations you can find the commit
quoted
quoted
Did we test hugetlbfs 4k config with this patch ? Will it work if we
start marking hugepage as gigantic page ?
-aneesh
Hello Rashmica,
With upstream linux kernel 4.8.0-rc1-00006-gbae9cc6 compiled with linux 4k page size we are not able set hugepages, Aneesh had a look at the problem and he mentioned this commit is causing the issue.
*Details:*
We are using pkvm ubuntu 16.04 guest with upstream kernel [4.8.0-rc1-00006-gbae9cc6] compiled with 4k page size
o/p from guest:
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 16384 kB
Page sizes from device-tree: [dmesg]
[ 0.000000] base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0
[ 0.000000] base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56
[ 0.000000] base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0
while trying to configure the hugepages inside the guest it throws the below error:
echo 100 > /proc/sys/vm/nr_hugepages
-bash: echo: write error: Invalid argument
*Note*: we do not see the problem when the linux page is 64k
Just to reiterate you are seeing this problem using 4k page size and 16M as the hugepage size.
With FORCE_MAX_ZONEORDER set to 9 to 13 for 4k pages, you can do upto 32M if FORCE_MAX_ZONEORDER
is 13 and same for 64K with FORCE_MAX_ZONEORDER set to 9.
Basically the constraint is
FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT
What is your value of FORCE_MAX_ZONEORDER in the .config?
The problem is the reverse of why the orginal fix was done. ie, When you
change the pae size from 64K to 4K using nconfig, we don't update the
FORCE_MAX_ZONEORDER and hence we end up a value of 9. That results in
the above error with hugetlb. So from a failed build when switching from
4k to 64K we now have a broken hugetlb when switching from 64K to 4K.
As suggested in the review of the original patch, we should make the
FORCE_MAX_ZONEORDER range such that it picks the right value that will
get 16MB hugetlb to work.
Something like the below. That range is strange, but without that it picks a value
of 11.