Hi Gerald,
On Tue, Nov 08, 2016 at 03:08:52PM +0800, Huang Shijie wrote:

On Tue, Nov 08, 2016 at 10:19:30AM +0800, Huang Shijie wrote:

quoted

On Mon, Nov 07, 2016 at 04:25:04PM +0100, Gerald Schaefer wrote:

quoted

On Thu, 3 Nov 2016 10:51:38 +0800
Huang Shijie [off-list ref] wrote:

quoted

When testing the gigantic page whose order is too large for the buddy
allocator, the libhugetlbfs test case "counter.sh" will fail.

The failure is caused by:
 1) kernel fails to allocate a gigantic page for the surplus case.
    And the gather_surplus_pages() will return NULL in the end.

 2) The condition checks for "over-commit" is wrong.

This patch adds code to allocate the gigantic page in the
__alloc_huge_page(). After this patch, gather_surplus_pages()
can return a gigantic page for the surplus case.

This patch also changes the condition checks for:
     return_unused_surplus_pages()
     nr_overcommit_hugepages_store()

After this patch, the counter.sh can pass for the gigantic page.

Acked-by: Steve Capper <redacted>
Signed-off-by: Huang Shijie <redacted>
---
 mm/hugetlb.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 0bf4444..2b67aff 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c

@@ -1574,7 +1574,7 @@ static struct page *__alloc_huge_page(struct hstate *h,
 	struct page *page;
 	unsigned int r_nid;

-	if (hstate_is_gigantic(h))
+	if (hstate_is_gigantic(h) && !gigantic_page_supported())
 		return NULL;

Is it really possible to stumble over gigantic pages w/o having
gigantic_page_supported()?

Also, I've just tried this on s390 and counter.sh still fails after these
patches, and it should fail on all archs as long as you use the gigantic

I guess the failure you met is caused by the libhugetlbfs itself, there are
several bugs in the libhugetlbfs. I have a patch set for the
libhugetlbfs too. I will send it as soon as possible.

quoted

hugepage size as default hugepage size. This is because you only changed
nr_overcommit_hugepages_store(), which handles nr_overcommit_hugepages
in sysfs, and missed hugetlb_overcommit_handler() which handles
/proc/sys/vm/nr_overcommit_hugepages for the default sized hugepages.

This is wrong. :)

Sorry, I was wrong :). The counters test does call the /proc/sys/vm/nr_overcommit_hugepages.
But in the arm64, it does not trigger a fail for the counters test.
In an other word, I did not change the hugetlb_overcommit_handler(),
the counters.sh also can pass in arm64. 

After I add the "default_hugepagesz=32M" to the kernel cmdlin, I can
reproduce this issue. Thanks for point this.

I will look at the lockdep issue.

I tested the new patch (will be sent out later) on the arm64 platform,
and I did not meet the lockdep issue when I enabled the lockdep.
The following is my config:

	CONFIG_LOCKD=y
	CONFIG_LOCKD_V4=y
	CONFIG_LOCKUP_DETECTOR=y
        # CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
	CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
	CONFIG_DEBUG_SPINLOCK=y
	CONFIG_DEBUG_LOCK_ALLOC=y
	CONFIG_PROVE_LOCKING=y
	CONFIG_LOCKDEP=y
	CONFIG_LOCK_STAT=y
	CONFIG_DEBUG_LOCKDEP=y
	CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
	
So do I miss something? 

Thanks	
Huang Shijie