On Wed, Dec 07, 2016 at 11:02:38PM +0800, Michal Hocko wrote:

On Tue 06-12-16 18:03:59, Huang Shijie wrote:

quoted

On Mon, Dec 05, 2016 at 05:31:01PM +0800, Michal Hocko wrote:

quoted

On Mon 05-12-16 17:17:07, Huang Shijie wrote:
[...]

quoted

   The failure is caused by:
    1) kernel fails to allocate a gigantic page for the surplus case.
       And the gather_surplus_pages() will return NULL in the end.

    2) The condition checks for some functions are wrong:
        return_unused_surplus_pages()
        nr_overcommit_hugepages_store()
        hugetlb_overcommit_handler()

add the  > > 

quoted

quoted

OK, so how is this any different from gigantic (1G) hugetlb pages on

I think there is no different from gigantic (1G) hugetlb pages on
x86_64. Do anyone ever tested the 1G hugetlb pages in x86_64 with the "counter.sh"
before? 

I suspect nobody has because the gigantic page support is still somehow
coarse and from a quick look into the code we only support pre-allocated

Yes, the x86_64 even does not support the gigantic page.
The default x86_64_defconfig does not enable the CONFIG_CMA.

I enabled the CONFIG_CMA, and did the test for gigantic page in x86_64.
(I appended "hugepagesz=1G hugepages=4" in the kernel cmdline.)
The result is got with my 16G x86_64 desktop:

   -------------------------------------------------
	counters.sh (1024M: 32):        FAIL mmap failed: Cannot allocate memory
	counters.sh (1024M: 64):        PASS
	********** TEST SUMMARY
	*                      1024M         
	*                      32-bit 64-bit 
	*     Total testcases:     1      1   
	*             Skipped:     0      0   
	*                PASS:     0      1   
	*                FAIL:     1      0   
	*    Killed by signal:     0      0   
	*   Bad configuration:     0      0   
	*       Expected FAIL:     0      0   
	*     Unexpected PASS:     0      0   
	*    Test not present:     0      0   
	* Strange test result:     0      0   
	**********
   -------------------------------------------------

The test passes for 64bit, but fails for 32bit (but I think it's okay,
since 1G hugetlb page is too large for the 32bit).				 

giga pages. In other words surplus pages and their accounting is not
supported at all.

Yes.

I haven't yet checked your patchset but I can tell you one thing.

Could you please review the patch set when you have time? Thanks a lot.

Surplus and subpool pages code is tricky as hell. And it is not just a

Agree. 

Do we really need so many accountings? such as reserve/ovorcommit/surplus.

matter of teaching the huge page allocation code to do the right thing.
There are subtle details all over the place. E.g. we currently
do not free giga pages AFAICS. In fact I believe that the giga pages are

Please correct me if I am wrong. :)

I think the free-giga-pages can work well.
Please see the code in update_and_free_page(). 

Could you please list all the subtle details you think the code is wrong?
I can check them one by one.

kind of implanted to the existing code without any higher level
consistency. This should change long term. But I am worried it is much

What's type of the "higher level consistency" we should care about?

Thanks
Huang Shijie

more work.

Now I might be wrong because I might misremember things which might have
been changed recently but please make sure you describe the current
state and changes of giga pages when touching this area much better if
you want to pursue this route...