Hello,

On (08/18/17 00:04), Laurent Dufour wrote:

This is a port on kernel 4.13 of the work done by Peter Zijlstra to
handle page fault without holding the mm semaphore [1].

The idea is to try to handle user space page faults without holding the
mmap_sem. This should allow better concurrency for massively threaded
process since the page fault handler will not wait for other threads memory
layout change to be done, assuming that this change is done in another part
of the process's memory space. This type page fault is named speculative
page fault. If the speculative page fault fails because of a concurrency is
detected or because underlying PMD or PTE tables are not yet allocating, it
is failing its processing and a classic page fault is then tried.

The speculative page fault (SPF) has to look for the VMA matching the fault
address without holding the mmap_sem, so the VMA list is now managed using
SRCU allowing lockless walking. The only impact would be the deferred file
derefencing in the case of a file mapping, since the file pointer is
released once the SRCU cleaning is done.  This patch relies on the change
done recently by Paul McKenney in SRCU which now runs a callback per CPU
instead of per SRCU structure [1].

The VMA's attributes checked during the speculative page fault processing
have to be protected against parallel changes. This is done by using a per
VMA sequence lock. This sequence lock allows the speculative page fault
handler to fast check for parallel changes in progress and to abort the
speculative page fault in that case.

Once the VMA is found, the speculative page fault handler would check for
the VMA's attributes to verify that the page fault has to be handled
correctly or not. Thus the VMA is protected through a sequence lock which
allows fast detection of concurrent VMA changes. If such a change is
detected, the speculative page fault is aborted and a *classic* page fault
is tried.  VMA sequence locks are added when VMA attributes which are
checked during the page fault are modified.

When the PTE is fetched, the VMA is checked to see if it has been changed,
so once the page table is locked, the VMA is valid, so any other changes
leading to touching this PTE will need to lock the page table, so no
parallel change is possible at this time.

[ 2311.315400] ======================================================
[ 2311.315401] WARNING: possible circular locking dependency detected
[ 2311.315403] 4.13.0-rc5-next-20170817-dbg-00039-gaf11d7500492-dirty #1743 Not tainted
[ 2311.315404] ------------------------------------------------------
[ 2311.315406] khugepaged/43 is trying to acquire lock:
[ 2311.315407]  (&mapping->i_mmap_rwsem){++++}, at: [<ffffffff8111b339>] rmap_walk_file+0x5a/0x147
[ 2311.315415] 
               but task is already holding lock:
[ 2311.315416]  (fs_reclaim){+.+.}, at: [<ffffffff810ebd80>] fs_reclaim_acquire+0x12/0x35
[ 2311.315420] 
               which lock already depends on the new lock.

[ 2311.315422] 
               the existing dependency chain (in reverse order) is:
[ 2311.315423] 
               -> #3 (fs_reclaim){+.+.}:
[ 2311.315427]        fs_reclaim_acquire+0x32/0x35
[ 2311.315429]        __alloc_pages_nodemask+0x8d/0x217
[ 2311.315432]        pte_alloc_one+0x13/0x5e
[ 2311.315434]        __pte_alloc+0x1f/0x83
[ 2311.315436]        move_page_tables+0x2c9/0x5ac
[ 2311.315438]        move_vma.isra.25+0xff/0x2a2
[ 2311.315439]        SyS_mremap+0x41b/0x49e
[ 2311.315442]        entry_SYSCALL_64_fastpath+0x18/0xad
[ 2311.315443] 
               -> #2 (&vma->vm_sequence/1){+.+.}:
[ 2311.315449]        write_seqcount_begin_nested+0x1b/0x1d
[ 2311.315451]        __vma_adjust+0x1b7/0x5d6
[ 2311.315453]        __split_vma+0x142/0x1a3
[ 2311.315454]        do_munmap+0x128/0x2af
[ 2311.315455]        vm_munmap+0x5a/0x73
[ 2311.315458]        elf_map+0xb1/0xce
[ 2311.315459]        load_elf_binary+0x8e0/0x1348
[ 2311.315462]        search_binary_handler+0x70/0x1f3
[ 2311.315464]        load_script+0x1a6/0x1b5
[ 2311.315466]        search_binary_handler+0x70/0x1f3
[ 2311.315468]        do_execveat_common+0x461/0x691
[ 2311.315471]        kernel_init+0x5a/0xf0
[ 2311.315472]        ret_from_fork+0x27/0x40
[ 2311.315473] 
               -> #1 (&vma->vm_sequence){+.+.}:
[ 2311.315478]        write_seqcount_begin_nested+0x1b/0x1d
[ 2311.315480]        __vma_adjust+0x19c/0x5d6
[ 2311.315481]        __split_vma+0x142/0x1a3
[ 2311.315482]        do_munmap+0x128/0x2af
[ 2311.315484]        vm_munmap+0x5a/0x73
[ 2311.315485]        elf_map+0xb1/0xce
[ 2311.315487]        load_elf_binary+0x8e0/0x1348
[ 2311.315489]        search_binary_handler+0x70/0x1f3
[ 2311.315490]        load_script+0x1a6/0x1b5
[ 2311.315492]        search_binary_handler+0x70/0x1f3
[ 2311.315494]        do_execveat_common+0x461/0x691
[ 2311.315496]        kernel_init+0x5a/0xf0
[ 2311.315497]        ret_from_fork+0x27/0x40
[ 2311.315498] 
               -> #0 (&mapping->i_mmap_rwsem){++++}:
[ 2311.315503]        lock_acquire+0x176/0x19e
[ 2311.315505]        down_read+0x3b/0x55
[ 2311.315507]        rmap_walk_file+0x5a/0x147
[ 2311.315508]        page_referenced+0x11c/0x134
[ 2311.315511]        shrink_page_list+0x36b/0xb80
[ 2311.315512]        shrink_inactive_list+0x1d9/0x437
[ 2311.315514]        shrink_node_memcg.constprop.71+0x3e7/0x571
[ 2311.315515]        shrink_node+0x3f/0x149
[ 2311.315517]        try_to_free_pages+0x270/0x45f
[ 2311.315518]        __alloc_pages_slowpath+0x34a/0xaa2
[ 2311.315520]        __alloc_pages_nodemask+0x111/0x217
[ 2311.315523]        khugepaged_alloc_page+0x17/0x45
[ 2311.315524]        khugepaged+0xa29/0x16b5
[ 2311.315527]        kthread+0xfb/0x103
[ 2311.315529]        ret_from_fork+0x27/0x40
[ 2311.315530] 
               other info that might help us debug this:

[ 2311.315531] Chain exists of:
                 &mapping->i_mmap_rwsem --> &vma->vm_sequence/1 --> fs_reclaim

[ 2311.315537]  Possible unsafe locking scenario:

[ 2311.315538]        CPU0                    CPU1
[ 2311.315539]        ----                    ----
[ 2311.315540]   lock(fs_reclaim);
[ 2311.315542]                                lock(&vma->vm_sequence/1);
[ 2311.315545]                                lock(fs_reclaim);
[ 2311.315547]   lock(&mapping->i_mmap_rwsem);
[ 2311.315549] 
                *** DEADLOCK ***

[ 2311.315551] 1 lock held by khugepaged/43:
[ 2311.315552]  #0:  (fs_reclaim){+.+.}, at: [<ffffffff810ebd80>] fs_reclaim_acquire+0x12/0x35
[ 2311.315556] 
               stack backtrace:
[ 2311.315559] CPU: 0 PID: 43 Comm: khugepaged Not tainted 4.13.0-rc5-next-20170817-dbg-00039-gaf11d7500492-dirty #1743
[ 2311.315560] Call Trace:
[ 2311.315564]  dump_stack+0x67/0x8e
[ 2311.315568]  print_circular_bug.isra.39+0x1c7/0x1d4
[ 2311.315570]  __lock_acquire+0xb1a/0xe06
[ 2311.315572]  ? graph_unlock+0x69/0x69
[ 2311.315575]  lock_acquire+0x176/0x19e
[ 2311.315577]  ? rmap_walk_file+0x5a/0x147
[ 2311.315579]  down_read+0x3b/0x55
[ 2311.315581]  ? rmap_walk_file+0x5a/0x147
[ 2311.315583]  rmap_walk_file+0x5a/0x147
[ 2311.315585]  page_referenced+0x11c/0x134
[ 2311.315587]  ? page_vma_mapped_walk_done.isra.15+0xb/0xb
[ 2311.315589]  ? page_get_anon_vma+0x6d/0x6d
[ 2311.315591]  shrink_page_list+0x36b/0xb80
[ 2311.315593]  ? _raw_spin_unlock_irq+0x29/0x46
[ 2311.315595]  shrink_inactive_list+0x1d9/0x437
[ 2311.315597]  shrink_node_memcg.constprop.71+0x3e7/0x571
[ 2311.315600]  shrink_node+0x3f/0x149
[ 2311.315602]  try_to_free_pages+0x270/0x45f
[ 2311.315604]  __alloc_pages_slowpath+0x34a/0xaa2
[ 2311.315608]  ? ___might_sleep+0xd5/0x234
[ 2311.315609]  __alloc_pages_nodemask+0x111/0x217
[ 2311.315612]  khugepaged_alloc_page+0x17/0x45
[ 2311.315613]  khugepaged+0xa29/0x16b5
[ 2311.315616]  ? remove_wait_queue+0x47/0x47
[ 2311.315618]  ? collapse_shmem.isra.43+0x882/0x882
[ 2311.315620]  kthread+0xfb/0x103
[ 2311.315622]  ? __list_del_entry+0x1d/0x1d
[ 2311.315624]  ret_from_fork+0x27/0x40

	-ss