Re: Is shmem page accounting wrong on split?

On Fri, Aug 28, 2020 at 10:08:52AM -0700, Hugh Dickins wrote:

On Fri, 28 Aug 2020, Yang Shi wrote:

quoted

On Fri, Aug 28, 2020 at 7:55 AM Matthew Wilcox [off-list ref] wrote:

quoted

On Fri, Aug 28, 2020 at 03:25:46PM +0100, Matthew Wilcox wrote:

quoted

If I understand truncate of a shmem THP correctly ...

Let's suppose the file has a single 2MB page at index 0, and is being
truncated down to 7 bytes in size.

shmem_setattr()
  i_size_write(7);
  shmem_truncate_range(7, -1);
    shmem_undo_range(7, -1)
      start = 1;
      page = &head[1];
      shmem_punch_compound();
        split_huge_page()
          end = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); # == 1
          __split_huge_page(..., 1, ...);
            __delete_from_page_cache(&head[1], ...);
      truncate_inode_page(page);
        delete_from_page_cache(page)
          __delete_from_page_cache(&head[1])

I think the solution is to call truncate_inode_page() from within
shmem_punch_compound() if we don't call split_huge_page().  I came across
this while reusing all this infrastructure for the XFS THP patchset,
so I'm not in a great position to test this patch.

It's a good observation of an oddity that I probably didn't think of,
but you haven't said which kind of shmem page accounting goes wrong here
(vm_enough_memory? df of filesystem? du of filesystem? memcg charge?
all of the above? observed in practice?), and what needs solving.

If that page has already been deleted from page cache when splitting,
truncate_inode_page() sees NULL page->mapping != mapping and returns
without doing anything.  What's the problem?

Ah!  I missed the check in truncate_inode_page().  This should be
fine then.

The problem I've observed in practice is following the same pattern in
truncate_inode_pages_range().  The call to delete_from_page_cache_batch()
trips the assertion that the page hasn't already been deleted from the
page cache.  I think the solution is obvious -- don't add the page to
locked_pvec if page->mapping is NULL.

                        if (thp_punch(page, lstart, lend))
                                pagevec_add(&locked_pvec, page);
                        else
                                unlock_page(page);
                }
                for (i = 0; i < pagevec_count(&locked_pvec); i++)
                        truncate_cleanup_page(mapping, locked_pvec.pages[i]);
                delete_from_page_cache_batch(mapping, &locked_pvec);
                for (i = 0; i < pagevec_count(&locked_pvec); i++)
                        unlock_page(locked_pvec.pages[i]);
                truncate_exceptional_pvec_entries(mapping, &pvec, indices);
                pagevec_release(&pvec);

(shmem_punch_compound() got renamed to thp_punch() and moved to truncate.c)