On Fri, Sep 17, 2021 at 9:29 AM Roman Gushchin [off-list ref] wrote:

Hi Muchun!

On Thu, Sep 16, 2021 at 09:47:35PM +0800, Muchun Song wrote:

quoted

This version is rebased over linux 5.15-rc1, because Shakeel has asked me
if I could do that. I rework some code suggested by Roman as well in this
version. I have not removed the Acked-by tags which are from Roman, because
this version is not based on the folio relevant. If Roman wants me to
do this, please let me know, thanks.

I'm fine with this, thanks for clarifying.

quoted

Since the following patchsets applied. All the kernel memory are charged
with the new APIs of obj_cgroup.

      [v17,00/19] The new cgroup slab memory controller[1]
      [v5,0/7] Use obj_cgroup APIs to charge kmem pages[2]

But user memory allocations (LRU pages) pinning memcgs for a long time -
it exists at a larger scale and is causing recurring problems in the real
world: page cache doesn't get reclaimed for a long time, or is used by the
second, third, fourth, ... instance of the same job that was restarted into
a new cgroup every time. Unreclaimable dying cgroups pile up, waste memory,
and make page reclaim very inefficient.

I've an idea: what if we use struct list_lru_memcg as an intermediate object
between an individual page and struct mem_cgroup?

It could contain a pointer to a memory cgroup structure (not even sure if a
reference is needed), and a lru page can contain a pointer to the lruvec instead
of memcg/objcg.

Hi Roman,

If I understand properly, here you mean the struct page has a pointer
to the struct lruvec not struct list_lru_memcg. What's the functionality
of the struct list_lru_memcg? Would you mind exposing more details?

This approach can probably simplify the locking scheme. But what's more
important, it can dramatically reduce the number of css_get()/put() calls.
The latter are not particularly cheap after the deletion of a cgroup:
they are atomic_dec()'s. As a result, the reclaim efficiency could be much
better. The downside: we will need to update page->lruvec_memcg pointers on
reparenting pages during the cgroup removal.

Here we need to update page->lruvec_memcg pointers one by one,
right? Because the lru lock is per lruvec, the locking scheme still need
to be as proposed by this series when the page->lruvec_memcg is
changed If I understand properly. It's likely that I don't get your point.
Looking forward to your further details.

Thanks.

This is a rough idea, maybe there are significant reasons why it's not possible
or will be way worse. But I think it's worth discussing. What do you think?

Thanks!