-----Original Message-----
From: David Rientjes [mailto:rientjes@google.com]
Sent: Monday, February 8, 2021 3:18 PM
To: Song Bao Hua (Barry Song) <redacted>
Cc: Matthew Wilcox <willy@infradead.org>; Wangzhou (B)
[off-list ref]; linux-kernel@vger.kernel.org;
iommu@lists.linux-foundation.org; linux-mm@kvack.org;
linux-arm-kernel@lists.infradead.org; linux-api@vger.kernel.org; Andrew
Morton [off-list ref]; Alexander Viro [off-list ref];
gregkh@linuxfoundation.org; jgg@ziepe.ca; kevin.tian@intel.com;
jean-philippe@linaro.org; eric.auger@redhat.com; Liguozhu (Kenneth)
[off-list ref]; zhangfei.gao@linaro.org; chensihang (A)
[off-list ref]
Subject: RE: [RFC PATCH v3 1/2] mempinfd: Add new syscall to provide memory
pin

On Sun, 7 Feb 2021, Song Bao Hua (Barry Song) wrote:

quoted

NUMA balancer is just one of many reasons for page migration. Even one
simple alloc_pages() can cause memory migration in just single NUMA
node or UMA system.

The other reasons for page migration include but are not limited to:
* memory move due to CMA
* memory move due to huge pages creation

Hardly we can ask users to disable the COMPACTION, CMA and Huge Page
in the whole system.

What about only for mlocked memory, i.e. disable
vm.compact_unevictable_allowed?

Adding syscalls is a big deal, we can make a reasonable inference that
we'll have to support this forever if it's merged.  I haven't seen mention
of what other unevictable memory *should* be migratable that would be
adversely affected if we disable that sysctl.  Maybe that gets you part of
the way there and there are some other deficiencies, but it seems like a
good start would be to describe how CONFIG_NUMA_BALANCING=n +
vm.compact_unevcitable_allowed + mlock() doesn't get you mostly there and
then look into what's missing.

I believe it can resolve the performance problem for the SVA
applications if we disable vm.compact_unevcitable_allowed and
NUMA_BALANCE, and use mlock().

The problem is that it is insensible to ask users to disable
unevictable_allowed or numa balancing of the whole system
only because there is one SVA application in the system.

SVA, for itself, is a mechanism to let cpu and devices share same
address space. In a typical server system, there are many processes,
the better way would be only changing the behavior of the specific
process rather than changing the whole system. It is hard to ask
users to do that only because there is a SVA monster.
Plus, this might negatively affect those applications not using SVA.

If it's a very compelling case where there simply are no alternatives, it
would make sense.  Alternative is to find a more generic way, perhaps in
combination with vm.compact_unevictable_allowed, to achieve what you're
looking to do that can be useful even beyond your originally intended use
case.

sensible. Actually pin is exactly the way to disable migration for specific
pages AKA. disabling "vm.compact_unevictable_allowed" on those pages.

It is hard to differentiate what pages should not be migrated. Only apps
know that as even SVA applications can allocate many non-IO pages which
should be able to move.

Thanks
Barry

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel