On Thu, 20 Aug 2020 08:44:33 -0700 Shakeel Butt [off-list ref] wrote:

On Thu, Aug 20, 2020 at 12:11 AM SeongJae Park [off-list ref] wrote:

quoted

On Wed, 19 Aug 2020 18:21:44 -0700 Shakeel Butt [off-list ref] wrote:

quoted

On Tue, Aug 18, 2020 at 12:25 AM SeongJae Park [off-list ref] wrote:

quoted

From: SeongJae Park <redacted>

Changes from Previous Version
=============================

- Use 42 as the fake target id for paddr instead of -1
- Fix a typo

Introduction
============

DAMON[1] programming interface users can extend DAMON for any address space by
configuring the address-space specific low level primitives with appropriate
ones including their own implementations.  However, because the implementation
for the virtual address space is only available now, the users should implement
their own for other address spaces.  Worse yet, the user space users who rely
on the debugfs interface and user space tool, cannot implement their own.

This patchset implements another reference implementation of the low level
primitives for the physical memory address space.  With this change, hence, the
kernel space users can monitor both the virtual and the physical address spaces
by simply changing the configuration in the runtime.  Further, this patchset
links the implementation to the debugfs interface and the user space tool for
the user space users.

Note that the implementation supports only the user memory, as same to the idle
page access tracking feature.

[1] https://lore.kernel.org/linux-mm/20200706115322.29598-1-sjpark@amazon.com/ (local)

I am still struggling to find the benefit of this feature the way it
is implemented i.e. region based physical address space monitoring.
What exactly am I supposed to do for a given hot (or cold) physical
region? In a containerized world, that region can contain pages from
any cgroup. I can not really do anything about the accesses PHY-DAMON
provides me for a region.

Technically speaking, this patchset introduces an implementation of DAMON's low
level primitives for physical address space of LRU-listed pages.  In other
words, it is not designed for cgroups case.

So, this RFC is for a system running a single workload which comprises
multiple processes. Instead of registering each process with DAMON,
just monitor the whole physical memory, right?

Right.  Also I assumed the kernel space DAMON users as the primary users of
this feature.

Though I am still not sure how the output from DAMON can be used in
this case. DAMON told me a physical region is cold, how do I find out
processes that have mapped the pages in that region to do
process_madvise(PAGEOUT) on them?

Seems you are saying about the user space DAMON use.  Indeed, it would be quite
complicated.  For example, maybe the user space could read '/proc/pid/pagemap'
of every process, but this really sounds awkward.  Thank you for pointing this
:)  Even in the virtual address case, it could be complicated.

For this reason, I am developing DAMON-based Operation Schemes[1].  It is
designed to do the monitoring and optimization of specific regions fit in
special monitoring result by DAMON itself.  Nonetheless, current version of it
supports only virtual address spaces.  A support of physical address space from
the feature will be another future work.

If the user space really need to do that by itself, we could provide the
physical address to virtual address mapping info in addition to the monitoring
result in future.

And, yes, we have so many TODO tasks that really necessary.  I will eventually
do all of those, but also concerning about my limited time.  Nevertheless, I
believe it would better to focus on the core of DAMON[2] for now and continue
further TODO tasks after it is merged in the mainline, because maintainance of
the code will be much easier and hopefully more developers could collaborate
for the many TODO tasks.

[1] https://lore.kernel.org/linux-mm/20200804142430.15384-1-sjpark@amazon.com/ (local)
[2] https://lore.kernel.org/linux-mm/20200817105137.19296-1-sjpark@amazon.com/ (local)


Thanks,
SeongJae Park