Re: [RFC PATCH 0/2] mm: Add ability to monitor task's memory changes

On Wed, 05 Dec 2012 13:53:17 +0400
Pavel Emelyanov [off-list ref] wrote:

On 12/05/2012 04:38 AM, Matt Mackall wrote:

quoted

On Tue, 2012-12-04 at 16:24 -0800, Andrew Morton wrote:

quoted

On Tue, 04 Dec 2012 18:17:08 -0600
Matt Mackall [off-list ref] wrote:

quoted

On Tue, 2012-12-04 at 15:21 -0800, Andrew Morton wrote:

quoted

On Tue, 04 Dec 2012 09:15:10 +0400
Pavel Emelyanov [off-list ref] wrote:

quoted

quoted

Two alternatives come to mind:

1)  Use /proc/pid/pagemap (Documentation/vm/pagemap.txt) in some
    fashion to determine which pages have been touched.

[momentarily coming out of kernel retirement for old man rant]

This is a popular interface anti-pattern.

You shouldn't use an interface that gives you huge amount of STATE to
detect small amounts of CHANGE via manual differentiation.

I'm not sure that's what checkpoint-restart will be doing.  If we want
to determine "which pages have been touched since the last checkpoint
ten minutes ago" then that set of touched pages *is* state.  And it's
not "small"!

Yeah, there is definitely a middle-ground here between "I want
high-frequency updates" and "I want to see the whole picture". 
The filesystem analogy is backups: we don't have any good way to say
"find me all files changed since yesterday" short of "find all files".
The closest thing is explicit snapshotting.

For what is required for checkpoint-restore is -- we want to query the kernel
for "what pages has been written to since moment X". But this "moment X" is
a little bit more tricky than just "mark all pages r/o". Consider we're doing
this periodically. So when defining the moment X for the 2nd time we should
query the "changed" state and remap the respective page r/o atomically. Full
snapshot is actually not required, since we don't need to keep the old copy
of a page that is written to. Just a sign, that this page was modified is OK.

How is all this going to work, btw?  What is the interface to query
page states and set them read-only?  How will dirty pagecache and dirty
swapcache be handled?  And anonymous memory?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>