On Tue, Feb 14, 2012 at 01:33:37PM -0800, Andrew Morton wrote:

On Sun, 12 Feb 2012 01:21:35 +0100
Andrea Righi [off-list ref] wrote:

quoted

The new proposal is to implement POSIX_FADV_NOREUSE as a way to perform a real
drop-behind policy where applications can mark certain intervals of a file as
FADV_NOREUSE before accessing the data.

I think you and John need to talk to each other, please.  The amount of
duplication here is extraordinary.

Yes, definitely. I'm currently reviewing and testing the John's patch
set. I was even considering to apply my patch set on top of the John's
patch, or at least propose my tree-based approach to manage the list of
the POSIX_FADV_VOLATILE ranges.

Both patchsets add fields to the address_space (and hence inode), which
is significant - we should convince ourselves that we're getting really
good returns from a feature which does this.



Regarding the use of fadvise(): I suppose it's a reasonable thing to do
in the long term - if the feature works well, popular data streaming
applications will eventually switch over.  But I do think we should
explore interfaces which don't require modification of userspace source
code.  Because there will always be unconverted applications, and the
feature becomes available immediately.

One such interface would be to toss the offending application into a
container which has a modified drop-behind policy.  And here we need to
drag out the crystal ball: what *is* the best way of tuning application
pagecache behaviour?  Will we gravitate towards containerization, or
will we gravitate towards finer-tuned fadvise/sync_page_range/etc
behaviour?  Thus far it has been the latter, and I don't think that has
been a great success.

Finally, are the problems which prompted these patchsets already
solved?  What happens if you take the offending streaming application
and toss it into a 16MB memcg?  That *should* avoid perturbing other
things running on that machine.

Moving the streaming application into a 16MB memcg can be dangerous in
some cases... the application might start to do "bad" things, like
swapping (if the memcg can swap) or just fail due to OOMs.

And yes, a container-based approach is pretty crude, and one can
envision applications which only want modified reclaim policy for one
particualr file.  But I suspect an application-wide reclaim policy
solves 90% of the problems.

I really like the container-based approach. But for this we need a
better file cache control in the memory cgroup; now we have the
accounting of file pages, but there's no way to limit them.

Thanks for your comments, Andrew.

-Andrea

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>