On Mon, May 9, 2011 at 2:58 AM, Johannes Weiner [off-list ref] wrote:

On Mon, May 09, 2011 at 09:21:12AM +0900, KAMEZAWA Hiroyuki wrote:

quoted

On Fri, 6 May 2011 16:22:57 +0200
Johannes Weiner [off-list ref] wrote:

quoted

On Fri, May 06, 2011 at 02:28:34PM +0900, KAMEZAWA Hiroyuki wrote:

quoted

Hmm, so, the interface should be

  memory.watermark  --- the total usage which kernel's memory shrinker starts.

?

I'm okay with this. And I think this parameter should be fully independent from
the limit.

Memcg can work without watermark reclaim. I think my patch just adds a new
_limit_ which a user can shrink usage of memory on deamand with kernel's help.
Memory reclaim works in background but this is not a kswapd, at all.

I guess performance benefit of using watermark under a cgroup which has limit
is very small and I think this is not for a performance tuning parameter.
This is just a new limit.

Comparing 2 cases,

 cgroup A)
   - has limit of 300M, no watermaks.
 cgroup B)
   - has limit of UNLIMITED, watermarks=300M

A) has hard limit and memory reclaim cost is paid by user threads, and have
risks of OOM under memcg.
B) has no hard limit and memory reclaim cost is paid by kernel threads, and
will not have risk of OOM under memcg, but can be CPU burning.

I think this should be called as soft-limit ;) But we have another soft-limit now.
Then, I call this as watermark. This will be useful to resize usage of memory
in online because application will not hit limit and get big latency even while
an admin makes watermark smaller.

I have two thoughts to this:

1. Even though the memcg will not hit the limit and the application
will not be forced to do memcg target reclaim, the watermark reclaim
will steal pages from the memcg and the application will suffer the
page faults, so it's not an unconditional win.

Considering the whole system, I never think this watermark can be a performance
help. This consumes the same amount of cpu as a memory freeing thread uses.
In realistic situaion, in busy memcy, several threads hits limit at the same
time and a help by a thread will not be much help.

quoted

2. I understand how the feature is supposed to work, but I don't
understand or see a use case for the watermark being configurable.
Don't get me wrong, I completely agree with watermark reclaim, it's a
good latency optimization.  But I don't see why you would want to
manually push back a memcg by changing the watermark.

For keeping free memory, when the system is not busy.

quoted

Ying wrote in another email that she wants to do this to make room fro,
another job that is about to get launched.  My reply to that was that
you should just launch the job and let global memory pressure push
back that memcg instead.  So instead of lowering the watermark, you
could lower the soft limit and don't do any reclaim at all until real
pressure arises.  You said yourself that the new feature should be
called soft limit.  And I think it is because it is a reimplementation
of the soft limit!

Soft limit works only when the system is in memory shortage. It means the
system need to use cpu for memory reclaim when the system is very busy.
This works always an admin wants. This difference will affects page allocation
latency and execution time of application. In some customer, when he wants to
start up an application in 1 sec, it must be in 1 sec. As you know, kswapd's
memory reclaim itself is too slow against rapid big allocation or burst of
network packet allocation and direct reclaim runs always. Then, it's not
avoidable to reclaim/scan memory when the system is busy.  This feature allows
admins to schedule memory reclaim when the systen is calm. It's like control of
scheduling GC.

IIRC, there was a trial to free memory when idle() runs....but it doesn't meet
current system requirement as idle() should be idle. What I think is a feature
like a that with a help of memcg.

Thanks a lot for the explanation, this certainly makes sense.

How about this: we put in memcg watermark reclaim first, as a pure
best-effort latency optimization, without the watermark configurable
from userspace.  It's not a new concept, we have it with kswapd on a
global level.

And on top of that, as a separate changeset, userspace gets a knob to
kick off async memcg reclaim when the system is idle.  With the
justification you wrote above.  We can still discuss the exact
mechanism, but the async memcg reclaim feature has value in itself and
should not have to wait until this second step is all figured out.

Would this be acceptable?

Agree on this. Although we have users for the configurable tunable for
the watermarks, in most of the cases the default watermarks
calculated by the kernel should be enough.

--Ying

Thanks again.

       Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>