On Thu, May 5, 2011 at 10:28 PM, KAMEZAWA Hiroyuki
[off-list ref] wrote:

On Thu, 5 May 2011 08:59:01 +0200
Michal Hocko [off-list ref] wrote:

quoted

On Wed 04-05-11 10:16:39, Ying Han wrote:

quoted

On Wed, May 4, 2011 at 1:58 AM, Michal Hocko [off-list ref] wrote:

quoted

On Tue 03-05-11 10:01:27, Ying Han wrote:

quoted

On Tue, May 3, 2011 at 1:25 AM, Michal Hocko [off-list ref] wrote:

quoted

On Tue 03-05-11 16:45:23, KOSAKI Motohiro wrote:

quoted

2011/5/3 Michal Hocko [off-list ref]:

quoted

On Sun 01-05-11 15:06:02, KOSAKI Motohiro wrote:

quoted

quoted

On Mon 25-04-11 18:28:49, KAMEZAWA Hiroyuki wrote:

[...]

quoted

quoted

quoted

quoted

quoted

Can you please clarify this? I feel it is not opposite semantics.

In the global reclaim low watermark represents the point when we _start_
background reclaim while high watermark is the _stopper_. Watermarks are
based on the free memory while this proposal makes it based on the used
memory.
I understand that the result is same in the end but it is really
confusing because you have to switch your mindset from free to used and
from under the limit to above the limit.

Ah, right. So, do you have an alternative idea?

Why cannot we just keep the global reclaim semantic and make it free
memory (hard_limit - usage_in_bytes) based with low limit as the trigger
for reclaiming?

[...]

quoted

The current scheme

What is the current scheme?

using the "usage_in_bytes" instead of "free"

quoted

quoted

is closer to the global bg reclaim which the low is triggering reclaim
and high is stopping reclaim. And we can only use the "usage" to keep
the same API.

Sorry for long absence.

quoted

And how is this closer to the global reclaim semantic which is based on
the available memory?

It's never be the same feature and not a similar feature, I think.

quoted

What I am trying to say here is that this new watermark concept doesn't
fit in with the global reclaim. Well, standard user might not be aware
of the zone watermarks at all because they cannot be set. But still if
you are analyzing your memory usage you still check and compare free
memory to min/low/high watermarks to find out what is the current memory
pressure.
If we had another concept with cgroups you would need to switch your
mindset to analyze things.

I am sorry, but I still do not see any reason why those cgroup watermaks
cannot be based on total-usage.

Hmm, so, the interface should be

 memory.watermark  --- the total usage which kernel's memory shrinker starts.

?

I'm okay with this. And I think this parameter should be fully independent from
the limit.

We need two watermarks like high/low where one is used to trigger the
background reclaim and the other one is for stopping it. Using the
limit to calculate the wmarks is straight-forward since doing
background reclaim reduces the latency spikes under direct reclaim.
The direct reclaim is triggered while the usage is hitting the limit.

This is different from the "soft_limit" which is based on the usage
and we don't want to reinvent the soft_limit implementation.

--Ying

Memcg can work without watermark reclaim. I think my patch just adds a new
_limit_ which a user can shrink usage of memory on deamand with kernel's help.
Memory reclaim works in background but this is not a kswapd, at all.

I guess performance benefit of using watermark under a cgroup which has limit
is very small and I think this is not for a performance tuning parameter.
This is just a new limit.

Comparing 2 cases,

 cgroup A)
  - has limit of 300M, no watermaks.
 cgroup B)
  - has limit of UNLIMITED, watermarks=300M

A) has hard limit and memory reclaim cost is paid by user threads, and have
risks of OOM under memcg.
B) has no hard limit and memory reclaim cost is paid by kernel threads, and
will not have risk of OOM under memcg, but can be CPU burning.

I think this should be called as soft-limit ;) But we have another soft-limit now.
Then, I call this as watermark. This will be useful to resize usage of memory
in online because application will not hit limit and get big latency even while
an admin makes watermark smaller.

Hmm, maybe I should allow watermark > limit setting ;).

Thanks,
-Kame






Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>