So the new low limit is not a rigid limit. šGlobal reclaim can reclaim
from a cgroup when its usage is below low_limit_in_bytes although such
reclaim is less aggressive than when usage is above low_limit_in_bytes.
Correct?

That's true.
But such reclaim occurs only on very small reclaiming priorities, 
so it's not a common behavior. It's mostly a protection against 
a case when all cgroups are under low limit (a results of wrong cgroups configuration).

Why doesn't memcg reclaim (i.e. !global_reclaim) also consider
low_limit_in_bytes?

For some configurations (for instance, low_limit_in_bytes == limit_in_bytes) it will work ugly.
May be it's better to introduce some restrictions on setting memcg limits, but it will be
a much more significant change from a user's point of view.

Do you have demonstration of how this improves system operation?

Assume, you have a machine with some production processes (db, web servers, etc) and a set 
of additional helper processes. You have to protect production processes from steeling theirs 
memory by other processes.
You have constant memory starvation, so kswapd works fast permanently. The production processes 
use, for instance, 80-90% of all physical memory.
Setting low limit for production cgroup to 80% of physical memory solves this problem easily and secure.

And I see no possibility to solve this task with current hard- and soft limits.
So, even if I set hard limit for all other processes to 20% of physical memory, it doesn't mean that 
production cgroup will not been scanned/reclaimed. Some magic with soft limits can help in some cases, 
but it's much more complex in configuration (see below).

Why is soft_limit insufficient?

1) If I want to grant (and protect) some amount of memory to a cgroup, i have to set soft limits for 
all other cgroups. I must consider total amount of memory, number of cgroups, theirs soft and hard limits.
Low limits provide an easier interface.
2) It works only on DEF_PRIORITY priority.
3) Also, it can be so, that my preferable cgroup is higher above it's soft limit than 
other cgroups (and it's hard to control), so it will be reclaimed more intensively than necessary.

quoted

šSigned-off-by: Roman Gushchin [off-list ref]
š---
ššinclude/linux/memcontrol.h š| ššš7 +++++
ššinclude/linux/res_counter.h | šš17 +++++++++++
šškernel/res_counter.c ššššššš| ššš2 ++
ššmm/memcontrol.c šššššššššššš| šš67 +++++++++++++++++++++++++++++++++++++++++++
ššmm/vmscan.c šššššššššššššššš| ššš5 ++++
šš5 files changed, 98 insertions(+)

Need to update Documentation/cgroups/memory.txt explaining the external
behavior of this new know and how it interacts with soft_limit_in_bytes.

Will do.

Thank you!

--
Regards,
Roman