On Wed, Sep 23, 2020 at 3:57 AM Shakeel Butt [off-list ref] wrote:

On Tue, Sep 22, 2020 at 5:37 AM Chunxin Zang [off-list ref] wrote:

quoted

On Tue, Sep 22, 2020 at 6:42 PM Chris Down [off-list ref] wrote:

quoted

Chunxin Zang writes:

quoted

On Tue, Sep 22, 2020 at 5:51 PM Chris Down [off-list ref] wrote:

quoted

Chunxin Zang writes:

quoted

My usecase is that there are two types of services in one server. They
have difference
priorities. Type_A has the highest priority, we need to ensure it's
schedule latency、I/O
latency、memory enough. Type_B has the lowest priority, we expect it
will not affect
Type_A when executed.
So Type_A could use memory without any limit. Type_B could use memory
only when the
memory is absolutely sufficient. But we cannot estimate how much
memory Type_B should
use. Because everything is dynamic. So we can't set Type_B's memory.high.

So we want to release the memory of Type_B when global memory is
insufficient in order
to ensure the quality of service of Type_A . In the past, we used the
'force_empty' interface
of cgroup v1.

This sounds like a perfect use case for memory.low on Type_A, and it's pretty
much exactly what we invented it for. What's the problem with that?

But we cannot estimate how much memory Type_A uses at least.

memory.low allows ballparking, you don't have to know exactly how much it uses.
Any amount of protection biases reclaim away from that cgroup.

quoted

For example:
total memory: 100G
At the beginning, Type_A was in an idle state, and it only used 10G of memory.
The load is very low. We want to run Type_B to avoid wasting machine resources.
When Type_B runs for a while, it used 80G of memory.
At this time Type_A is busy, it needs more memory.

Ok, so set memory.low for Type_A close to your maximum expected value.

Please forgive me for not being able to understand why setting
memory.low for Type_A can solve the problem.
In my scene, Type_A is the most important, so I will set 100G to memory.low.
But 'memory.low' only takes effect passively when the kernel is
reclaiming memory. It means that reclaim Type_B's memory only when
Type_A  in alloc memory slow path. This will affect Type_A's
performance.
We want to reclaim Type_B's memory in advance when A is expected to be busy.

How will you know when to reclaim from B? Are you polling /proc/meminfo?

Monitor global memory usage through the daemon. If the memory is used
80% or 90%, it will reclaim B's memory.

From what I understand, you want to proactively reclaim from B, so
that A does not go into global reclaim and in the worst case kill B,
right?

Yes, it is.

BTW you can use memory.high to reclaim from B by setting it lower than
memory.current of B and reset it to 'max' once the reclaim is done.
Since 'B' is not high priority (I am assuming not a latency sensitive
workload), B hitting temporary memory.high should not be an issue.
Also I am assuming you don't much care about the amount of memory to
be reclaimed from B, so I think memory.high can fulfil your use-case.
However if in future you decide to proactively reclaim from all the
jobs based on their priority i.e. more aggressive reclaim from B and a
little bit reclaim from A then memory.high is not a good interface.

Shakeel

Thanks for these suggestions, I will give it a try.

Best wishes
Chunxin