Am Fr., 2. Nov. 2018 um 15:59 Uhr schrieb Vlastimil Babka [off-list ref]:

Forgot to answer this:

On 10/31/18 3:53 PM, Marinko Catovic wrote:

quoted

Well caching of any operations with find/du is not necessary imho
anyway, since walking over all these millions of files in that time
period is really not worth caching at all - if there is a way you
mentioned to limit the commands there, that would be great.
Also I want to mention that these operations were in use with 3.x
kernels as well, for years, with absolutely zero issues.

Yep, something had to change at some point. Possibly the
reclaim/compaction loop. Probably not the way dentries/inodes are being
cached though.

quoted

2 > drop_caches right after that is something I considered, I just had
some bad experience with this, since I tried it around 5:00 AM in the
first place to give it enough spare time to finish, since sync; echo 2

quoted

drop_caches can take some time, hence my question about lowering the

limits in mm/vmscan.c, void drop_slab_node(int nid)

I could do this effectively right after find/du at 07:45, just hoping
that this is finished soon enough - in one worst case it took over 2
hours (from 05:00 AM to 07:00 AM), since the host was busy during that
time with find/du, never having freed enough caches to continue, hence

Dropping caches while find/du is still running would be
counter-productive. If done after it's already finished, it shouldn't be
so disruptive.

quoted

my question to let it stop earlier with the modification of
drop_slab_node ... it was just an idea, nevermind if you believe that
it was a bad one :)

Finding a universally "correct" threshold could easily be impossible. I
guess the proper solution would be to drop the while loop and
restructure the shrinking so that it would do a single pass through all
objects.

well after a few weeks to make sure, the results seem very promising.
There were no issues any more after setting up the cgroup with the limit.

This workaround is anyway a good idea to prevent the nightly processed
from eating up all the caching/buffers which become useless anyway in
the morning, so performance got even better - although the issue is
not fixed with that workaround.
Since other people will be affected sooner or later as well imho,
hopefully you'll figure out a fix soon.

Nevertheless I also ran into a new problem there.
While writing the PID into the tasks-file (echo $$ > ../tasks) or a
direct fputs(getpid(), tasks_fp);
works very well, I also had problems with daemons that I wanted to
start (e.g. a SQL server) from within that cgroup-controlled binary.
This results in the sql server's task kill, since the memory limit is
exceeded. I would not like to set the memory.limit_in_bytes to
something that huge, such as 30G to make sure, I'd rather just use a
wrapper script to handle this, for example:
1) the cgroup-controlled instance starts the wrapper script
2) which excludes itself from the tasks-PID-list (hence the wrapper
script it is not controlled any more)
3) it starts or does whatever necessary that should continue normally
without the memory restriction

Currently I fail to manage this, since I do not know how to do step 2.
echo $PID > tasks writes into it and adds the PID, but how would one
remove the wrapper script's PID from there?
I came up with: cat /cgpath/A/tasks | sed "/$$/d" | cat >
/cgpath/A/tasks ..which results in a list without the current PID,
however, it fails to write to tasks with cat: write error: Invalid
argument, since this is not a regular file.