On Tue 28-08-18 22:29:56, Tetsuo Handa wrote:
[...]

The OOM reaper may set MMF_OOM_SKIP without reclaiming any memory (due
to e.g. mlock()ed memory, shared memory, unable to grab mmap_sem for read).
We haven't reached to the point where the OOM reaper reclaims all memory
nor allocating threads wait some more after setting MMF_OOM_SKIP.
Therefore, this

  if (tsk_is_oom_victim(current) && !(oc->gfp_mask & __GFP_NOFAIL))
      return true;

is the simplest mitigation we can do now.

But this is adding a mess because you pretend to make a forward progress
even the OOM path didn't do anything at all and rely on another kludge
elsewhere to work. This just makes the code fragile for not strong
reason. Yes, this whole area is racy and there are rare corner cases as
you mentioned. I have already mentioned how to deal with some of them
several times. It would be so much more helpful to go after those and
address them rather than post some random hacks and build castles on a
sand.
-- 
Michal Hocko
SUSE Labs