Michal Hocko said the following on 2012-1-6 18:12:

quoted

If there is something wrong, I think the bug will be in mem_cgroup_do_charge()
of mm/memcontrol.c

2210         ret = res_counter_charge(&memcg->res, csize, &fail_res);
2211 
2212         if (likely(!ret)) {

...

quoted

2221                 flags |= MEM_CGROUP_RECLAIM_NOSWAP;
2222         } else
2223                 mem_over_limit = mem_cgroup_from_res_counter(fail_res, res);

When hit memory.limit_in_bytes, res_counter_charge() will return -ENOMEM,
this will execute line 2222: } else.
But I think when hit memory.limit_in_bytes, the function should determine further
to memory.memsw.limit_in_bytes.
This think is OK?

I don't think so. We have an invariant (hard limit is "stronger" than
memsw limit) memory.limit_in_bytes <= memory.memsw.limit_in_bytes so
when we hit the hard limit we do not have to consider memsw because
resource counter:
 a) we already have to do reclaim for hard limit
 b) we check whether we might swap out later on in
 mem_cgroup_hierarchical_reclaim (root_memcg->memsw_is_minimum) so we
 will not end up swapping just to make hard limit ok and go over memsw
 limit.

Please also note that we will retry charging after reclaim if there is a
chance to meet the limit.
Makes sense?

Yeah.

But I want to test memory.memsw.failcnt is nonzero, how steps?
Thanks.

-- 
Best Regards,
Peng