Re: [PATCH v2 2/2] virtio_balloon: replace oom notifier with shrinker

On Wed 01-08-18 19:12:25, Wei Wang wrote:

On 07/30/2018 05:00 PM, Michal Hocko wrote:

quoted

On Fri 27-07-18 17:24:55, Wei Wang wrote:

quoted

The OOM notifier is getting deprecated to use for the reasons mentioned
here by Michal Hocko: https://lkml.org/lkml/2018/7/12/314

This patch replaces the virtio-balloon oom notifier with a shrinker
to release balloon pages on memory pressure.

It would be great to document the replacement. This is not a small
change...

OK. I plan to document the following to the commit log:

  The OOM notifier is getting deprecated to use for the reasons:
    - As a callout from the oom context, it is too subtle and easy to
      generate bugs and corner cases which are hard to track;
    - It is called too late (after the reclaiming has been performed).
      Drivers with large amuont of reclaimable memory is expected to be
      released them at an early age of memory pressure;
    - The notifier callback isn't aware of the oom contrains;
    Link: https://lkml.org/lkml/2018/7/12/314

    This patch replaces the virtio-balloon oom notifier with a shrinker
    to release balloon pages on memory pressure. Users can set the amount of
    memory pages to release each time a shrinker_scan is called via the
    module parameter balloon_pages_to_shrink, and the default amount is 256
    pages. Historically, the feature VIRTIO_BALLOON_F_DEFLATE_ON_OOM has
    been used to release balloon pages on OOM. We continue to use this
    feature bit for the shrinker, so the shrinker is only registered when
    this feature bit has been negotiated with host.

Do you have any numbers for how does this work in practice? Let's say
you have a medium page cache workload which triggers kswapd to do a
light reclaim? Hardcoded shrinking sounds quite dubious to me but I have
no idea how people expect this to work. Shouldn't this be more
adaptive? How precious are those pages anyway?
-- 
Michal Hocko
SUSE Labs