On Wed, May 19, 2021 at 02:28:39PM +0100, Christoph Hellwig wrote:

On Mon, May 17, 2021 at 01:17:22PM -0400, Brian Foster wrote:

quoted

The iomap writeback infrastructure is currently able to construct
extremely large bio chains (tens of GBs) associated with a single
ioend. This consolidation provides no significant value as bio
chains increase beyond a reasonable minimum size. On the other hand,
this does hold significant numbers of pages in the writeback
state across an unnecessarily large number of bios because the ioend
is not processed for completion until the final bio in the chain
completes. Cap an individual ioend to a reasonable size of 4096
pages (16MB with 4k pages) to avoid this condition.

Note that once we get huge page/folio support in the page cache this
sucks as we can trivially handle much larger sizes with very little
iteration.

I wonder if both this limit and the previous one should be based on the
number of pages added instead.  And in fact maybe if we only want the
limit at add to ioend time and skip the defer to workqueue part entirely.

Both limits are already based on pages. I imagine they could change to
folios when appropriate.

The defer to workqueue part was based on your suggestion[1]. The primary
purpose of this series is to address the completion processing soft
lockup warning, so I don't have a strong preference on whether we do
that by capping ioend size, processing (and yielding) from non-atomic
context, or both.

Brian

[1] https://lore.kernel.org/linux-fsdevel/20200917080455.GY26262@infradead.org/ (local)