Re: [PATCH 2/2] core.fsyncobjectfiles: batch disk flushes

On Wed, Aug 25, 2021 at 01:51:32AM +0000, Neeraj Singh via GitGitGadget wrote:

From: Neeraj Singh <redacted>

When adding many objects to a repo with core.fsyncObjectFiles set to
true, the cost of fsync'ing each object file can become prohibitive.

One major source of the cost of fsync is the implied flush of the
hardware writeback cache within the disk drive. Fortunately, Windows,
MacOS, and Linux each offer mechanisms to write data from the filesystem
page cache without initiating a hardware flush.

This patch introduces a new 'core.fsyncObjectFiles = 2' option that
takes advantage of the bulk-checkin infrastructure to batch up hardware
flushes.

Another interesting way to flush on linux would be the syncfs call,
which syncs all files on a file system.  Once you write more than
handful or two of files that tends to win out over a batch of fsync
calls.