[PATCH] read-cache: make the index write buffer size 128K

From: Neeraj Singh <redacted>

Writing an index 8K at a time invokes the OS filesystem and caching code
very frequently, introducing noticeable overhead while writing large
indexes. When experimenting with different write buffer sizes on Windows
writing the Windows OS repo index (260MB), most of the benefit came by
bumping the index write buffer size to 64K. I picked 128K to ensure that
we're past the knee of the curve.

With this change, the time under do_write_index for an index with 3M
files goes from ~1.02s to ~0.72s.

Signed-off-by: Neeraj Singh <redacted>
---
    read-cache: make the index write buffer size 128K
    
    Writing an index 8K at a time invokes the OS filesystem and caching code
    very frequently, introducing noticeable overhead while writing large
    indexes. When experimenting with different write buffer sizes on Windows
    writing the Windows OS repo index (260MB), most of the benefit came by
    bumping the index write buffer size to 64K. I picked 128K to ensure that
    we're past the knee of the curve.
    
    With this change, the time under do_write_index for an index with 3M
    files goes from ~1.02s to ~0.72s.
    
    Signed-off-by: Neeraj Singh neerajsi@ntdev.microsoft.com
    
    Note: This was previously discussed on the mailing list in 2016 at:
    https://lore.kernel.org/git/1458350341-12276-1-git-send-email-dturner@twopensource.com/.
    
    Since then, I believe we have a couple changes:
    
     * 'small' development platforms like raspberry pi have gotten larger
       (4GB RAM).
     * spectre and meltdown make individual system calls more expensive when
       mitigations are enabled
     * there have been many investments to make very large repos scale well
       in git, so huge repos are more common now.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-877%2Fneerajsi-msft%2Fneerajsi%2Findex-buffer-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-877/neerajsi-msft/neerajsi/index-buffer-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/877

 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/read-cache.c b/read-cache.c
index 29144cf879e7..a5b2779b9586 100644
--- a/read-cache.c
+++ b/read-cache.c

@@ -2447,7 +2447,7 @@ int repo_index_has_changes(struct repository *repo,
 	}
 }
 
-#define WRITE_BUFFER_SIZE 8192
+#define WRITE_BUFFER_SIZE (128 * 1024)
 static unsigned char write_buffer[WRITE_BUFFER_SIZE];
 static unsigned long write_buffer_len;
 

base-commit: 45526154a57d15947cad7262230d0b935cedb9d3
-- 
gitgitgadget

On 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote:

From: Neeraj Singh <redacted>

Writing an index 8K at a time invokes the OS filesystem and caching code
very frequently, introducing noticeable overhead while writing large
indexes. When experimenting with different write buffer sizes on Windows
writing the Windows OS repo index (260MB), most of the benefit came by
bumping the index write buffer size to 64K. I picked 128K to ensure that
we're past the knee of the curve.

With this change, the time under do_write_index for an index with 3M
files goes from ~1.02s to ~0.72s.

[...]

  
-#define WRITE_BUFFER_SIZE 8192
+#define WRITE_BUFFER_SIZE (128 * 1024)
  static unsigned char write_buffer[WRITE_BUFFER_SIZE];
  static unsigned long write_buffer_len;

[...]

Very nice.

I can confirm that this gives nice gains on Windows.  (I'm using
the Office repo which has a 188MB index file (2.1M files at HEAD).
Running "git status" shows a gain of about 200ms.

We get a smaller gain on Mac of about 50ms (again, using the Office
repo).

So, you may add my sign-off or ACK to this.
     Signed-off-by: Jeff Hostetler [off-list ref]



FWIW, You might take a look at `t/perf/p0007-write-cache.sh`
Update it as follows:

diff --git a/t/perf/p0007-write-cache.sh b/t/perf/p0007-write-cache.sh
index 09595264f0..337280ff1c 100755
--- a/t/perf/p0007-write-cache.sh
+++ b/t/perf/p0007-write-cache.sh
@@ -4,7 +4,8 @@ test_description="Tests performance of writing the index"

  . ./perf-lib.sh

-test_perf_default_repo
+test_perf_large_repo

  test_expect_success "setup repo" '
         if git rev-parse --verify refs/heads/p0006-ballast^{commit}

Then you can run it like this:

     $ cd t/perf
     $ GIT_PERF_LARGE_REPO=/path/to/your/enlistment ./p0007-write-cache

Then you can run it with the small and then with the large buffer and
get times for essentially just the index write in isolation.

Hope this helps,
Jeff