Thread (74 messages) flat view 74 messages, 4 authors, 2021-02-23
STALE2038d

Revision v1 of 4 in this series.

Revisions (4)
  1. v1 current
  2. v2 [diff vs current]
  3. v3 [diff vs current]
  4. v4 [diff vs current]

[PATCH 05/10] p5303: measure time to repack with keep

From: Taylor Blau <hidden>
Date: 2021-01-19 23:27:52
Subsystem: the rest · Maintainer: Linus Torvalds

From: Jeff King <redacted>

This is the same as the regular repack test, except that we mark the
single base pack as "kept" and use --assume-kept-packs-closed. The
theory is that this should be faster than the normal repack, because
we'll have fewer objects to traverse and process.

And indeed, it is much faster in the single-pack case (all timings
measured on the kernel):

  5303.5: repack (1)                 57.29(54.88+10.39)
  5303.6: repack with keep (1)       1.25(1.19+0.05)

and in the 50-pack case:

  5303.10: repack (50)               89.71(132.78+6.14)
  5303.11: repack with keep (50)     6.92(26.93+0.58)

but our improvements vanish as we approach 1000 packs.

  5303.15: repack (1000)             217.14(493.76+15.29)
  5303.16: repack with keep (1000)   209.46(387.83+8.42)

That's because the code paths around handling .keep files are known to
scale badly; they look in every single pack file to find each object.
Our solution to that was to notice that most repos don't have keep
files, and to make that case a fast path. But as soon as you add a
single .keep, that part of pack-objects slows down again (even if we
have fewer objects total to look at).

Signed-off-by: Jeff King <redacted>
Signed-off-by: Taylor Blau <redacted>
---
 t/perf/p5303-many-packs.sh | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/t/perf/p5303-many-packs.sh b/t/perf/p5303-many-packs.sh
index 277d22ec4b..85b077b72b 100755
--- a/t/perf/p5303-many-packs.sh
+++ b/t/perf/p5303-many-packs.sh
@@ -27,8 +27,11 @@ repack_into_n () {
 	>pushes &&
 
 	# create base packfile
-	head -n 1 pushes |
-	git pack-objects --delta-base-offset --revs staging/pack &&
+	base_pack=$(
+		head -n 1 pushes |
+		git pack-objects --delta-base-offset --revs staging/pack
+	) &&
+	test_export base_pack &&
 
 	# and then incrementals between each pair of commits
 	last= &&
@@ -87,6 +90,15 @@ do
 		  --reflog --indexed-objects --delta-base-offset \
 		  --stdout </dev/null >/dev/null
 	'
+
+	test_perf "repack with keep ($nr_packs)" '
+		git pack-objects --keep-true-parents \
+		  --honor-pack-keep --assume-kept-packs-closed \
+		  --keep-pack=pack-$base_pack.pack \
+		  --non-empty --all \
+		  --reflog --indexed-objects --delta-base-offset \
+		  --stdout </dev/null >/dev/null
+	'
 done
 
 # Measure pack loading with 10,000 packs.
-- 
2.30.0.138.g6d7191ea01
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help