Thread (4 messages) 4 messages, 2 authors, 20d ago

Re: [PATCH] midx-write: skip empty incremental layers

From: Taylor Blau <hidden>
Date: 2026-09-08 04:09:17
Subsystem: the rest · Maintainer: Linus Torvalds

On Mon, Sep 07, 2026 at 06:46:40PM -0700, Pia Park wrote:
An incremental MIDX write can find new packs without finding any new
objects, either because the packs are empty or because their objects
are already indexed by an earlier layer.

The existing early exit checks the number of packs, so these cases
First, thanks for working on this :-).

Second, what you wrote makes sense. It may be worth saying "[...] checks
*only* the number of packs", or "[...] but not the number of objects".
publish a zero-object layer without a reverse index. A subsequent
incremental write with --bitmap fails when loading that reverse index.

Reproducible on master as of
b8242b093d9e941a34460d715e3ce616a34ac3fe (2026-09-07), using:
Nit: we typically would abbreviate this using the "reference" pretty
formatter, as in:

    b8242b093d (The 23rd batch, 2026-09-07)

, but I think that it would be more interesting to include the commit
that introduced this breakage, which I would guess (though haven't
bisected) that we've had this bug at least as long as we've been able to
write incremental MIDXs. Though see below for perhaps an earlier origin.
    git init --bare --object-format=sha1 empty.git &&
    (
        cd empty.git &&
        git config midx.version 2 &&
        git pack-objects objects/pack/pack </dev/null &&
        git multi-pack-index write --incremental --bitmap &&
        git multi-pack-index write --incremental --bitmap
    )

The first write succeeds but publishes the empty layer
3c8853aad425100c5ee2ee22209bb0bb3df9ca37. The second exits with status
255, reporting "could not load reverse index for MIDX".
Right. This patch message suggests (and I agree with) the fact that the
first layer wrote anything at all is a bug.

It only happened to work because the first invocation did not require
loading the empty reverse index, and so did not read the corruption that
it just wrote. The second invocation notices the bug because we eagerly
read reverse indexes for pack(s) in previous layer(s) when writing
reachability bitmaps.

That makes me wonder whether this bug is unique to incremental MIDXs at
all. I tried testing this out locally with:

    git.compile init --bare empty.git &&
    (
      cd empty.git &&

      git.compile pack-objects objects/pack/pack </dev/null &&
      git.compile multi-pack-index write
    )

, and it happily wrote a MIDX.
Return success through the existing cleanup path when
compute_sorted_entries() finds no entries for a non-compacting
incremental write. This prevents publishing an empty layer that causes
subsequent incremental writes with --bitmap to fail with exit status
255. Exit before acquiring a lock or creating a temporary MIDX file,
leaving the existing chain untouched.
So I wonder if we should apply the fix even earlier in
write_midx_internal(), perhaps like:
--- 8< ---
git rev-parse 2>/dev/null || cd ~/src/git; git: line 0: cd: /Users/ttaylorr/src/git: No such file or directory
diff --git a/midx-write.c b/midx-write.c
index 580724d21a..5b2aa9acc8 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -1617,9 +1617,8 @@ static int write_midx_internal(struct write_midx_opts *opts)
 	}

 	if (!ctx.entries_nr) {
-		if (opts->flags & MIDX_WRITE_BITMAP)
-			warning(_("refusing to write multi-pack .bitmap without any objects"));
-		opts->flags &= ~(MIDX_WRITE_REV_INDEX | MIDX_WRITE_BITMAP);
+		error(_("no objects to index."));
+		goto cleanup;
 	}

 	if (ctx.incremental) {
--- >8 ---
(as an aside, we can probably rework those error messages to be a bit
more descriptive, perhaps, "cannot create a multi-pack-index without any
packs". But that is besides the point of your patch.)
quoted hunk ↗ jump to hunk
diff --git a/midx-write.c b/midx-write.c
index 8537102254..cdb2ef0474 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -1518,6 +1518,11 @@ static int write_midx_internal(struct write_midx_opts *opts)

 	compute_sorted_entries(&ctx, start_pack);

+	if (ctx.incremental && !ctx.compact && !ctx.entries_nr) {
+		result = 0;
+		goto cleanup;
+	}
+
Hmm. So we will avoid writing an empty MIDX when we have no object
entries, but only when doing a non-compact, incremental write? I imagine
that we would want similar treatment for both incremental and
non-incremental MIDXs, regardless of whether we are compacting.
quoted hunk ↗ jump to hunk
diff --git a/t/t5334-incremental-multi-pack-index.sh b/t/t5334-incremental-multi-pack-index.sh
index f0b82b5f65..4bdfa61d38 100755
--- a/t/t5334-incremental-multi-pack-index.sh
+++ b/t/t5334-incremental-multi-pack-index.sh
I suspect that these tests will change a bit, so I'll avoid reviewing
them too carefully for the time being. I am glad, however, that you are
testing cases besides explicitly empty packs, e.g., dropping objects
which are represented in earlier layers.

Thanks,
Taylor
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help