Thread (12 messages) 12 messages, 4 authors, 9d ago

[PATCH v3] filter-branch: fix commit map init from state branch

flat view
COOLING9d REVIEWED: 1 (0M)

From: Grant Moyer <hidden>
Date: 2026-10-03 01:09:00
Subsystem: the rest · Maintainer: Linus Torvalds

1 review trailer.

The "--state-branch" option asks git-filter-branch(1) to write a
mapping from old to new objects into a branch to enable incremental
processing of large histories. This object mapping is stored as
a simple blob at "$state_branch:filter.map" with one object pair
per line in the format "$from_commit:$to_commit". Before processing
commits in a subsequent run, the state branch is used to populate an
object mapping directory, where each file is named "map/$from_commit"
and has contents "$to_commit".

In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16),
we refactored git-filter-branch(1) to no longer require Perl,
but accidentally started to interpret object pairs in reverse (as
"$to_commit:$from_commit") when populating the map directory from
the state branch. This can cause all kinds of bad behavior from the
state-branch being effectively ignored to previously filtered objects
accidentally being mapped back to unfiltered objects. One especially
evident case occurs when "git filter-branch --prune-empty ..." maps
some commits to nothing, then on subsequent runs outputs many errors
while trying to create files with empty names, like:
/usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
This regression went unnoticed because, since the introduction of
the only test for "--state-branch" in 709cfe848a (filter-branch: skip
commits present on --state-branch, 2018-06-26), the test accidentally
passes even if commits are not skipped. The test checks that after
populating a state branch with git-filter-branch(1), then running
it again with that state branch, the resulting filtered commits for
the first run and second run match. However since the filter used is
deterministic, the commits always match, even if the commits in the
state branch are re-filtered.

Fix the population of the object mapping dir by interpreting
object pairs as "$from_commit:$to_commit". Also fix the existing
"--state-branch" test by directly exiting with a non-zero code if
any commits from the state branch aren't skipped. Finally, add a new
"--state-branch" test which directly checks that a commit from the
state branch is used when incrementally filtering a repo.

Tested-by: Michele Locati <redacted>
Co-authored-by: Michele Locati [off-list ref]
Signed-off-by: Michele Locati <redacted>
Signed-off-by: Grant Moyer <redacted>
---
 git-filter-branch.sh     |  4 +++-
 t/t7003-filter-branch.sh | 25 ++++++++++++++++++++++++-
 2 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 24fa317aaa..9aa07be6e1 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -302,7 +302,9 @@ then
 		do
 			case "$line" in
 			*:*)
-				echo "${line%:*}" >../map/"${line#*:}";;
+				from_commit=${line%:*}
+				to_commit=${line#*:}
+				echo "$to_commit" >../map/"$from_commit";;
 			*)
 				die "Unable to load state from $state_branch:filter.map";;
 			esac
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 86011e7b1f..801cc83e5e 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -121,10 +121,33 @@ W=$(git rev-parse HEAD)
 test_expect_success 'using --state-branch to skip already rewritten commits' '
 	test_when_finished git reset --hard $V &&
 	git reset --hard $V &&
-	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
+	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
 	test_cmp_rev $W HEAD
 '
 
+test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
+	test_when_finished "rm -fr incremental" &&
+	git init incremental &&
+	(
+		cd incremental &&
+		mkdir sub &&
+		test_commit first sub/file &&
+		test_commit outside root-file &&
+		git filter-branch --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- HEAD &&
+		rewritten_first=$(git rev-parse HEAD) &&
+		git reset --hard outside &&
+		test_commit second sub/file &&
+		git filter-branch -f --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
+		test_cmp_rev $rewritten_first HEAD^ &&
+		git show refs/state:filter.map >map &&
+		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
+		grep "^$(git rev-parse second):" map >actual &&
+		test_cmp expect actual
+	)
+'
+
 git tag oldD HEAD~4
 test_expect_success 'rewrite one branch, keeping a side branch' '
 	git branch modD oldD &&
-- 
2.55.0
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help