p2000 failure due to empty reflog

p2000 fails for me and reports:

   perf 18 - git checkout -f - (full-v3):
   running:
   			(
   				cd full-v3 &&
   				echo >>f2/f4/a &&
   				git checkout -f -
   			)

   error: pathspec '-' did not match any file(s) known to git

checkout fails because the reflog is empty, so the "-" can't be
resolved.  The pathspec error message is confusing, though.

The patch below adds a reflog entry and allows the script to
succeed.

Before the "test_perf_on_all git commit -a -m A", there are two
reflog entries in each of the five clones, after it there are
none.  How is that even possible?


---
 t/perf/p2000-sparse-operations.sh | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh
index 597626276f..8529e3d295 100755
--- a/t/perf/p2000-sparse-operations.sh
+++ b/t/perf/p2000-sparse-operations.sh

@@ -25,6 +25,7 @@ test_expect_success 'setup repo and indexes' '
 	git commit -m "level 0" &&
 	BLOB=$(git rev-parse HEAD:a) &&
 	OLD_COMMIT=$(git rev-parse HEAD) &&
+	test_export OLD_COMMIT &&
 	OLD_TREE=$(git rev-parse HEAD^{tree}) &&

 	for i in $(test_seq 1 3)

@@ -109,6 +110,14 @@ test_perf_on_all git status
 test_perf_on_all git add -A
 test_perf_on_all git add .
 test_perf_on_all git commit -a -m A
+
+test_expect_success 'add reflog entry' '
+	for repo in full-v3 full-v4 sparse-v3 sparse-v4
+	do
+		git -C $repo checkout $OLD_COMMIT
+	done
+'
+
 test_perf_on_all git checkout -f -

 test_done
--

2.33.0

On 10/2/2021 1:37 PM, René Scharfe wrote:

p2000 fails for me and reports:

   perf 18 - git checkout -f - (full-v3):
   running:
   			(
   				cd full-v3 &&
   				echo >>f2/f4/a &&
   				git checkout -f -
   			)

   error: pathspec '-' did not match any file(s) known to git

checkout fails because the reflog is empty, so the "-" can't be
resolved.  The pathspec error message is confusing, though.

The patch below adds a reflog entry and allows the script to
succeed.

Before the "test_perf_on_all git commit -a -m A", there are two
reflog entries in each of the five clones, after it there are
none.  How is that even possible?

That is certainly confusing. Is there something about your global
(or local to your test repo) GC settings that cause an auto-GC to
prune the reflog aggressively?

While I believe this will fix the situation, it might only be
a band-aid on the real problem of losing the reflog during the
test.

Thanks,
-Stolee

Am 04.10.21 um 21:55 schrieb Derrick Stolee:

On 10/2/2021 1:37 PM, René Scharfe wrote:

quoted

p2000 fails for me and reports:

   perf 18 - git checkout -f - (full-v3):
   running:
   			(
   				cd full-v3 &&
   				echo >>f2/f4/a &&
   				git checkout -f -
   			)

   error: pathspec '-' did not match any file(s) known to git

checkout fails because the reflog is empty, so the "-" can't be
resolved.  The pathspec error message is confusing, though.

The patch below adds a reflog entry and allows the script to
succeed.

Before the "test_perf_on_all git commit -a -m A", there are two
reflog entries in each of the five clones, after it there are
none.  How is that even possible?

That is certainly confusing. Is there something about your global
(or local to your test repo) GC settings that cause an auto-GC to
prune the reflog aggressively?

Good point.  I have gc.auto=0 in my config, but the tests use their own,
empty config.  A trace shows that "git gc --auto --no-quiet" is started.
The following patch turns that off and allows the tests to succeed.

Not doing maintenance in parallel to a performance test is a good idea
anyway, but I still don't understand why it would empty the reflog --
that seems excessive, dangerous even.

One of the maintenance commands from the trace is "git reflog expire
--all".  If I put that in before the "checkout -" test (on top of the
patch below) then the reflog is emptied again and the test fails.

René

---
 t/perf/p2000-sparse-operations.sh | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh
index 597626276f..9a06904247 100755
--- a/t/perf/p2000-sparse-operations.sh
+++ b/t/perf/p2000-sparse-operations.sh

@@ -105,6 +105,13 @@ test_perf_on_all () {
 	done
 }

+test_expect_success 'disable housekeeping' '
+	for repo in full-v3 full-v4 sparse-v3 sparse-v4
+	do
+		git -C $repo config gc.auto 0
+	done
+'
+
 test_perf_on_all git status
 test_perf_on_all git add -A
 test_perf_on_all git add .
--

2.33.0

On Mon, Oct 04 2021, Derrick Stolee wrote:

On 10/2/2021 1:37 PM, René Scharfe wrote:

quoted

p2000 fails for me and reports:

   perf 18 - git checkout -f - (full-v3):
   running:
   			(
   				cd full-v3 &&
   				echo >>f2/f4/a &&
   				git checkout -f -
   			)

   error: pathspec '-' did not match any file(s) known to git

checkout fails because the reflog is empty, so the "-" can't be
resolved.  The pathspec error message is confusing, though.

The patch below adds a reflog entry and allows the script to
succeed.

Before the "test_perf_on_all git commit -a -m A", there are two
reflog entries in each of the five clones, after it there are
none.  How is that even possible?

That is certainly confusing. Is there something about your global
(or local to your test repo) GC settings that cause an auto-GC to
prune the reflog aggressively?

Perhaps something to do with certain parts of the code doing a
time(NULL), and other parts faking up times to be 2005-something?

I have some WIP patches that run concurrent "git gc --auto" with a
detach across the whole test suite, and IIRC I had to search/replace all
time(NULL) with my own time() that faked up (looks it up) an epoch of
1113465166 to avoid this exact issue.

If you try doing that across our tests I think you'll find that the
@{-1} syntax (reflog-powered) will start failing.

But that's from vague memory, but perhaps it'll point you in the right
direction...

Am 05.10.21 um 23:38 schrieb Ævar Arnfjörð Bjarmason:

On Mon, Oct 04 2021, Derrick Stolee wrote:

quoted

On 10/2/2021 1:37 PM, René Scharfe wrote:

quoted

p2000 fails for me and reports:

   perf 18 - git checkout -f - (full-v3):
   running:
   			(
   				cd full-v3 &&
   				echo >>f2/f4/a &&
   				git checkout -f -
   			)

   error: pathspec '-' did not match any file(s) known to git

checkout fails because the reflog is empty, so the "-" can't be
resolved.  The pathspec error message is confusing, though.

The patch below adds a reflog entry and allows the script to
succeed.

Before the "test_perf_on_all git commit -a -m A", there are two
reflog entries in each of the five clones, after it there are
none.  How is that even possible?

That is certainly confusing. Is there something about your global
(or local to your test repo) GC settings that cause an auto-GC to
prune the reflog aggressively?

Perhaps something to do with certain parts of the code doing a
time(NULL), and other parts faking up times to be 2005-something?

I have some WIP patches that run concurrent "git gc --auto" with a
detach across the whole test suite, and IIRC I had to search/replace all
time(NULL) with my own time() that faked up (looks it up) an epoch of
1113465166 to avoid this exact issue.

If you try doing that across our tests I think you'll find that the
@{-1} syntax (reflog-powered) will start failing.

But that's from vague memory, but perhaps it'll point you in the right
direction...

Yes, that helped.  The reflog entries have fake dates in 2005, and "git
reflog expire --all" called by "git gc --auto" deletes entries older
than 90 days based on the actual current time.

René