[PATCH v2] test: fix for TEST_OUTPUT_DIRECTORY

The test_atexit unit test relies on the specific location of the
generated files.

When TEST_OUTPUT_DIRECTORY is unset, _run_sub_test_lib_test_common sets
it to pwd, which is two levels under the pwd of the parent unit test,
and the parent can find the generated files just fine.

But when TEST_OUTPUT_DIRECTORY is set, it's stored in GIT-BUILD-OPTIONS,
and even though _run_sub_test_lib_test_common correctly overrides it,
when the child script is run it sources GIT-BUILD-OPTIONS and
TEST_OUTPUT_DIRECTORY is overridden.

Effectively both the parent and child scripts output to the same
directory.

  make TEST_OUTPUT_DIRECTORY=/tmp/foobar GIT-BUILD-OPTIONS &&
  make -C t t0000-basic.sh

We could try to specify --root, as 6883047071 (t0000: set
TEST_OUTPUT_DIRECTORY for sub-tests, 2013-12-28) suggested, but then the
results of subtests would leak out because TEST_RESULTS_DIR would not
be changed from the parent.

Instead, let's revert part of 2d14e13c56 (test output: respect
$TEST_OUTPUT_DIRECTORY, 2013-04-29) by removing TEST_OUTPUT_DIRECTORY
from GIT-BUILD-OPTIONS.

It's unclear how much value t/valgrind/analyze.sh provides today, but
users of that script that use TEST_OUTPUT_DIRECTORY as well can simply
call the script with that variable in the environment.

It doesn't make much sense to break t0000-basic.sh for users of
TEST_OUTPUT_DIRECTORY, just to provide a little convenience for the
users of t/valgrind/analyze.sh.

Presumably this was broken since 900721e15c (test-lib: introduce
'test_atexit', 2019-03-13).

Cc: John Keeping <redacted>
Cc: Johannes Schindelin <redacted>
Helped-by: Jeff King [off-list ref]
Signed-off-by: Felipe Contreras <redacted>
---

Since v1 I completely changed the approach and instead of using --root
which leaks test results, I remove TEST_OUTPUT_DIRECTORY from
GIT-BUILD-OPTIONS.

Apparently only people who would care are the users of
t/valgrind/analyze.sh which now would need to specify that variable
themselves.

Marginal convenience for the users of an obscure script is not a good
reason to break t0000-basic.sh.

Range-diff against v1:
1:  04047359b9 < -:  ---------- test: fix for TEST_OUTPUT_DIRECTORY
-:  ---------- > 1:  d8430aee08 test: fix for TEST_OUTPUT_DIRECTORY

 Makefile              | 3 ---
 t/valgrind/analyze.sh | 3 ---
 2 files changed, 6 deletions(-)

diff --git a/Makefile b/Makefile
index c3565fc0f8..2e25489569 100644
--- a/Makefile
+++ b/Makefile

@@ -2790,9 +2790,6 @@ GIT-BUILD-OPTIONS: FORCE
 	@echo PAGER_ENV=\''$(subst ','\'',$(subst ','\'',$(PAGER_ENV)))'\' >>$@+
 	@echo DC_SHA1=\''$(subst ','\'',$(subst ','\'',$(DC_SHA1)))'\' >>$@+
 	@echo X=\'$(X)\' >>$@+
-ifdef TEST_OUTPUT_DIRECTORY
-	@echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+
-endif
 ifdef GIT_TEST_OPTS
 	@echo GIT_TEST_OPTS=\''$(subst ','\'',$(subst ','\'',$(GIT_TEST_OPTS)))'\' >>$@+
 endif

diff --git a/t/valgrind/analyze.sh b/t/valgrind/analyze.sh
index 2ffc80f721..378d0a8daa 100755
--- a/t/valgrind/analyze.sh
+++ b/t/valgrind/analyze.sh

@@ -1,8 +1,5 @@
 #!/bin/sh
 
-# Get TEST_OUTPUT_DIRECTORY from GIT-BUILD-OPTIONS if it's there...
-. "$(dirname "$0")/../../GIT-BUILD-OPTIONS"
-# ... otherwise set it to the default value.
 : ${TEST_OUTPUT_DIRECTORY=$(dirname "$0")/..}
 
 output=

-- 
2.32.0

On Tue, Jun 15 2021, Felipe Contreras wrote:

The test_atexit unit test relies on the specific location of the
generated files.

When TEST_OUTPUT_DIRECTORY is unset, _run_sub_test_lib_test_common sets
it to pwd, which is two levels under the pwd of the parent unit test,
and the parent can find the generated files just fine.

But when TEST_OUTPUT_DIRECTORY is set, it's stored in GIT-BUILD-OPTIONS,
and even though _run_sub_test_lib_test_common correctly overrides it,
when the child script is run it sources GIT-BUILD-OPTIONS and
TEST_OUTPUT_DIRECTORY is overridden.

Effectively both the parent and child scripts output to the same
directory.

  make TEST_OUTPUT_DIRECTORY=/tmp/foobar GIT-BUILD-OPTIONS &&
  make -C t t0000-basic.sh

We could try to specify --root, as 6883047071 (t0000: set
TEST_OUTPUT_DIRECTORY for sub-tests, 2013-12-28) suggested, but then the
results of subtests would leak out because TEST_RESULTS_DIR would not
be changed from the parent.

Instead, let's revert part of 2d14e13c56 (test output: respect
$TEST_OUTPUT_DIRECTORY, 2013-04-29) by removing TEST_OUTPUT_DIRECTORY
from GIT-BUILD-OPTIONS.

It's unclear how much value t/valgrind/analyze.sh provides today, but
users of that script that use TEST_OUTPUT_DIRECTORY as well can simply
call the script with that variable in the environment.

I've only skimmed this & the patch, but this approach sounds good to me.

It doesn't make much sense to break t0000-basic.sh for users of
TEST_OUTPUT_DIRECTORY, just to provide a little convenience for the
users of t/valgrind/analyze.sh.

I would also think that if anyone really cares about this
valgrind/analyze.sh script the thing to do is to make a Makefile target
for it, similar to the aggregate target(s).

We could then make that target go through some light wrapper in
test-lib.sh so it would e.g. pick up GIT_TEST_OPTS (which can have
--root=*), and otherwise find things in the same location that
test-lib.sh put them / thinks the are in.