On Sat, Mar 04, 2023 at 09:22:20PM +0000, Adam Dinwoodie wrote:

Currently, test result files are only stored in test-results/*.counts if
$HARNESS_ACTIVE is not set.  This dates from 8ef1abe550 (test-lib: Don't
write test-results when HARNESS_ACTIVE, 2010-08-11), where the
assumption was that if someone were using a test harness like prove,
that would track results and the count files wouldn't be required.
However, as of 49da404070 (test-lib: show missing prereq summary,
2021-11-20), those files also store the list of git test prerequisites
that were missing during the test run, which isn't something that a
generic test harness like prove can provide.

To allow folk using test harnesses to access the lists of missing
prerequisites, add a --counts argument to test-lib that will keep these
counts files even if a test harness is in use.  This means that a
subsequent call of, say, `make -C t aggregate-results` will report
useful information.

Your goal seems reasonable. I have to wonder if it is even worth
requiring "--counts" here, though. Even 8ef1abe550 claims that the I/O
from writing the results files is minimal. And certainly I run under
prove with "--root=/some/ram/disk", and I haven't noticed any difference
with and without my usual "--verbose-log", which writes a lot more data
into test-results/.

So would it be worth it to just revert 8ef1abe550, and always store the
meta-files? That's one less option to support, and one less surprise
when some other feature is built around them.

Or is there some reason that we really want to have a mode where nothing
is written into t/? From reading 8ef1abe550 it sounded like this was
mostly a hygiene / optimization thing, and not some special mode we
cared about supporting.

 t/test-lib.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

The patch itself looks correct, if we want to go with a --counts option.

-Peff