Thread (24 messages) 24 messages, 3 authors, 2016-06-15

Re: test &&-chain lint (was: [PATCH 1/5] t5312: test object deletion code paths in a corrupted repository)

From: Jeff King <hidden>
Date: 2016-06-15 23:04:13

[+cc Jonathan, whose patch I apparently subconsciously copied]

On Thu, Mar 19, 2015 at 10:08:51PM -0400, Jeff King wrote:
quoted hunk ↗ jump to hunk
diff --git a/t/test-lib.sh b/t/test-lib.sh
index c096778..02a03d5 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -524,6 +524,21 @@ test_eval_ () {
 test_run_ () {
 	test_cleanup=:
 	expecting_failure=$2
+
+	if test -n "$GIT_TEST_CHAIN_LINT"; then
+		# 117 is unlikely to match the exit code of
+		# another part of the chain
+		test_eval_ "(exit 117) && $1"
+		if test "$?" != 117; then
+			# all bets are off for continuing with other tests;
+			# we expected none of the rest of the test commands to
+			# run, but at least some did. Who knows what weird
+			# state we're in? Just bail, and the user can diagnose
+			# by running in --verbose mode
+			error "bug in the test script: broken &&-chain"
+		fi
+	fi
+
 	setup_malloc_check
 	test_eval_ "$1"
 	eval_ret=$?
This turns up an appalling number of failures, but AFAICT they are all
"real" in the sense that the &&-chains are broken. In some cases these
are real, but in others the tests are of an older style where they did
not expect some early commands to fail (and we would catch their bogus
output if they did). E.g., in the patch below, I think the first one is
a real potential bug, and the other two are mostly noise. I do not mind
setting a rule and fixing all of them, though.

I seem to recall people looked at doing this sort of lint a while ago,
but we never ended up committing anything. I wonder if it was because of
all of these "false positives".
This turns out to be rather annoying to grep for in the list archives,
but I found at least one discussion:

  http://article.gmane.org/gmane.comp.version-control.git/235913

I don't know why we didn't follow it up then. Perhaps because the patch
there (which is rather similar to what I have above) was not
conditional, so whole chunks of the test suite needed fixing. There are
enough problems that we would probably want to do this conditionally,
fix them over time, and then finally flip the feature on by default.

-Peff
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help