Re: [PATCH] t/t0000-basic.sh: Don't run a passing TODO unless TEST_PASSING_TODO=1

Heya,

On Tue, Aug 17, 2010 at 04:43, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

Change the sanity tests in t/t0000-basic.sh to not to run a passing
TODO test unless the TEST_PASSING_TODO environment variable is set.

The motivation is to have nothing out of the ordinary on a normal test
run for test smoking purposes.

If every normal test run has a passing TODO you're more likely to turn
a blind eye to it and not to investigate cases where things really are
passing unexpectedly.

Thanks, this has bothered me ever since I first ran the test suite.
While I understand that the test suite needs testing to, it should be
done in such a way that it doesn't produce false positives. That is,
my preference would be to still keep the test in the suite as-is, but
somehow get it to show up as a passing test instead of a fixed one.
E.g., something like:

test_in_new_harness 0 1 0 0 '
    test_expect_failure 'pretend we have fixed a known breakage' '
           :
     '
'

Where there '0 1 0 0' stands for '0 failures, 1 fixed, 0 broken, 0
success'. Not sure there's any other use for a test_in_new_harness
function though...

PS: To clarify, test_in_new_harness would not change any of the
counters (test_failure, test_count, test_fixed, test_broken,
test_success), but would instead compare those against the values
specified as argument to test_in_new_harness.

PSPS: I have no idea how easy it would be to implement something like this.

-- 
Cheers,

Sverre Rabbelier