Re: [PATCH 8/8] t/README: Document the dangers of printing "ok" / "not ok"
From: Junio C Hamano <hidden>
Date: 2016-06-15 22:49:04
Ævar Arnfjörð Bjarmason [off-list ref] writes:
quoted hunk
index 42a4c25..29f795e 100644--- a/t/README +++ b/t/README@@ -50,6 +50,12 @@ prove and other harnesses come with a lot of useful options. The # Repeat until no more failures $ prove -j 15 --state=failed,save ./t[0-9]*.sh +The TAP support is completely incidental. A TAP harness is just a +program that interprets the "ok"/"not ok" (and some other strings) in +a special way. The only limitation this imposes is that you shouldn't +echo "ok" or "not ok" by yourself at the beginning of a line, that'll +confuse the TAP harness. +
I think the above is not suitable for the target audience of the section
("Running Tests"); they are not _writing_ tests.
This information (or an enhanced version of it) should come between
"Source 'test-lib.sh'" that talks about what the harness gives, and
"Skipping tests". Perhaps name that to "Test your stuff". An example...
Test your stuff
---------------
After sourcing test-lib.sh, you test the parts of the system using the
test harness. Your tests will be done in the <script> part of
test_expect_success helper function, and the exit status of the last
command in that <script> signals the success/failure of a test.
You are not supposed to "exit" from the <script> part; the harness
will catch this as a programming error of the test. Use test_done
instead, if you need to stop the tests early (see "Skipping tests"
below).
Inside <script> part, the standard output and standard error streams
are discarded, and the test harness only reports "ok" or "not ok" to
the end user running the tests. Under --verbose, they are shown to
help debugging the tests.
The output from the test suite is designed to be TAP compliant (see
http://..... to learn more about TAP), to allow programs by other
people to parse our output. This further places restriction on what
your tests can say to the standard output and standard error. The TAP
readers expect a line that begins with either "ok" and "not ok" to
signal a test passed or failed (and our harness already produces such
lines), so your script shouldn't emit such lines to their output.
I suspect the last paragraph needs a bit more work. For example, don't
you have some restriction not to say things like "1..43"???