Here are the the t/README improvements I promised Junio in
[off-list ref], and
more. I took the opportunity while I was at it to document some
non-TAP things in the test suite like the prereq functions and
test_external_*.
Ævar Arnfjörð Bjarmason (8):
t/README: Tests are all +x, ./test, not sh ./test
t/README: The trash is in 't/trash directory.$name'
t/README: Typo: paralell -> parallel
t/README: Document the prereq functions, and 3-arg test_*
t/README: Document test_external*
t/README: Document test_expect_code
t/README: Add a section about skipping tests
t/README: Document the dangers of printing "ok" / "not ok"
t/README | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 92 insertions(+), 8 deletions(-)
They all run with a #!/bin/sh shebang, but it's probably better to use
the ./ form anyway.
Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---
t/README | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
@@ -52,7 +52,7 @@ prove and other harnesses come with a lot of useful options. The You can also run each test individually from command line, like this:- $ sh ./t3010-ls-files-killed-modified.sh+ $ ./t3010-ls-files-killed-modified.sh ok 1 - git update-index --add to add various paths. ok 2 - git ls-files -k to show killed files. ok 3 - validate git ls-files -k output.
There was no documentation for the test_set_prereq and
test_have_prereq functions, or the three-arg form of
test_expect_success and test_expect_failure.
Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---
t/README | 37 ++++++++++++++++++++++++++++++++++---
1 files changed, 34 insertions(+), 3 deletions(-)
@@ -246,9 +246,9 @@ Test harness library There are a handful helper functions defined in the test harness library for your script to use.- - test_expect_success <message> <script>+ - test_expect_success [<prereq>] <message> <script>- This takes two strings as parameter, and evaluates the+ Usually takes two strings as parameter, and evaluates the <script>. If it yields success, test is considered successful. <message> should state what it is testing.
@@ -257,8 +257,15 @@ library for your script to use. test_expect_success \ 'git-write-tree should be able to write an empty tree.' \ 'tree=$(git-write-tree)'++ If you supply three parameters the first will be taken to be a+ prerequisite, see the test_set_prereq and test_have_prereq+ documentation below:- - test_expect_failure <message> <script>+ test_expect_success TTY 'git --paginate rev-list uses a pager' \+ ' ... '++ - test_expect_failure [<prereq>] <message> <script> This is NOT the opposite of test_expect_success, but is used to mark a test that demonstrates a known breakage. Unlike
@@ -266,6 +273,9 @@ library for your script to use. success and "FAIL" on failure, this will say "FIXED" on success and "still broken" on failure. Failures from these tests won't cause -i (immediate) to stop.++ Like test_expect_success this function can optionally use a three+ argument invocation with a prerequisite as the first argument. - test_debug <script>
@@ -299,6 +309,27 @@ library for your script to use. Merges the given rev using the given message. Like test_commit, creates a tag and calls test_tick before committing.+ - test_set_prereq SOME_PREREQ++ Set a test prerequisite to be used later with test_have_prereq. The+ test-lib will set some prerequisites for you, e.g. PERL and PYTHON+ which are derived from ./GIT-BUILD-OPTIONS (grep test_set_prereq+ test-lib.sh for more). Others you can set yourself and use later+ with either test_have_prereq directly, or the three argument+ invocation of test_expect_success and test_expect_failure.++ - test_have_prereq SOME PREREQ++ Check if we have a prerequisite previously set with+ test_set_prereq. The most common use of this directly is to skip+ all the tests if we don't have some essential prerequisite:++ if ! test_have_prereq PERL+ then+ skip_all='skipping perl interface tests, perl not available'+ test_done+ fi+ Tips for Writing Tests ----------------------
@@ -231,6 +231,18 @@ This test harness library does the following things: consistently when command line arguments --verbose (or -v), --debug (or -d), and --immediate (or -i) is given.+Skipping tests+--------------++If you need to skip all the remaining tests you should set skip_all+and immediately call test_done. The string you give to skip_all will+be used as an explanation for why the test was skipped. for instance:++ if ! test_have_prereq PERL+ then+ skip_all='skipping perl interface tests, perl not available'+ test_done+ fi End with test_done ------------------
@@ -276,6 +276,13 @@ library for your script to use. Like test_expect_success this function can optionally use a three argument invocation with a prerequisite as the first argument.++ - test_expect_code [<prereq>] <code> <message> <script>++ Analogous to test_expect_success, but pass the test if it exits+ with a given exit <code>++ test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master' - test_debug <script>
@@ -33,7 +33,7 @@ the tests. ok 3 - plain bare Since the tests all output TAP (see http://testanything.org) they can-be run with any TAP harness. Here's an example of paralell testing+be run with any TAP harness. Here's an example of parallel testing powered by a recent version of prove(1): $ prove --timer --jobs 15 ./t[0-9]*.sh
@@ -221,9 +221,10 @@ This test harness library does the following things: - If the script is invoked with command line argument --help (or -h), it shows the test_description and exits.- - Creates an empty test directory with an empty .git/objects- database and chdir(2) into it. This directory is 't/trash directory'- if you must know, but I do not think you care.+ - Creates an empty test directory with an empty .git/objects database+ and chdir(2) into it. This directory is 't/trash+ directory.$test_name_without_dotsh' if you must know, but I do not+ think you care. - Defines standard test helper functions for your scripts to use. These functions are designed to make all scripts behave
There was do documentation for the test_external_without_stderr and
test_external functions.
Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---
t/README | 27 +++++++++++++++++++++++++++
1 files changed, 27 insertions(+), 0 deletions(-)
@@ -329,6 +329,33 @@ library for your script to use. skip_all='skipping perl interface tests, perl not available' test_done fi++ - test_external [<prereq>] <message> <external> <script>++ Execute a <script> with an <external> interpreter (like perl). This+ was added for tests like t9700-perl-git.sh which do most of their+ work in an external test script.++ test_external_without_stderr \+ 'Perl API' \+ "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl++ If the test is outputting its own TAP you should set the+ test_external_has_tap variable somewhere before calling the first+ test_external* function. See t9700-perl-git.sh for an example.++ # The external test will outputs its own plan+ test_external_has_tap=1++ - test_external_without_stderr [<prereq>] <message> <external> <script>++ Like test_external but fail if there's any output on stderr,+ instead of checking the exit code.++ test_external_without_stderr \+ 'Perl API' \+ "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl+ Tips for Writing Tests ----------------------
A TAP harness will get confused if you print "ok" or "not ok" at the
beginning of a line. This had to be fixed in
335f87871fe5aa6b3fd55b2b4e80f16fe9681483 when TAP support was
introduced.
Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---
t/README | 6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)
@@ -50,6 +50,12 @@ prove and other harnesses come with a lot of useful options. The # Repeat until no more failures $ prove -j 15 --state=failed,save ./t[0-9]*.sh+The TAP support is completely incidental. A TAP harness is just a+program that interprets the "ok"/"not ok" (and some other strings) in+a special way. The only limitation this imposes is that you shouldn't+echo "ok" or "not ok" by yourself at the beginning of a line, that'll+confuse the TAP harness.+ You can also run each test individually from command line, like this: $ ./t3010-ls-files-killed-modified.sh
From: Jeff King <hidden> Date: 2016-06-15 22:49:03
On Thu, Jul 01, 2010 at 03:09:57PM +0000, Ævar Arnfjörð Bjarmason wrote:
They all run with a #!/bin/sh shebang, but it's probably better to use
the ./ form anyway.
If we want to be nitpicky, they are actually run by "make test" with
$SHELL_PATH. We don't re-write the shebang lines in the test scripts (as
we do with the actual git-* scripts), so running "./t-whatever" won't
work if you have a broken shell (e.g., Solaris). You have to do "bash
./t-whatever".
-Peff
@@ -221,9 +221,10 @@ This test harness library does the following things: - If the script is invoked with command line argument --help (or -h), it shows the test_description and exits.- - Creates an empty test directory with an empty .git/objects- database and chdir(2) into it. This directory is 't/trash directory'- if you must know, but I do not think you care.+ - Creates an empty test directory with an empty .git/objects database+ and chdir(2) into it. This directory is 't/trash+ directory.$test_name_without_dotsh' if you must know, but I do not+ think you care.
You might want to mention the impact of "--root" here. I.e., it is
actually "$root${root:+/}trash directory.$test_name_without_dotsh", and
it is in t/ only if $root is not an absolute path.
-Peff