Hi Eric,

On Mon, 3 Sep 2018, Eric Sunshine wrote:

On Mon, Sep 3, 2018 at 5:10 PM Johannes Schindelin via GitGitGadget
[off-list ref] wrote:

quoted

This will come in handy when publishing the results of Git's test suite
during an automated VSTS CI run.

Signed-off-by: Johannes Schindelin <redacted>
---

diff --git a/t/test-lib.sh b/t/test-lib.sh

@@ -431,11 +434,24 @@ trap 'exit $?' INT
 test_failure_ () {
+       if test -n "$write_junit_xml"
+       then
+               junit_insert="<failure message=\"not ok $test_count -"
+               junit_insert="$junit_insert $(xml_attr_encode "$1")\">"
+               junit_insert="$junit_insert $(xml_attr_encode \
+                       "$(printf '%s\n' "$@" | sed 1d)")"
+               junit_insert="$junit_insert</failure>"

This is a genuine failure, so you're creating a <failure> node. Okay.

quoted

+               write_junit_xml_testcase "$1" "      $junit_insert"
+       fi

@@ -444,11 +460,19 @@ test_failure_ () {
 test_known_broken_ok_ () {
+       if test -n "$write_junit_xml"
+       then
+               write_junit_xml_testcase "$* (breakage fixed)"
+       fi
        test_fixed=$(($test_fixed+1))
        say_color error "ok $test_count - $@ # TODO known breakage vanished"
 }

This was expected to fail but didn't, which means it probably needs
some sort of attention. test_known_broken_ok_() prints this result in
the 'error' color, and test_done() re-inforces that by printing a
message, also in 'error' color:

    42 known breakage(s) vanished; please update test(s)

So, should this emit a <failure> node also, perhaps with 'type'
attribute set to "warning" or something? (<failure type="WARNING"
message="...">)

My primary aim is to display the test results in the web interface, see
e.g.
https://git.visualstudio.com/git/_build/results?buildId=128&view=ms.vss-test-web.test-result-details

The parser for JUnit XML (and in fact, the JUnit XML schema itself) do not
allow for such a warning. If you add a `<failure>`, then the build fails.

And we do not want the build to fail. Historically, I saw quite a couple
of "vanished" breakages depending on the platform where I ran the tests.

quoted

@@ -758,9 +793,58 @@ test_at_end_hook_ () {
+xml_attr_encode () {
+       # We do not translate CR to 
 because BSD sed does not handle
+       # \r in the regex. In practice, the output should not even have any
+       # carriage returns.
+       printf '%s\n' "$@" |
+       sed -e 's/&/\&/g' -e "s/'/\'/g" -e 's/"/\"/g' \
+               -e 's/</\&lt;/g' -e 's/>/\&gt;/g' \
+               -e 's/  /\&#x09;/g' -e 's/$/\&#x0a;/' -e '$s/&#x0a;$//' |
+       tr -d '\012\015'
+}

It's possible to insert a literal CR in the 'sed' expression, which
does match correctly on BSD (and MacOS). For instance:

    CR=$(printf "\r")
    sed -e "s/$CR/\
/g"

Okay. But since we are talking about displaying some chunk of text, I
would rather just delete the CR here anyway.

Ciao,
Dscho