Re: [PATCH] travis-ci: include the trash directories of failed tests in the trace log

On Tue, Aug 7, 2018 at 5:12 PM Lars Schneider [off-list ref] wrote:

quoted

On Aug 1, 2018, at 12:56 AM, SZEDER Gábor [off-list ref] wrote:

The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs.  The only feedback we
get from there is the build job's trace log, so...

100% agree that keeping the trash directory is valuable.

quoted

Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log.  Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.

I feel that might be complicated to work with.

It works quite conveniently in practice, that was one of the points of
the next paragraph.

How about this:
We create an publicly available Git repo (e.g. github.com/git/git-test-trash`)

I'm not sure I understand.  A publicly _writable_ repository?  That
sounds like asking for trouble.

and then we push the trash directory of every failed test run to
this repo. Then we link the commit in the logs to help people find
them.

Wouldn't that make it easier to access the trash dir?

With this patch, if there are any trash directories embedded in the
log, then the log will show a command near the very end that you can
copy-paste into a terminal, and bam! in less than 3 seconds you have
all those trash directories downloaded and unpacked.

I'm not sure how it could be made any easier than that.

quoted

Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input".  For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.

A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB.  And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.

Signed-off-by: SZEDER Gábor <redacted>
---