Re: Prove "Tests out of sequence" Error

On Fri, Oct 21, 2016 at 8:35 AM, Jeff King [off-list ref] wrote:

On Fri, Oct 21, 2016 at 08:29:49AM -0700, Jacob Keller wrote:

quoted

quoted

The Travis tests do exactly this (along with --tee to actually save the
output). It seems like a minor miracle that this is the first test
output that has actually triggered as TAP input. I'd suggest that the
problem is not in the test, though, but that our "--verbose" option is
unsuitable for using with a TAP harness.

The obvious fix would be to send "--verbose" output to stderr, but I
suspect that would end up annoying for people who do:

  ./t5547-push-quarantine.sh -v | less

to read long output. Probably we need some option like "--log" which
logs in the same way that "--tee" does, but _without_ sending the data
to stdout. Naively, that just means replacing the "tee" invocation with
"cat", but I suspect it will be a lot more complicated than that,
because we still need to let the TAP output go to stdout.

Can we determine that we're running with something monitoring the TAP
output? Because then we could make verbose go to stderr instead
dynamically?

I think $HARNESS_ACTIVE could tell us that. But the hard part isn't
activating it; it's directing the verbose output to the log without
sending it to stdout.

See the patch I posted later in the thread, and my musings on
auto-activating it. I guess we could do so safely when we see
$HARNESS_ACTIVE along with "--tee" and "--verbose", though I don't know
if it's worth the trouble.

For $HARNESS_ACTIVE with _just_ "--verbose", I don't think it would be a
good idea to activate it. We should either silently ignore --verbose
then, or complain and die.

We should probably do that to make sure people realize what might
happen. I read your series and it has a good explanation of the
possible alternatives. I like the approach you chose.

Thanks,
Jake

-Peff