Re: [PATCH] travis-ci: run previously failed tests first, then slowest to fastest

Hi Junio,

On Tue, 19 Jan 2016, Junio C Hamano wrote:

larsxschneider@gmail.com writes:

quoted

From: Lars Schneider <redacted>

Use the Travis-CI cache feature to store prove test results and make
them available in subsequent builds. This allows to run previously
failed tests first and run remaining tests in slowest to fastest
order. As a result it is less likely that Travis-CI needs to wait for
a single test at the end which speeds up the test suite execution by
~2 min.

Unfortunately the cache feature is only available (for free) on the
Travis-CI Linux environment.

Suggested-by: Jeff King <redacted>
Signed-off-by: Lars Schneider <redacted>
---
 .travis.yml | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

This is cute, but isn't it useful even outside Travis's context?  I am
not suggesting to touch anything other than .travis.yml file in this
patch, but if I wanted to get the benefit from the idea in this patch
when I run my tests manually, I can just tell prove to use the cached
states, no?

You are basically talking about prove's support to modify behavior based
on previous runs. This patch does not introduce that support, it was
introduced long ago: 5099b99 (test-lib: Adjust output to be valid TAP
format, 2010-06-24). This commit also advertises that feature in t/README.

IOW, I am confused by the beginning of the log message that says
this is taking advantage of "the Travis-CI cache feature".

Lars' patch is really about Travis and its useful feature to retain state
from previous runs. AFAICT this is the feature Lars is talking about, and
it is absolutely necessary to make use of this prove feature (don't try
this with BuildHive, for example, its workspaces are typically
garbage-collected before the next CI run).

As such, I think it makes sense to talk about the Travis-CI feature in the
commit message (and not about the prove feature because we introduced
support for prove much, much earlier in Git's history, and advertised its
benefits at that time, as I stated above).

Ciao,
Dscho