Jeff King [off-list ref] writes:
Right, I think the right solution is some amount of peeling. Recognizing
that the commit sha1 is the same, or better yet, not bothering to retest
trees which have already been tested.
Yup, I also have a hack to avoid testing a version that is only
different in insignificant way (e.g. the only difference being
GIT-VERSION-GEN or Documentation/RelNotes/*) from an installed one
in the script I use after each integration attempt I do, which I use
a few times a day (that's "Meta/Dothem" if anybody is interested).
If we had some kind of persistent storage, we could do a quick:
...
The "git test" script[1] uses this strategy with git-notes as the
storage, and I've found it quite useful. I don't think we can rely on
git-notes, but I think Travis gives us some storage options. Even just a
best-effort cache directory would probably be sufficient (this is an
optimization, after all).
We do seem to use some persistence to order prove tests already, but
I do not think it helps the common case, where my end-of-day push
pushes out 'maint' and 'v2.13.3' at the same time, because the push
is made with "git push --follow-tags $there maint master next pu"
and the new tag happens to be at 'maint'. It would be nice if
Travis runs were sequential, but I often observe that it creates
jobs for these multiple branches and tags pushed at the same time,
and start running a few of them.
On Fri, Jul 14, 2017 at 07:54:16AM -0700, Junio C Hamano wrote:
quoted
The "git test" script[1] uses this strategy with git-notes as the
storage, and I've found it quite useful. I don't think we can rely on
git-notes, but I think Travis gives us some storage options. Even just a
best-effort cache directory would probably be sufficient (this is an
optimization, after all).
We do seem to use some persistence to order prove tests already, but
I do not think it helps the common case, where my end-of-day push
pushes out 'maint' and 'v2.13.3' at the same time, because the push
is made with "git push --follow-tags $there maint master next pu"
and the new tag happens to be at 'maint'. It would be nice if
Travis runs were sequential, but I often observe that it creates
jobs for these multiple branches and tags pushed at the same time,
and start running a few of them.
Ah, right, I didn't think about how these are racing. You'd need storage
which allows some kind of atomic operation to "claim" the tree as a
work-in-progress (and anybody who loses the race to get the lock would
have to spin waiting for the winner to tell them the real status).
I don't know if Travis's cache storage is up to that challenge. We could
probably build such a lock on top of third-party storage, but things are
rapidly getting more complex.
-Peff
On 14 Jul 2017, at 17:32, Jeff King [off-list ref] wrote:
On Fri, Jul 14, 2017 at 07:54:16AM -0700, Junio C Hamano wrote:
quoted
quoted
The "git test" script[1] uses this strategy with git-notes as the
storage, and I've found it quite useful. I don't think we can rely on
git-notes, but I think Travis gives us some storage options. Even just a
best-effort cache directory would probably be sufficient (this is an
optimization, after all).
We do seem to use some persistence to order prove tests already, but
I do not think it helps the common case, where my end-of-day push
pushes out 'maint' and 'v2.13.3' at the same time, because the push
is made with "git push --follow-tags $there maint master next pu"
and the new tag happens to be at 'maint'. It would be nice if
Travis runs were sequential, but I often observe that it creates
jobs for these multiple branches and tags pushed at the same time,
and start running a few of them.
Ah, right, I didn't think about how these are racing. You'd need storage
which allows some kind of atomic operation to "claim" the tree as a
work-in-progress (and anybody who loses the race to get the lock would
have to spin waiting for the winner to tell them the real status).
I don't know if Travis's cache storage is up to that challenge. We could
probably build such a lock on top of third-party storage, but things are
rapidly getting more complex.
I think we shouldn't go there because of the complexity. I reached out
to TravisCI and asked about the "hash build twice" problem [1]. Unfortunately,
I got no response, yet. The issue could also be considered a feature as you
could perform different actions in your TravisCI configuration based on
the branch name.
I think Junio's original suggestions for the Windows build makes a lot
of sense because it saves Dscho's compute resources:
--- a/ci/run-windows-build.sh
+++ b/ci/run-windows-build.sh
@@ -12,6 +12,12 @@ test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit
BRANCH=$1
COMMIT=$2
+if TAG=$(git describe --exact-match "$COMMIT" 2>/dev/null)
+then
+ echo "Tip of $BRANCH exactly at $TAG"
+ exit 0
+fi
+
gfwci () {
local CURL_ERROR_CODE HTTP_CODE
CONTENT_FILE=$(mktemp -t "git-windows-ci-XXXXXX")
However, I don't think we need to do the same for the builds that
use TravisCI resources. If they would be concerned about that then
they wouldn't build the same hash twice in the first place.
- Lars
[1] https://twitter.com/kit3bus/status/885902189692112896