Re: [PATCH v1 3/3] travis-ci: run Git bisect on failed tests
From: Pranit Bauva <hidden>
Date: 2016-06-16 02:19:33
Hey Lars, On Sun, May 22, 2016 at 4:30 PM, [off-list ref] wrote:
quoted hunk ↗ jump to hunk
From: Lars Schneider <redacted> Junio usually pushes many commits at once to the public "pu"/"next"/ "master" branches. If a test fails then it is not obvious what commit caused the failure. Therefore we run Git bisect with the merge base between the failing rev and its more stable branch ("next" for "pu", "master" for "next", and "maint" for "master") as good ref to find the offending commit. This is only enabled on "github.com/git/git" because there we can assume that all relevant branches are up to date. Signed-off-by: Lars Schneider <redacted> --- .travis.yml | 2 +- ci/test-report.sh | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+), 1 deletion(-)diff --git a/.travis.yml b/.travis.yml index 81d2027..922807b 100644 --- a/.travis.yml +++ b/.travis.yml@@ -101,7 +101,7 @@ script: make --quiet test after_failure: - > : '<-- Click here to see detailed test output! '; - ./ci/test-report.sh + ./ci/test-report.sh $TRAVIS_REPO_SLUG $TRAVIS_BRANCH; notifications: email: falsediff --git a/ci/test-report.sh b/ci/test-report.sh index d08a999..8f7adad 100755 --- a/ci/test-report.sh +++ b/ci/test-report.sh@@ -1,5 +1,12 @@ #!/bin/sh # +# Print test results and run Git bisect on failed tests. +# +REPO_ORG_NAME=$1 +CURRENT_BRANCH_NAME=$2 + + +# # Print test results # for TEST_EXIT in t/test-results/*.exit@@ -17,3 +24,61 @@ do fi done + +# +# Run Git bisect +# +run_bisect () { + TEST_SCRIPT=$1 + BAD_REV=$2 + GOOD_RV=$3 + TMPDIR=$(mktemp -d -t "ci-report-bisect-XXXXXX" 2>/dev/null) + cat > "$TMPDIR/bisect-run.sh" <<EOF
If you are doing a re-roll, then you could probably fix the style issue. ' cat >"$TMPDIR..." '
+
+EOF
+ chmod +x "$TMPDIR/bisect-run.sh"
+ git bisect start $BAD_REV $GOOD_RV
+ git bisect run "$TMPDIR/bisect-run.sh"
+ if test -e ./t/$TEST_SCRIPT.sh && make --jobs=2 >/dev/null 2>&1
+ then
+ cd t && ./$TEST_SCRIPT.sh >/dev/null 2>&1
+ else
+ # If the test file does not exist or the build fails then tell
+ # Git bisect to skip the commit.
+ exit 125
+ fi
+ git bisect reset >/dev/null 2>&1
+}
+
+case "$CURRENT_BRANCH_NAME" in
+ master) STABLE_BRANCH="maint";;
+ next) STABLE_BRANCH="master";;
+ pu) STABLE_BRANCH="next";;
+esac
+
+if test "$REPO_ORG_NAME" = "git/git" && test -n $STABLE_BRANCH
+then
+ BAD_REV=$(git rev-parse HEAD)
+
+ # Travis CI clones are shallow. It is possible that the last good revision
+ # was not fetched, yet. Therefore we need to fetch all commits on the
+ # stable branch.
+ git config remote.origin.fetch "+refs/heads/$STABLE_BRANCH:refs/remotes/origin/$STABLE_BRANCH"
+ git fetch --unshallow --quiet
+ LAST_GOOD_REV=$(git merge-base $BAD_REV "remotes/origin/$STABLE_BRANCH")
+
+ for TEST_EXIT in t/test-results/*.exit
+ do
+ if test "$(cat "$TEST_EXIT")" != "0"
+ then
+ TEST="${TEST_EXIT%.exit}"
+ TEST_SCRIPT=${TEST#t/test-results/}
+ echo "------------------------------------------------------------------------"
+ echo " $(tput setaf 1)${TEST} Bisect$(tput sgr0)"
+ echo "------------------------------------------------------------------------"
+ run_bisect $TEST_SCRIPT $BAD_REV $LAST_GOOD_REV
+ echo ""
+ echo ""
+ fi
+ done
+fiRegards, Pranit Bauva