Hi Stolee,

On Fri, 31 May 2019, Derrick Stolee wrote:

On 5/30/2019 2:24 PM, Derrick Stolee wrote:

quoted

Further, these tests failed

t3400-rebase.sh                           (Wstat: 256 Tests: 28 Failed: 2)
  Failed tests:  20, 28
  Non-zero exit status: 1
t3420-rebase-autostash.sh                 (Wstat: 256 Tests: 38 Failed: 6)
  Failed tests:  6, 13, 16, 23, 26, 33
  Non-zero exit status: 1
t3404-rebase-interactive.sh               (Wstat: 256 Tests: 110 Failed: 5)
  Failed tests:  3, 9-10, 100-101
  Non-zero exit status: 1
t5521-pull-options.sh                     (Wstat: 256 Tests: 19 Failed: 1)
  Failed test:  3
  Non-zero exit status: 1
t5551-http-fetch-smart.sh                 (Wstat: 256 Tests: 37 Failed: 1)
  Failed test:  26
  Non-zero exit status: 1

They don't fail locally, so perhaps we shouldn't blindly trust the coverage data
until I work out why these errors occurred. (Many of the cases I called out
above I couldn't hit locally with a die() statement.)

These tests all failed during the second run that set optional GIT_TEST
environment variables. Specifically, GIT_TEST_REBASE_USE_BUILTIN=false
caused these tests to break. We now output this message:

	warning: the rebase.useBuiltin support has been removed!
	See its entry in 'git help config' for details.

I'm removing that variable from the build definition.

Would it make sense to have a file in t/ (or a script-let in ci/)
specifying all of the `GIT_TEST_*` variables that are currently supported
(and that actually make sense to be set)?

I saw a similar issue recently in a now-defunct Azure Pipeline that also
tried to replicate what half of the `linux-gcc` job [*1*] does: to run the
test suite with those variables overriding the defaults. That Pipeline
broke for the exact same reason you mentioned: we now handle
`GIT_TEST_REBASE_USE_BUILTIN` by showing that warning.

And issues like this could easily be avoided if we had, say,
`ci/non-standard-settings.sh` that simply set all those `GIT_TEST_*`
variables in the way that the `linux-gcc` job does (and of course, this
job should then source that file instead of duplicating those
assignments).

What do you think?
Dscho

Footnote *1*: It is a thorn in my side ever since I started work on our
Azure Pipeline support that the `linux-gcc` job actually runs *two* jobs:
it runs the vanilla test suite, and then it runs it again after setting
all supported `GIT_TEST_*` variables to the non-default settings. This
almost doubles the running time of that job, often making it the very last
job to finish, and it also makes it unclear whether a test failure stems
from said `GIT_TEST_*` settings or not.

I got so annoyed by this, in fact, that I finally broke down and opened
https://github.com/gitgitgadget/git/issues/242.