Re: [PATCH] tests: handle "funny" exit code 127 produced by MSVC-compiled exes
From: Jeff King <hidden>
Date: 2023-10-30 17:57:00
On Mon, Oct 30, 2023 at 03:45:32PM +0000, Johannes Schindelin via GitGitGadget wrote:
Now, `jk/tree-name-and-depth-limit` introduces a pair of test cases that expect a command that produces a stack overflow to fail, which it typically does with exit code 139 (which means SIGSEGV).
I think you're misinterpreting the purpose of the tests from that series; they're not intended to segfault. Quoting from t6700: # We'll test against two depths here: a small one that will let us check the # behavior of the config setting easily, and a large one that should be # forbidden by default. Testing the default depth will let us know whether our # default is enough to prevent segfaults on systems that run the tests. So for the "big tree" tests in that file, we are looking for a controlled failure rather than a segfault. And indeed, the end of that series already lowered the default to accommodate the msys windows build; see the discussion in 4d5693ba05 (lower core.maxTreeDepth default to 2048, 2023-08-31). So I think the test is working as designed here: it is showing us that the default value is not sufficient to protect MSVC builds from running out of stack space. There are a few options there: 1. We can lower the default everywhere. 2. We can lower it just for MSVC builds. 3. We can accept the situation and skip the tests for that build. There's a bit more discussion in the commit I referenced above.
Let's work around this by: 1) recording which C compiler was used, and 2) adding an MSVC-only exception to `test_must_fail` to treat 127 as a regular failure. There is a slight downside of this approach in that a real missing command could be mistaken for a failure. However, this would be caught on other platforms, and besides, we use `test_must_fail` only for `git` and `scalar` anymore, and we can be pretty certain that both are there.
I think there is another much worse downside to your patch: we will stop noticing when MSVC builds segfault in the tests. The purpose of test_must_fail is to allow controlled and expected failure returns from the command, but still report on unexpected situations (signal death, command not found, and so on). -Peff