From: Junio C Hamano <hidden> Date: 2021-12-16 23:11:16
Earlier we marked that this patch-id test is leak-sanitizer clean,
but if we read the test script carefully, it is merely because we
have too many invocations of commands in the "git log" family on the
upstream side of the pipe, hiding breakages from them.
Split the pipeline so that breakages from these commands can be
caught (not limited to aborts due to leak-sanitizer) and unmark
the script as not passing the test with leak-sanitizer in effect.
Signed-off-by: Junio C Hamano <redacted>
---
A quick grep tells me that tests 3302, 3303, 3305, 4020 and 6236
all use "git log" and still are marked as passing tests with
leak-sanitizer in effect. I've taken a deep look at none of them,
but I suspect they share the same kind of breakage.
t/t4204-patch-id.sh | 29 +++++++++++++++++------------
1 file changed, 17 insertions(+), 12 deletions(-)
Earlier we marked that this patch-id test is leak-sanitizer clean,
but if we read the test script carefully, it is merely because we
have too many invocations of commands in the "git log" family on the
upstream side of the pipe, hiding breakages from them.
Split the pipeline so that breakages from these commands can be
caught (not limited to aborts due to leak-sanitizer) and unmark
the script as not passing the test with leak-sanitizer in effect.
Signed-off-by: Junio C Hamano <redacted>
---
A quick grep tells me that tests 3302, 3303, 3305, 4020 and 6236
all use "git log" and still are marked as passing tests with
leak-sanitizer in effect. I've taken a deep look at none of them,
but I suspect they share the same kind of breakage.
This change looks good to me.
FWIW this is not a mistake on my part, but something I'm perfectly aware
of. I don't consider it to be "brekage".
We have plenty of place in the test suite where we hide exit codes on
the LHS of a pipe, or where we call a function that doesn't &&-chain its
git invocations.
In those cases we can and usually will "succeed" under LSAN, because it
allows the program to emit its full output, and will abort() at the very
end.
I have an unsubmitted logging mode (using LSAN_OPTIONS=log_path=<path>)
where I log every one of these to test-results/*, there's a lot more of
these.
But in the meantime I think the best way forward is to gradually mark
the tests that pass with LSAN as passing, to ensure that we at least
don't have regressions in the meantime. Before this we'd at least check
the "git checkout" etc. for leaks.
If I made fixing all broken &&-chains or git on the LHS of a pipe a
prerequisite for marking as passing under under LSAN I'd end up with
something that's approximately the size of [1] and more (i.e. Eric's
upcoming patches to do that).
I don't see why we'd consider perfect the enemy of the good in these
cases. Yes we won't catch the successful exit of every single git
invocation, but our tests aren't doing that now, LSAN or not. But until
that's fixed we'll at least catch some, which helps our overall memory
leak regression coverage.
More importantly it makes it a lot easier to reason about future memory
leak patches, as we'll be able to get to a 1=1 mapping of tests that
pass, and those that are marked being known to pass. I'm using that
locally to fake-fail those that start passing unexpectedly that aren't
on the list, which then helps to inform the addition of "this test now
passes with no leaks".
1. https://lore.kernel.org/git/20211213063059.19424-1-sunshine@sunshineco.com/
From: Junio C Hamano <hidden> Date: 2021-12-17 20:49:03
Ævar Arnfjörð Bjarmason [off-list ref] writes:
This change looks good to me.
FWIW this is not a mistake on my part, but something I'm perfectly aware
of. I don't consider it to be "brekage".
We have plenty of place in the test suite where we hide exit codes on
the LHS of a pipe, or where we call a function that doesn't &&-chain its
git invocations.
In those cases we can and usually will "succeed" under LSAN, because it
allows the program to emit its full output, and will abort() at the very
end.
But pipes do not hide ONLY deaths by sanitizer. And by relying on
the presence of pipe hiding deaths of git tools to mark the script
sanitizer-clean, the TEST_PASSES_SANITIZE_LEAK=true line adds an
unnecessary road-block for those who are cleaning up the "git whose
crash are hidden by being on the left hand side of the pipe"
pattern.
I do not know what to call it if not "breakage".
This change looks good to me.
FWIW this is not a mistake on my part, but something I'm perfectly aware
of. I don't consider it to be "brekage".
We have plenty of place in the test suite where we hide exit codes on
the LHS of a pipe, or where we call a function that doesn't &&-chain its
git invocations.
In those cases we can and usually will "succeed" under LSAN, because it
allows the program to emit its full output, and will abort() at the very
end.
But pipes do not hide ONLY deaths by sanitizer. And by relying on
the presence of pipe hiding deaths of git tools to mark the script
sanitizer-clean, the TEST_PASSES_SANITIZE_LEAK=true line adds an
unnecessary road-block for those who are cleaning up the "git whose
crash are hidden by being on the left hand side of the pipe"
pattern.
I do not know what to call it if not "breakage".
Yes it's broken as far as the test is concerned. I meant as far as
"GIT_TEST_PASSING_SANITIZE_LEAK" goes I consider it somewhere between
"meh" and "don't care yet".
I.e. these are pretty irrelevant for finding leaks, as we've got a huge
deluge of them elsewhere. At some point we might have a last few stray
memory leaks in git hidden by such patterns, but we're very far away
from that.
Sometimes fixing those is trivial as in 3247919a758 (commit-graph tests:
fix error-hiding graph_git_two_modes() helper, 2021-10-15), and
sometimes we'll find that the test was broken all along in some other
subtle way, as in the a046aa38ca9 (commit-graph tests: fix another
graph_git_two_modes() helper, 2021-10-15) follow-up.
But as to the "roadblock" I don't mind the
TEST_PASSES_SANITIZE_LEAK=true being removed from the script at the
slightest sign of trouble. Nobody should have to shift gears and chase
down some memory leak in "git log" just because they needed it for their
test setup.
And I'd very much prefer that to UNLEAK() just to avoid that
TEST_PASSES_SANITIZE_LEAK=true removal, because it makes fixing the leak
itself harder as far as what topic to target, re-adding
TEST_PASSES_SANITIZE_LEAK=true once it's fixed etc. goes.