Re: [PATCH v2 09/27] userdiff tests: match full hunk headers

Am 23.02.21 um 14:11 schrieb Ævar Arnfjörð Bjarmason:

On Wed, Feb 17 2021, Junio C Hamano wrote:

quoted

Sounds good.  It shouldn't be too hard to satisfy both camps,
i.e. the quoted demonstrates one way to allow test writers to
give expectation in-place in the single test file, and replacing
how it uses "grep" to check the output with test_cmp or whatever
wouldn't make the resulting tests too hard to write and maintain.

It doesn't satisfy both camps, because I'd like to convert all these
tests to test_cmp because for a subsequent refactoring of userdiff.c by
me or others I don't know in advance what might break, so I'd like to
assert the exact current behavior.

Whereas your patch provides a way to opt-in individual tests to a
test_cmp-alike, but leaves the rest at grepping for the "RIGHT"
substring. Failures in the tests who aren't opted-in will be hidden.

It also means that subsequent changes to the behavior in the form of
submitted patches won't be as self-documenting, e.g. I've wondered if we
could introduce a case to balance parens in this code (sometimes C
function declarations stretch across lines), and there's e.g. the
arbitrary limit of 80 bytes on the line (which to be fair, we don't
curretly have tests for).

Anyway, as noted in [1] I don't see how this custom format of grepping
stuff out of plain-text files is simpler, particularly when its behavior
would start to rely on other things like "# HEADER |right()|" whose
behavior is a function of what we grep/sed when/where in the logic
driving the tests.

But if you & Johannes S. disagree with that I don't really say a way
forward with this series. I think e.g. squashing 09/27 into the rest
would make things simpler/less verbose, but the end-state would still be
matching the full hunk line, and if that's not something that's wanted
in any shape or form as a default...

1. https://lore.kernel.org/git/87h7mba3h3.fsf@evledraar.gmail.com/ (local)

I could live with a version of Junio's suggestion that is not opt-in,
i.e., the checks are mandatory and exact. The important point is that
there is only one file per test case; that would still count as
"sufficiently simple" in my book.

-- Hannes