From: Andrei Rybak <hidden> Date: 2018-08-03 13:38:09
I was tweaking is_scissors_line function in mailinfo.c and tried writing
some tests for it. And discovered that existing test for git am option
--scissors is broken. I then confirmed that by intentionally breaking
is_scissors_line, like this:
--- 8< ---
Subject: [PATCH 1/2] mailinfo.c: intentionally break is_scissors_line
It seems that tests for "git am" don't actually test the --scissor
option logic. Break is_scissors_line function by using bogus symbols to
be able to check the tests.
Note that test suite does not pass with this patch applied. The
expected failure does not happen.
---
mailinfo.c | 4 ++--
t/t4150-am.sh | 3 ++-
2 files changed, 4 insertions(+), 3 deletions(-)
@@ -412,7 +412,8 @@ test_expect_success 'am with failing post-applypatch hook' 'test_cmphead.expectedhead.actual'-test_expect_success'am --scissors cuts the message at the scissors line''+# Test should fail, but succeeds+test_expect_failure'am --scissors cuts the message at the scissors line''rm-fr.git/rebase-apply&&gitreset--hard&&gitcheckoutsecond&&--->8---
Here's a proof-of-concept patch for the test, to make it actually fail
when is_scissors_line is broken. It is the easiest way to do so, that I
could come up with, it is not ready to be applied. I think the two
tests for --scissors should be rewritten pretty much from scratch, with
more obvious naming of files used.
(I made the changes to files in both tests the same just to be able to
re-use file "no-scissors-patch.eml", it's not relevant to the scissor
line in the commit messages.)
--- 8< ---
Subject: [PATCH 2/2] t4150-am.sh: fix test for --scissors
Test for option --scissors should check that the eml file with a scissor
line inside will be cut up and only the part under the cut will be
turned into commit.
However, the test for --scissors generates eml file without such line.
Fix the test for --scissors option.
Signed-off-by: Andrei Rybak <redacted>
---
t/t4150-am.sh | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
@@ -417,7 +422,7 @@ test_expect_failure 'am --scissors cuts the message at the scissors line' 'rm-fr.git/rebase-apply&&gitreset--hard&&gitcheckoutsecond&&-gitam--scissorsscissors-patch.eml&&+gitam--scissorsno-scissors-patch.eml&&test_path_is_missing.git/rebase-apply&&gitdiff--exit-codescissors&&test_cmp_revscissorsHEAD----->8---
From: Andrei Rybak <hidden> Date: 2018-08-03 22:39:47
Tests for "git am --[no-]scissors" [1] work in the following way:
1. Create files with commit messages
2. Use these files to create expected commits
3. Generate eml file with patch from expected commits
4. Create commits using git am with these eml files
5. Compare these commits with expected
The test for "git am --scissors" is supposed to take a message with a
scissors line above commit message and demonstrate that only the text
below the scissors line is included in the commit created by invocation
of "git am --scissors". However, the setup of the test uses commits
without the scissors line in the commit message, therefore creating an
eml file without scissors line.
This can be checked by intentionally breaking is_scissors_line function
in mailinfo.c. Test t4150-am.sh should fail, but does not.
Fix broken test by generating only one eml file--with scissors line, and
by using it both for --scissors and --no-scissors. To clarify the
intention of the test, give files and tags more explicit names.
[1]: introduced in bf72ac17d (t4150: tests for am --[no-]scissors,
2015-07-19)
Signed-off-by: Andrei Rybak <redacted>
---
Applies on top of 980a3d3dd (Merge branch 'pt/am-tests', 2015-08-03).
This patch is also available at
https://github.com/rybak/git fix-am-scissors-test
t/t4150-am.sh | 40 ++++++++++++++++++++--------------------
1 file changed, 20 insertions(+), 20 deletions(-)
From: Andrei Rybak <hidden> Date: 2018-08-04 18:11:01
Tests for "git am --[no-]scissors" [1] work in the following way:
1. Create files with commit messages
2. Use these files to create expected commits
3. Generate eml file with patch from expected commits
4. Create commits using git am with these eml files
5. Compare these commits with expected
The test for "git am --scissors" is supposed to take a message with a
scissors line and demonstrate that the subject line from the e-mail
itself is overridden by the in-body "Subject:" header and that only text
below the scissors line is included in the commit message of the commit
created by the invocation of "git am --scissors". However, the setup of
the test incorrectly uses a commit without the scissors line and in-body
"Subject:" header in the commit message, and thus, creates eml file not
suitable for testing of "git am --scissors".
This can be checked by intentionally breaking is_scissors_line function
in mailinfo.c, for example, by changing string ">8", which is used by
the test. With such change the test should fail, but does not.
Fix broken test by generating eml file with scissors line and in-body
header "Subject:". Since the two tests for --scissors and --no-scissors
options are there to test cutting or keeping the commit message, update
both tests to change the test file in the same way, which allows us to
generate only one eml file to be passed to git am. To clarify the
intention of the test, give files and tags more explicit names.
[1]: introduced in bf72ac17d (t4150: tests for am --[no-]scissors,
2015-07-19)
Signed-off-by: Andrei Rybak <redacted>
---
Applies on top of 980a3d3dd (Merge branch 'pt/am-tests', 2015-08-03).
This patch is also available at
https://github.com/rybak/git fix-am-scissors-test-v2
Changes since v1:
- Reword commit message after feedback from Junio
- Keep the empty line under scissors in the test e-mail, as it does not
affect the test
t/t4150-am.sh | 39 ++++++++++++++++++++-------------------
1 file changed, 20 insertions(+), 19 deletions(-)
Hi,
I've taken a look at the original test, and it is pretty broken. My
deepest apologies for this mess.
On Sun, Aug 5, 2018 at 2:10 AM, Andrei Rybak [off-list ref] wrote:
Tests for "git am --[no-]scissors" [1] work in the following way:
1. Create files with commit messages
2. Use these files to create expected commits
3. Generate eml file with patch from expected commits
4. Create commits using git am with these eml files
5. Compare these commits with expected
The test for "git am --scissors" is supposed to take a message with a
scissors line and demonstrate that the subject line from the e-mail
itself is overridden by the in-body "Subject:" header and that only text
below the scissors line is included in the commit message of the commit
created by the invocation of "git am --scissors". However, the setup of
the test incorrectly uses a commit without the scissors line and in-body
"Subject:" header in the commit message, and thus, creates eml file not
suitable for testing of "git am --scissors".
I think what really happened was that I simply forgot that the first
line of the commit message would be pulled out into the formatted
patch's "Subject" header, and would thus not be affected by the
scissors line :-S.
This can be checked by intentionally breaking is_scissors_line function
in mailinfo.c, for example, by changing string ">8", which is used by
the test. With such change the test should fail, but does not.
The main reason why the test still passes even with a broken
is_scissors_line() would be because it uses the wrong patch to pass to
"git am --scissors" -- it uses the patch _without_ a scissors line
rather than the patch _with_ the scissors line.
However, after fixing this problem, which I'll call problem (1), the
test will actually fail, due to:
(2) The trees of the commits `scissors` and `no-scissors` not being
identical, thus making test_cmp_rev fail even though the commit
messages of the commits are identical.
(3) As mentioned above, the test not accounting for the first line of
the commit message being used as the "Subject" header and thus not
affected by the scissors line.
So, there are 3 problems that will need to be fixed.
quoted hunk
Fix broken test by generating eml file with scissors line and in-body
header "Subject:". Since the two tests for --scissors and --no-scissors
options are there to test cutting or keeping the commit message, update
both tests to change the test file in the same way, which allows us to
generate only one eml file to be passed to git am. To clarify the
intention of the test, give files and tags more explicit names.
[1]: introduced in bf72ac17d (t4150: tests for am --[no-]scissors,
2015-07-19)
Signed-off-by: Andrei Rybak <redacted>
---
t/t4150-am.sh | 39 ++++++++++++++++++++-------------------
1 file changed, 20 insertions(+), 19 deletions(-)
Nit: I'm not quite sure about naming the tag "scissors-used" though,
since this commit was not created from the output of "git am
--scissors". Maybe it should be named `commit-without-scissors-line`
or something?
This hunk removes the line:
git format-patch --stdout scissors^ >scissors-patch.eml &&
without a corresponding replacement, but that is fine because the test
should not be using a patch without a scissors line.
Nit: I'm not quite sure about naming the tag "scissors-used" though,
since this commit was not created from the output of "git am
--scissors". Maybe it should be named `commit-without-scissors-line`
or something?
Nit: Likewise, perhaps this tag could be named `commit-with-scissors-line`?
How about "expected-for-scissors" and "expected-for-no-scissors"?
Junio, I'll send out v3 with updated tag names, if that's OK.
Also, squash-able patch below.
So, this patch fixes the 3 problems with the tests, and so looks correct to me.
Thank you for such thorough review.
--- 8< ---
From: Andrei Rybak <redacted>
Date: Mon, 6 Aug 2018 19:29:03 +0200
Subject: [PATCH] squash! t4150: fix broken test for am --scissors
clarify tag names
---
t/t4150-am.sh | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
From: Andrei Rybak <hidden> Date: 2018-08-06 17:49:44
Tests for "git am --[no-]scissors" [1] work in the following way:
1. Create files with commit messages
2. Use these files to create expected commits
3. Generate eml file with patch from expected commits
4. Create commits using git am with these eml files
5. Compare these commits with expected
The test for "git am --scissors" is supposed to take an e-mail with a
scissors line and in-body "Subject:" header and demonstrate that the
subject line from the e-mail itself is overridden by the in-body header
and that only text below the scissors line is included in the commit
message of the commit created by the invocation of "git am --scissors".
However, the setup of the test incorrectly uses a commit without the
scissors line and without the in-body header in the commit message,
producing eml file not suitable for testing of "git am --scissors".
This can be checked by intentionally breaking is_scissors_line function
in mailinfo.c, for example, by changing string ">8", which is used by
the test. With such change the test should fail, but does not.
Fix broken test by generating eml file with scissors line and in-body
header "Subject:". Since the two tests for --scissors and --no-scissors
options are there to test cutting or keeping the commit message, update
both tests to change the test file in the same way, which allows us to
generate only one eml file to be passed to git am. To clarify the
intention of the test, give files and tags more explicit names.
[1]: introduced in bf72ac17d (t4150: tests for am --[no-]scissors,
2015-07-19)
Signed-off-by: Andrei Rybak <redacted>
---
Applies on top of 980a3d3dd (Merge branch 'pt/am-tests', 2015-08-03).
This patch is also available at
https://github.com/rybak/git fix-am-scissors-test-v3
Only changes since v2 are more clear tag names.
t/t4150-am.sh | 39 ++++++++++++++++++++-------------------
1 file changed, 20 insertions(+), 19 deletions(-)
Nit: I'm not quite sure about naming the tag "scissors-used" though,
since this commit was not created from the output of "git am
--scissors". Maybe it should be named `commit-without-scissors-line`
or something?