Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data
From: Junio C Hamano <hidden>
Date: 2021-11-19 17:11:51
Subsystem:
the rest · Maintainer:
Linus Torvalds
Possibly related (same subject, not in this thread)
- 2021-11-19 · Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data · Carlo Arenas <hidden>
- 2021-11-19 · Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data · René Scharfe <hidden>
- 2021-11-19 · Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data · Ævar Arnfjörð Bjarmason <hidden>
- 2021-11-18 · Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data · René Scharfe <hidden>
- 2021-11-18 · Re: [PATCH v13 3/3] grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data · Junio C Hamano <hidden>
Ævar Arnfjörð Bjarmason [off-list ref] writes:
On Thu, Nov 18 2021, René Scharfe wrote:quoted
Makes sense to me. It gives a better starting point to solve the issue afresh without getting entangled in mind-melting boolean expressions.Yes, agreed. As noted I haven't had time to dig deeply into this, but from what I've seen so far there doesn't seem to be any obvious way forward in terms of a quick fix. I thought perhaps your patch would be that (but I haven't looked into it carefully enough), but since you're on-board with reverting & retrying.
OK. Here is what I'll queue 'master', in the hope of later merging down and be part of v2.34.1. Thanks, all. ----- >8 --------- >8 --------- >8 --------- >8 ----- Subject: [PATCH] Revert "grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data" This reverts commit ae39ba431ab861548eb60b4bd2e1d8b8813db76f, as it breaks "grep" when looking for a string in non UTF-8 haystack, when linked with certain versions of PCREv2 library. Signed-off-by: Junio C Hamano <redacted> --- grep.c | 6 ++--- t/t7812-grep-icase-non-ascii.sh | 48 --------------------------------- 2 files changed, 2 insertions(+), 52 deletions(-)
diff --git a/grep.c b/grep.c
index f6e113e9f0..fe847a0111 100644
--- a/grep.c
+++ b/grep.c@@ -382,10 +382,8 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt } options |= PCRE2_CASELESS; } - if ((!opt->ignore_locale && !has_non_ascii(p->pattern)) || - (!opt->ignore_locale && is_utf8_locale() && - has_non_ascii(p->pattern) && !(!opt->ignore_case && - (p->fixed || p->is_fixed)))) + if (!opt->ignore_locale && is_utf8_locale() && has_non_ascii(p->pattern) && + !(!opt->ignore_case && (p->fixed || p->is_fixed))) options |= (PCRE2_UTF | PCRE2_MATCH_INVALID_UTF); #ifdef GIT_PCRE2_VERSION_10_36_OR_HIGHER
diff --git a/t/t7812-grep-icase-non-ascii.sh b/t/t7812-grep-icase-non-ascii.sh
index 22487d90fd..e5d1e4ea68 100755
--- a/t/t7812-grep-icase-non-ascii.sh
+++ b/t/t7812-grep-icase-non-ascii.sh@@ -53,54 +53,6 @@ test_expect_success REGEX_LOCALE 'pickaxe -i on non-ascii' ' test_cmp expected actual ' -test_expect_success GETTEXT_LOCALE,PCRE 'log --author with an ascii pattern on UTF-8 data' ' - cat >expected <<-\EOF && - Author: <BOLD;RED>À Ú Thor<RESET> <author@example.com> - EOF - test_write_lines "forth" >file4 && - git add file4 && - git commit --author="À Ú Thor <author@example.com>" -m sécond && - git log -1 --color=always --perl-regexp --author=".*Thor" >log && - grep Author log >actual.raw && - test_decode_color <actual.raw >actual && - test_cmp expected actual -' - -test_expect_success GETTEXT_LOCALE,PCRE 'log --committer with an ascii pattern on ISO-8859-1 data' ' - cat >expected <<-\EOF && - Commit: Ç<BOLD;RED> O Mîtter <committer@example.com><RESET> - EOF - test_write_lines "fifth" >file5 && - git add file5 && - GIT_COMMITTER_NAME="Ç O Mîtter" && - GIT_COMMITTER_EMAIL="committer@example.com" && - git -c i18n.commitEncoding=latin1 commit -m thïrd && - git -c i18n.logOutputEncoding=latin1 log -1 --pretty=fuller --color=always --perl-regexp --committer=" O.*" >log && - grep Commit: log >actual.raw && - test_decode_color <actual.raw >actual && - test_cmp expected actual -' - -test_expect_success GETTEXT_LOCALE,PCRE 'log --grep with an ascii pattern on UTF-8 data' ' - cat >expected <<-\EOF && - sé<BOLD;RED>con<RESET>d - EOF - git log -1 --color=always --perl-regexp --grep="con" >log && - grep con log >actual.raw && - test_decode_color <actual.raw >actual && - test_cmp expected actual -' - -test_expect_success GETTEXT_LOCALE,PCRE 'log --grep with an ascii pattern on ISO-8859-1 data' ' - cat >expected <<-\EOF && - <BOLD;RED>thïrd<RESET> - EOF - git -c i18n.logOutputEncoding=latin1 log -1 --color=always --perl-regexp --grep="th.*rd" >log && - grep "th.*rd" log >actual.raw && - test_decode_color <actual.raw >actual && - test_cmp expected actual -' - test_expect_success GETTEXT_LOCALE,LIBPCRE2 'PCRE v2: setup invalid UTF-8 data' ' printf "\\200\\n" >invalid-0x80 && echo "ævar" >expected &&
--
2.34.0-202-gd9146917d7