From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-07 20:02:20
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
--cone' that isn't necessary, especially with Elijah doing some work in that
space right now.
[1] https://github.com/git-for-windows/git/issues/3498 [2]
https://lore.kernel.org/git/pull.1043.git.1632160658.gitgitgadget@gmail.com
Thanks, -Stolee
Derrick Stolee (3):
sparse-checkout: fix segfault on malformed patterns
sparse-checkout: fix OOM error with mixed patterns
sparse-checkout: refuse to add to bad patterns
builtin/sparse-checkout.c | 5 ++++-
dir.c | 5 +----
t/t1091-sparse-checkout-builtin.sh | 31 +++++++++++++++++++++++++++++-
3 files changed, 35 insertions(+), 6 deletions(-)
base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1069%2Fderrickstolee%2Fsparse-checkout%2Finput-bug-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1069/derrickstolee/sparse-checkout/input-bug-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1069
--
gitgitgadget
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-07 20:02:23
From: Derrick Stolee <redacted>
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.
One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
no invalid.
The fix here is to stop trying to remove from the hashset. Better to leave
bad data in the sparse-checkout matching logic (with a warning) than to
segfault. If we are in this state, then we are already traversing into
undefined behavior, so this change to keep the entry in the hashset is no
worse than removing it.
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
dir.c | 3 ---
t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
@@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern/* we already included this at the parent level */warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"),given->pattern);-hashmap_remove(&pl->parent_hashmap,&translated->ent,&data);-free(data);-free(translated);}return;
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-07 20:02:26
From: Derrick Stolee <redacted>
Add a test to t1091-sparse-checkout-builtin.sh that would result in an
infinite loop and out-of-memory error before this change. The issue
relies on having non-cone-mode patterns while trying to modify the
patterns in cone-mode.
The fix is simple, allowing us to break from the loop when the input
path does not contain a slash, as the "dir" pattern we added does not.
This is only a fix to the critical out-of-memory error. A better
response to such a strange state will follow in a later change.
Reported-by: Calbabreaker <redacted>
Helped-by: Taylor Blau [off-list ref]
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-07 20:02:27
From: Derrick Stolee <redacted>
When in cone mode sparse-checkout, it is unclear how 'git
sparse-checkout add <dir1> ...' should behave if the existing
sparse-checkout file does not match the cone mode patterns. Change the
behavior to fail with an error message about the existing patterns.
Also, all cone mode patterns start with a '/' character, so add that
restriction. This is necessary for our example test 'cone mode: warn on
bad pattern', but also requires modifying the example sparse-checkout
file we use to test the warnings related to recognizing cone mode
patterns.
This error checking would cause a failure further down the test script
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 3 +++
dir.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 7 +++++--
3 files changed, 9 insertions(+), 3 deletions(-)
@@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' 'gitsparse-checkoutinit&&gitsparse-checkoutadddir&&gitconfigcore.sparseCheckoutConetrue&&-gitsparse-checkoutadddir+test_must_failgitsparse-checkoutadddir2>err&&+grep"existing sparse-checkout patterns do not use cone mode"err)'
@@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' '' test_expect_success'add to sparse-checkout''-catrepo/.git/info/sparse-checkout>expect&&+catrepo/.git/info/sparse-checkout>old&&+test_when_finishedcpoldrepo/.git/info/sparse-checkout&&cat>add<<-\EOF&&pattern1/folder1/pattern2EOF+catold>expect&&catadd>>expect&&git-Creposparse-checkoutadd--stdin<add&&git-Creposparse-checkoutlist>actual&&
On Tue, Dec 7, 2021 at 12:02 PM Derrick Stolee via GitGitGadget
[off-list ref] wrote:
From: Derrick Stolee <redacted>
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.
One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
no invalid.
s/no invalid/now invalid/ ?
quoted hunk
The fix here is to stop trying to remove from the hashset. Better to leave
bad data in the sparse-checkout matching logic (with a warning) than to
segfault. If we are in this state, then we are already traversing into
undefined behavior, so this change to keep the entry in the hashset is no
worse than removing it.
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
dir.c | 3 ---
t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
@@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern/* we already included this at the parent level */warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"),given->pattern);-hashmap_remove(&pl->parent_hashmap,&translated->ent,&data);-free(data);-free(translated);}return;
On Tue, Dec 7, 2021 at 12:02 PM Derrick Stolee via GitGitGadget
[off-list ref] wrote:
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
I was only able to find what I think is a small typo in one of the
commits. Everything else looks good to me.
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
Sorry for missing that series earlier. Glad we've got some of them now.
On Tue, Dec 7, 2021 at 12:02 PM Derrick Stolee via GitGitGadget
[off-list ref] wrote:
quoted
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
I was only able to find what I think is a small typo in one of the
commits. Everything else looks good to me.
Thanks. I'll give this a few more days for more feedback before sending
a v2 with that fix.
quoted
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
Sorry for missing that series earlier. Glad we've got some of them now.
The fault was my own for dropping this during a particularly busy time.
Thanks,
-Stolee
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-10 15:18:18
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
--cone' that isn't necessary, especially with Elijah doing some work in that
space right now.
[1] https://github.com/git-for-windows/git/issues/3498 [2]
https://lore.kernel.org/git/pull.1043.git.1632160658.gitgitgadget@gmail.com
Thanks, -Stolee
Derrick Stolee (4):
sparse-checkout: fix segfault on malformed patterns
sparse-checkout: fix OOM error with mixed patterns
sparse-checkout: refuse to add to bad patterns
amend! sparse-checkout: fix segfault on malformed patterns
builtin/sparse-checkout.c | 5 ++++-
dir.c | 5 +----
t/t1091-sparse-checkout-builtin.sh | 31 +++++++++++++++++++++++++++++-
3 files changed, 35 insertions(+), 6 deletions(-)
base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1069%2Fderrickstolee%2Fsparse-checkout%2Finput-bug-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1069/derrickstolee/sparse-checkout/input-bug-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1069
Range-diff vs v1:
1: becbee16d2e ! 1: a0e3dd335c9 sparse-checkout: fix segfault on malformed patterns
@@ Commit message
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett [off-list ref]
+ Reviewed-by: Elijah Newren [off-list ref]
Signed-off-by: Derrick Stolee [off-list ref]
## dir.c ##
2: 239bf23eacb ! 2: 86fbf130c03 sparse-checkout: fix OOM error with mixed patterns
@@ Commit message
Reported-by: Calbabreaker [off-list ref]
Helped-by: Taylor Blau [off-list ref]
+ Reviewed-by: Elijah Newren [off-list ref]
Signed-off-by: Derrick Stolee [off-list ref]
## builtin/sparse-checkout.c ##
3: cc52fb2b0b7 ! 3: 5d096e380a4 sparse-checkout: refuse to add to bad patterns
@@ Commit message
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.
+ Reviewed-by: Elijah Newren [off-list ref]
Signed-off-by: Derrick Stolee [off-list ref]
## builtin/sparse-checkout.c ##
-: ----------- > 4: 7bacb3760f3 amend! sparse-checkout: fix segfault on malformed patterns
--
gitgitgadget
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-10 15:18:21
From: Derrick Stolee <redacted>
Add a test to t1091-sparse-checkout-builtin.sh that would result in an
infinite loop and out-of-memory error before this change. The issue
relies on having non-cone-mode patterns while trying to modify the
patterns in cone-mode.
The fix is simple, allowing us to break from the loop when the input
path does not contain a slash, as the "dir" pattern we added does not.
This is only a fix to the critical out-of-memory error. A better
response to such a strange state will follow in a later change.
Reported-by: Calbabreaker <redacted>
Helped-by: Taylor Blau [off-list ref]
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-10 15:18:22
From: Derrick Stolee <redacted>
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.
One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
no invalid.
The fix here is to stop trying to remove from the hashset. Better to leave
bad data in the sparse-checkout matching logic (with a warning) than to
segfault. If we are in this state, then we are already traversing into
undefined behavior, so this change to keep the entry in the hashset is no
worse than removing it.
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett <redacted>
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
dir.c | 3 ---
t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
@@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern/* we already included this at the parent level */warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"),given->pattern);-hashmap_remove(&pl->parent_hashmap,&translated->ent,&data);-free(data);-free(translated);}return;
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-10 15:18:23
From: Derrick Stolee <redacted>
When in cone mode sparse-checkout, it is unclear how 'git
sparse-checkout add <dir1> ...' should behave if the existing
sparse-checkout file does not match the cone mode patterns. Change the
behavior to fail with an error message about the existing patterns.
Also, all cone mode patterns start with a '/' character, so add that
restriction. This is necessary for our example test 'cone mode: warn on
bad pattern', but also requires modifying the example sparse-checkout
file we use to test the warnings related to recognizing cone mode
patterns.
This error checking would cause a failure further down the test script
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 3 +++
dir.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 7 +++++--
3 files changed, 9 insertions(+), 3 deletions(-)
@@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' 'gitsparse-checkoutinit&&gitsparse-checkoutadddir&&gitconfigcore.sparseCheckoutConetrue&&-gitsparse-checkoutadddir+test_must_failgitsparse-checkoutadddir2>err&&+grep"existing sparse-checkout patterns do not use cone mode"err)'
@@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' '' test_expect_success'add to sparse-checkout''-catrepo/.git/info/sparse-checkout>expect&&+catrepo/.git/info/sparse-checkout>old&&+test_when_finishedcpoldrepo/.git/info/sparse-checkout&&cat>add<<-\EOF&&pattern1/folder1/pattern2EOF+catold>expect&&catadd>>expect&&git-Creposparse-checkoutadd--stdin<add&&git-Creposparse-checkoutlist>actual&&
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-15 13:46:12
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
--cone' that isn't necessary, especially with Elijah doing some work in that
space right now.
[1] https://github.com/git-for-windows/git/issues/3498 [2]
https://lore.kernel.org/git/pull.1043.git.1632160658.gitgitgadget@gmail.com
Thanks, -Stolee
Updates in v2 and v3
====================
* I intended to fix a typo in a patch, but accidentally sent the amend!
commit in v2
* v3 has the typo fix properly squashed in.
* Added Elijah's review.
Derrick Stolee (3):
sparse-checkout: fix segfault on malformed patterns
sparse-checkout: fix OOM error with mixed patterns
sparse-checkout: refuse to add to bad patterns
builtin/sparse-checkout.c | 5 ++++-
dir.c | 5 +----
t/t1091-sparse-checkout-builtin.sh | 31 +++++++++++++++++++++++++++++-
3 files changed, 35 insertions(+), 6 deletions(-)
base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1069%2Fderrickstolee%2Fsparse-checkout%2Finput-bug-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1069/derrickstolee/sparse-checkout/input-bug-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1069
Range-diff vs v2:
1: a0e3dd335c9 ! 1: 1744a26845f sparse-checkout: fix segfault on malformed patterns
@@ Commit message
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
- no invalid.
+ now invalid.
The fix here is to stop trying to remove from the hashset. Better to leave
bad data in the sparse-checkout matching logic (with a warning) than to
2: 86fbf130c03 = 2: a2fe867222e sparse-checkout: fix OOM error with mixed patterns
3: 5d096e380a4 = 3: a0e5a942ae0 sparse-checkout: refuse to add to bad patterns
4: 7bacb3760f3 < -: ----------- amend! sparse-checkout: fix segfault on malformed patterns
--
gitgitgadget
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-15 13:46:13
From: Derrick Stolee <redacted>
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.
One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
now invalid.
The fix here is to stop trying to remove from the hashset. Better to leave
bad data in the sparse-checkout matching logic (with a warning) than to
segfault. If we are in this state, then we are already traversing into
undefined behavior, so this change to keep the entry in the hashset is no
worse than removing it.
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett <redacted>
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
dir.c | 3 ---
t/t1091-sparse-checkout-builtin.sh | 15 +++++++++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
@@ -819,9 +819,6 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern/* we already included this at the parent level */warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"),given->pattern);-hashmap_remove(&pl->parent_hashmap,&translated->ent,&data);-free(data);-free(translated);}return;
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-15 13:46:14
From: Derrick Stolee <redacted>
Add a test to t1091-sparse-checkout-builtin.sh that would result in an
infinite loop and out-of-memory error before this change. The issue
relies on having non-cone-mode patterns while trying to modify the
patterns in cone-mode.
The fix is simple, allowing us to break from the loop when the input
path does not contain a slash, as the "dir" pattern we added does not.
This is only a fix to the critical out-of-memory error. A better
response to such a strange state will follow in a later change.
Reported-by: Calbabreaker <redacted>
Helped-by: Taylor Blau [off-list ref]
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-15 13:46:17
From: Derrick Stolee <redacted>
When in cone mode sparse-checkout, it is unclear how 'git
sparse-checkout add <dir1> ...' should behave if the existing
sparse-checkout file does not match the cone mode patterns. Change the
behavior to fail with an error message about the existing patterns.
Also, all cone mode patterns start with a '/' character, so add that
restriction. This is necessary for our example test 'cone mode: warn on
bad pattern', but also requires modifying the example sparse-checkout
file we use to test the warnings related to recognizing cone mode
patterns.
This error checking would cause a failure further down the test script
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 3 +++
dir.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 7 +++++--
3 files changed, 9 insertions(+), 3 deletions(-)
@@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' 'gitsparse-checkoutinit&&gitsparse-checkoutadddir&&gitconfigcore.sparseCheckoutConetrue&&-gitsparse-checkoutadddir+test_must_failgitsparse-checkoutadddir2>err&&+grep"existing sparse-checkout patterns do not use cone mode"err)'
@@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' '' test_expect_success'add to sparse-checkout''-catrepo/.git/info/sparse-checkout>expect&&+catrepo/.git/info/sparse-checkout>old&&+test_when_finishedcpoldrepo/.git/info/sparse-checkout&&cat>add<<-\EOF&&pattern1/folder1/pattern2EOF+catold>expect&&catadd>>expect&&git-Creposparse-checkoutadd--stdin<add&&git-Creposparse-checkoutlist>actual&&
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-16 16:13:47
This series fixes some issues with parsing sparse-checkout patterns when
core.sparseCheckoutCone is enabled but the sparse-checkout file itself
contains patterns that don't match the cone mode format.
The first patch fixes a segfault first reported in [1]. The other two
patches are from an earlier submission [2] that never got picked up and I
lost track of. There was another patch involving 'git sparse-checkout init
--cone' that isn't necessary, especially with Elijah doing some work in that
space right now.
[1] https://github.com/git-for-windows/git/issues/3498 [2]
https://lore.kernel.org/git/pull.1043.git.1632160658.gitgitgadget@gmail.com
Thanks, -Stolee
Update in v4
============
* For added precaution, this kind of unexpected duplicate pattern will
disable cone mode matching.
* Tests are updated to verify this new behavior.
Updates in v2 and v3
====================
* I intended to fix a typo in a patch, but accidentally sent the amend!
commit in v2
* v3 has the typo fix properly squashed in.
* Added Elijah's review.
Derrick Stolee (3):
sparse-checkout: fix segfault on malformed patterns
sparse-checkout: fix OOM error with mixed patterns
sparse-checkout: refuse to add to bad patterns
builtin/sparse-checkout.c | 5 +++-
dir.c | 6 ++---
t/t1091-sparse-checkout-builtin.sh | 37 +++++++++++++++++++++++++++++-
3 files changed, 42 insertions(+), 6 deletions(-)
base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1069%2Fderrickstolee%2Fsparse-checkout%2Finput-bug-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1069/derrickstolee/sparse-checkout/input-bug-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1069
Range-diff vs v3:
1: 1744a26845f ! 1: 5353c541d9f sparse-checkout: fix segfault on malformed patterns
@@ Commit message
list' command because it iterates over the contents of the hashset, which is
now invalid.
- The fix here is to stop trying to remove from the hashset. Better to leave
- bad data in the sparse-checkout matching logic (with a warning) than to
- segfault. If we are in this state, then we are already traversing into
- undefined behavior, so this change to keep the entry in the hashset is no
- worse than removing it.
+ The fix here is to stop trying to remove from the hashset. In addition,
+ we disable cone mode sparse-checkout because of the malformed data. This
+ results in the pattern-matching working with a possibly-slower
+ algorithm, but using the patterns as they are in the sparse-checkout
+ file.
+
+ This also changes the behavior of commands such as 'git sparse-checkout
+ list' because the output patterns will be the contents of the
+ sparse-checkout file instead of the list of directories. This is an
+ existing behavior for other types of bad patterns.
Add a test that triggers the segfault without the code change.
@@ dir.c: static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_
- hashmap_remove(&pl->parent_hashmap, &translated->ent, &data);
- free(data);
- free(translated);
++ goto clear_hashmaps;
}
return;
@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'cone mode clears ignore
+ !/foo/*/
+ /foo/\*/
+ EOF
-+ cat repo/.git/info/sparse-checkout &&
-+ git -C repo sparse-checkout list
++
++ # Listing the patterns will notice the duplicate pattern and
++ # emit a warning. It will list the patterns directly instead
++ # of using the cone-mode translation to a set of directories.
++ git -C repo sparse-checkout list >actual 2>err &&
++ test_cmp repo/.git/info/sparse-checkout actual &&
++ grep "warning: your sparse-checkout file may have issues: pattern .* is repeated" err &&
++ grep "warning: disabling cone pattern matching" err
+'
+
test_done
2: a2fe867222e = 2: 3fd625290a3 sparse-checkout: fix OOM error with mixed patterns
3: a0e5a942ae0 = 3: f5f7b8b8e04 sparse-checkout: refuse to add to bad patterns
--
gitgitgadget
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-16 16:13:48
From: Derrick Stolee <redacted>
Then core.sparseCheckoutCone is enabled, the sparse-checkout patterns are
used to populate two hashsets that accelerate pattern matching. If the user
modifies the sparse-checkout file outside of the 'sparse-checkout' builtin,
then strange patterns can happen, triggering some error checks.
One of these error checks is possible to hit when some special characters
exist in a line. A warning message is correctly written to stderr, but then
there is additional logic that attempts to remove the line from the hashset
and free the data. This leads to a segfault in the 'git sparse-checkout
list' command because it iterates over the contents of the hashset, which is
now invalid.
The fix here is to stop trying to remove from the hashset. In addition,
we disable cone mode sparse-checkout because of the malformed data. This
results in the pattern-matching working with a possibly-slower
algorithm, but using the patterns as they are in the sparse-checkout
file.
This also changes the behavior of commands such as 'git sparse-checkout
list' because the output patterns will be the contents of the
sparse-checkout file instead of the list of directories. This is an
existing behavior for other types of bad patterns.
Add a test that triggers the segfault without the code change.
Reported-by: John Burnett <redacted>
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
dir.c | 4 +---
t/t1091-sparse-checkout-builtin.sh | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+), 3 deletions(-)
@@ -819,9 +819,7 @@ static void add_pattern_to_hashsets(struct pattern_list *pl, struct path_pattern/* we already included this at the parent level */warning(_("your sparse-checkout file may have issues: pattern '%s' is repeated"),given->pattern);-hashmap_remove(&pl->parent_hashmap,&translated->ent,&data);-free(data);-free(translated);+gotoclear_hashmaps;}return;
@@ -708,4 +708,25 @@ test_expect_success 'cone mode clears ignored subdirectories' 'test_cmpexpectout'+test_expect_success'malformed cone-mode patterns''+git-Creposparse-checkoutinit--cone&&+mkdir-prepo/foo/bar&&+touchrepo/foo/bar/xrepo/foo/y&&+cat>repo/.git/info/sparse-checkout<<-\EOF&&+/*+!/*/+/foo/+!/foo/*/+/foo/\*/+EOF++# Listing the patterns will notice the duplicate pattern and+# emit a warning. It will list the patterns directly instead+# of using the cone-mode translation to a set of directories.+git-Creposparse-checkoutlist>actual2>err&&+test_cmprepo/.git/info/sparse-checkoutactual&&+grep"warning: your sparse-checkout file may have issues: pattern .* is repeated"err&&+grep"warning: disabling cone pattern matching"err+'+ test_done
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-16 16:13:52
From: Derrick Stolee <redacted>
Add a test to t1091-sparse-checkout-builtin.sh that would result in an
infinite loop and out-of-memory error before this change. The issue
relies on having non-cone-mode patterns while trying to modify the
patterns in cone-mode.
The fix is simple, allowing us to break from the loop when the input
path does not contain a slash, as the "dir" pattern we added does not.
This is only a fix to the critical out-of-memory error. A better
response to such a strange state will follow in a later change.
Reported-by: Calbabreaker <redacted>
Helped-by: Taylor Blau [off-list ref]
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 11 +++++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-16 16:13:52
From: Derrick Stolee <redacted>
When in cone mode sparse-checkout, it is unclear how 'git
sparse-checkout add <dir1> ...' should behave if the existing
sparse-checkout file does not match the cone mode patterns. Change the
behavior to fail with an error message about the existing patterns.
Also, all cone mode patterns start with a '/' character, so add that
restriction. This is necessary for our example test 'cone mode: warn on
bad pattern', but also requires modifying the example sparse-checkout
file we use to test the warnings related to recognizing cone mode
patterns.
This error checking would cause a failure further down the test script
because of a test that adds non-cone mode patterns without cleaning them
up. Perform that cleanup as part of the test now.
Reviewed-by: Elijah Newren <redacted>
Signed-off-by: Derrick Stolee <redacted>
---
builtin/sparse-checkout.c | 3 +++
dir.c | 2 +-
t/t1091-sparse-checkout-builtin.sh | 7 +++++--
3 files changed, 9 insertions(+), 3 deletions(-)
@@ -110,7 +110,8 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' 'gitsparse-checkoutinit&&gitsparse-checkoutadddir&&gitconfigcore.sparseCheckoutConetrue&&-gitsparse-checkoutadddir+test_must_failgitsparse-checkoutadddir2>err&&+grep"existing sparse-checkout patterns do not use cone mode"err)'
@@ -176,12 +177,14 @@ test_expect_success 'set sparse-checkout using --stdin' '' test_expect_success'add to sparse-checkout''-catrepo/.git/info/sparse-checkout>expect&&+catrepo/.git/info/sparse-checkout>old&&+test_when_finishedcpoldrepo/.git/info/sparse-checkout&&cat>add<<-\EOF&&pattern1/folder1/pattern2EOF+catold>expect&&catadd>>expect&&git-Creposparse-checkoutadd--stdin<add&&git-Creposparse-checkoutlist>actual&&