Re: Odd .gitignore behaviour

Subsystems: the rest

2 messages, 1 author, 2016-06-15 · open the first message on its own page

Re: Odd .gitignore behaviour

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:43:51

Bruce Stephens [off-list ref] writes:
So I think the output from git-ls-files is as expected (as I interpret
the manpage and your explanation).  So is git-add just using some
different code?
No, you found one of the longstanding bugs in dir.c:read_directory().

The funny thing is that I just sent out a message pointing out
bogus handling of per-directory exclude files in ls-files last
night.  Somehow people have a tendency to encounter the bugs in
the same vicinity independently.

The initial loop in read_directory() to push per-directory
exclusion elements into the stack for directories above the
given base forgets that push() does not make a copy of the path
given as its parameter but stores the pointer to it instead, so
multiple calls to push() need to use separate path buffers.

Here is a tentative patch.  I do not think the patch is broken
but I call it tentative because:

 - It is ugly -- I never get this "walking path delimited by
   slashes" loop right;

 - It leaks the path buffer given to push(), but it is inherent
   in the design of "push/pop exclude per-directory" API.  They
   were designed to be called from the recursive directory
   walking, and the path buffers are placed on the function call
   stack to be reclaimed automatically upon function return;

---

 dir.c |   55 ++++++++++++++++++++++++++++++++++---------------------
 1 files changed, 34 insertions(+), 21 deletions(-)
diff --git a/dir.c b/dir.c
index fa9f902..d32f437 100644
--- a/dir.c
+++ b/dir.c
@@ -651,38 +651,51 @@ static void free_simplify(struct path_simplify *simplify)
 		free(simplify);
 }
 
+static int push_excludes(struct dir_struct *dir, const char *base, int len)
+{
+	/*
+	 * base is like "a/b/c/" -- cause .gitignore, b/.gitignore and
+	 * b/c/.gitignore to be read in this order, as if we recursed
+	 * into it.
+	 */
+	int stk = -1;
+	int partlen = 0;
+
+	if (!(dir->exclude_per_dir && len))
+		return stk;
+
+	while (1) {
+		char *part = xmalloc(partlen + 1);
+		memcpy(part, base, partlen);
+		part[partlen] = '\0';
+		stk = push_exclude_per_directory(dir, part, partlen);
+
+		if (len <= partlen++)
+			break;
+
+		while (partlen < len && base[partlen] != '/')
+			partlen++;
+		partlen++; /* point at one past the found '/' */
+	}
+	return stk;
+}
+
 int read_directory(struct dir_struct *dir, const char *path, const char *base, int baselen, const char **pathspec)
 {
 	struct path_simplify *simplify = create_simplify(pathspec);
+	int stk;
 
 	/*
 	 * Make sure to do the per-directory exclude for all the
 	 * directories leading up to our base.
 	 */
-	if (baselen) {
-		if (dir->exclude_per_dir) {
-			char *p, *pp = xmalloc(baselen+1);
-			memcpy(pp, base, baselen+1);
-			p = pp;
-			while (1) {
-				char save = *p;
-				*p = 0;
-				push_exclude_per_directory(dir, pp, p-pp);
-				*p++ = save;
-				if (!save)
-					break;
-				p = strchr(p, '/');
-				if (p)
-					p++;
-				else
-					p = pp + baselen;
-			}
-			free(pp);
-		}
-	}
+	stk = push_excludes(dir, base, baselen);
 
 	read_directory_recursive(dir, path, base, baselen, 0, simplify);
 	free_simplify(simplify);
+	if (0 <= stk)
+		pop_exclude_per_directory(dir, stk);
+
 	qsort(dir->entries, dir->nr, sizeof(struct dir_entry *), cmp_name);
 	qsort(dir->ignored, dir->ignored_nr, sizeof(struct dir_entry *), cmp_name);
 	return dir->nr;

Fix per-directory exclude handing for "git add"

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:43:52

In "dir_struct", each exclusion element in the exclusion stack records a
base string (pointer to the beginning with length) so that we can tell
where it came from, but this pointer is just pointing at the parameter
that is given by the caller to the push_exclude_per_directory()
function.

While read_directory_recursive() runs, calls to excluded() makes use
the data in the exclusion elements, including this base string.  The
caller of read_directory_recursive() is not supposed to free the
buffer it gave to push_exclude_per_directory() earlier, until it
returns.

The test case Bruce Stephens gave in the mailing list discussion
was simplified and added to the t3700 test.

Signed-off-by: Junio C Hamano <redacted>
---

 Junio C Hamano [off-list ref] writes:

 > Here is a tentative patch.  I do not think the patch is broken
 > but I call it tentative because:
 >
 >  - It is ugly -- I never get this "walking path delimited by
 >    slashes" loop right;
 >
 >  - It leaks the path buffer given to push(), but it is inherent
 >    in the design of "push/pop exclude per-directory" API.

 It turns out that a minimally invasive fix was a lot simpler
 than I thought.

 This still does not fix the other codepaths in ls-files that does not
 use read_directory() but walks the cache.

 dir.c          |    6 ++++--
 t/t3700-add.sh |   24 ++++++++++++++++++++++++
 2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/dir.c b/dir.c
index fa9f902..225fdfb 100644
--- a/dir.c
+++ b/dir.c
@@ -654,6 +654,7 @@ static void free_simplify(struct path_simplify *simplify)
 int read_directory(struct dir_struct *dir, const char *path, const char *base, int baselen, const char **pathspec)
 {
 	struct path_simplify *simplify = create_simplify(pathspec);
+	char *pp = NULL;
 
 	/*
 	 * Make sure to do the per-directory exclude for all the
@@ -661,7 +662,8 @@ int read_directory(struct dir_struct *dir, const char *path, const char *base, i
 	 */
 	if (baselen) {
 		if (dir->exclude_per_dir) {
-			char *p, *pp = xmalloc(baselen+1);
+			char *p;
+			pp = xmalloc(baselen+1);
 			memcpy(pp, base, baselen+1);
 			p = pp;
 			while (1) {
@@ -677,12 +679,12 @@ int read_directory(struct dir_struct *dir, const char *path, const char *base, i
 				else
 					p = pp + baselen;
 			}
-			free(pp);
 		}
 	}
 
 	read_directory_recursive(dir, path, base, baselen, 0, simplify);
 	free_simplify(simplify);
+	free(pp);
 	qsort(dir->entries, dir->nr, sizeof(struct dir_entry *), cmp_name);
 	qsort(dir->ignored, dir->ignored_nr, sizeof(struct dir_entry *), cmp_name);
 	return dir->nr;
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index a328bf5..287e058 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -104,9 +104,33 @@ test_expect_success 'add ignored ones with -f' '
 	git ls-files --error-unmatch d.ig/d.if d.ig/d.ig
 '
 
+test_expect_success 'add ignored ones with -f' '
+	rm -f .git/index &&
+	git add -f d.?? &&
+	git ls-files --error-unmatch d.ig/d.if d.ig/d.ig
+'
+
+test_expect_success '.gitignore with subdirectory' '
+
+	rm -f .git/index &&
+	mkdir -p sub/dir &&
+	echo "!dir/a.*" >sub/.gitignore &&
+	>sub/a.ig &&
+	>sub/dir/a.ig &&
+	git add sub/dir &&
+	git ls-files --error-unmatch sub/dir/a.ig &&
+	rm -f .git/index &&
+	(
+		cd sub/dir &&
+		git add .
+	) &&
+	git ls-files --error-unmatch sub/dir/a.ig
+'
+
 mkdir 1 1/2 1/3
 touch 1/2/a 1/3/b 1/2/c
 test_expect_success 'check correct prefix detection' '
+	rm -f .git/index &&
 	git add 1/2/a 1/3/b 1/2/c
 '
 
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help