cg-status and empty directories

Hello,

Short story: Recently I noticed a change in the way, cogito handles
empty directories.  Before, empty directories have been silently
ignored. Now cg-status always lists the status of empty directories as
unknown, but it still refuses to add them. If there is a good reason
for this behaviour, can someone enlighten me?

Long story: I'm using cogito to track and distribute changes on the
/etc directories of a few (almost) identical machines.  Whenever I
install a package which modifies somthing in /etc, I commit those
changes.  But with cg-status reporting all the empty directories as
"unknown", my brain needs a long time to parse the list and find the
really unknown files which shall be put under version control.

Many packages put empty directories under /etc, and although only a
few of those directories are actually needed, the automatic removal of
those packages will fail if I remove the empty directories manually.  
Equally, the removal will fail, if I put a .placeholder file into
those direrectories and cg-add it.  Is there a simple way out?

Regards,
Jim

Jim MacBaine wrote:

Hello,

Short story: Recently I noticed a change in the way, cogito handles
empty directories.  Before, empty directories have been silently
ignored. Now cg-status always lists the status of empty directories as
unknown, but it still refuses to add them. If there is a good reason
for this behaviour, can someone enlighten me?

Long story: I'm using cogito to track and distribute changes on the
/etc directories of a few (almost) identical machines.  Whenever I
install a package which modifies somthing in /etc, I commit those
changes.  But with cg-status reporting all the empty directories as
"unknown", my brain needs a long time to parse the list and find the
really unknown files which shall be put under version control.

Many packages put empty directories under /etc, and although only a
few of those directories are actually needed, the automatic removal of
those packages will fail if I remove the empty directories manually.  
Equally, the removal will fail, if I put a .placeholder file into
those direrectories and cg-add it.  Is there a simple way out?

I'm afraid not.

You should also note that git doesn't track permissions exactly. It just 
notices an execution bit and uses it to determine if it should write the 
working tree using (0666 ^ umask) or (0777 ^ umask). This makes it 
fairly unsuitable for /etc tracking unless you add some sort of 
permission restoring thing to it.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

On 2/27/06, Andreas Ericsson [off-list ref] wrote:

I'm afraid not.

You should also note that git doesn't track permissions exactly. It just
notices an execution bit and uses it to determine if it should write the
working tree using (0666 ^ umask) or (0777 ^ umask). This makes it
fairly unsuitable for /etc tracking unless you add some sort of
permission restoring thing to it.

I'm aware of those limitations. It is not a backup mechanism at all
and permissions are handled by a small perl script.

However, I'm just curious to know, why cg-status suddenly started to
care about empty directories.

Regards,
Jim

  Hi,

Dear diary, on Mon, Feb 27, 2006 at 03:43:32PM CET, I got a letter
where Jim MacBaine [off-list ref] said that...

Many packages put empty directories under /etc, and although only a
few of those directories are actually needed, the automatic removal of
those packages will fail if I remove the empty directories manually.  
Equally, the removal will fail, if I put a .placeholder file into
those direrectories and cg-add it.  Is there a simple way out?

  this is caused by git-ls-files behaviour - we now call it with
the --directory argument which is nice since it will show a non-empty
unknown directory as a single entry and won't list all its contents.
What is not so nice is the side-effect you are describing, and I tend
to agree that if the directory is empty, it should not be listed.

---

Without the --directory flag, git-ls-files wouldn't ever list directories,
producing no output for empty directories, which is good since they cannot
be added and they bear no content, even untracked one (if Git ever starts
tracking directories on their own, this should obviously change since the
content notion will change).

With the --directory flag however, git-ls-files would list even empty
directories. This patch fixes this.

Signed-off-by: Petr Baudis <redacted>
---

 ls-files.c |   19 ++++++++++++++-----
 1 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/ls-files.c b/ls-files.c
index e42119c..4502b51 100644
--- a/ls-files.c
+++ b/ls-files.c

@@ -258,11 +258,12 @@ static int dir_exists(const char *dirnam
  * Also, we ignore the name ".git" (even if it is not a directory).
  * That likely will not change.
  */
-static void read_directory(const char *path, const char *base, int baselen)
+static int read_directory(const char *path, const char *base, int baselen)
 {
-	DIR *dir = opendir(path);
+	DIR *fdir = opendir(path);
+	int contents = 0;
 
-	if (dir) {
+	if (fdir) {
 		int exclude_stk;
 		struct dirent *de;
 		char fullname[MAXPATHLEN + 1];

@@ -270,7 +271,7 @@ static void read_directory(const char *p
 
 		exclude_stk = push_exclude_per_directory(base, baselen);
 
-		while ((de = readdir(dir)) != NULL) {
+		while ((de = readdir(fdir)) != NULL) {
 			int len;
 
 			if ((de->d_name[0] == '.') &&

@@ -288,6 +289,7 @@ static void read_directory(const char *p
 
 			switch (DTYPE(de)) {
 			struct stat st;
+			int subdir, rewind_base;
 			default:
 				continue;
 			case DT_UNKNOWN:

@@ -301,22 +303,31 @@ static void read_directory(const char *p
 			case DT_DIR:
 				memcpy(fullname + baselen + len, "/", 2);
 				len++;
-				if (show_other_directories &&
-				    !dir_exists(fullname, baselen + len))
+				rewind_base = nr_dir;
+				subdir = read_directory(fullname, fullname,
+				                        baselen + len);
+				if (show_other_directories && subdir &&
+				    !dir_exists(fullname, baselen + len)) {
+					// Rewind the read subdirectory
+					while (nr_dir > rewind_base)
+						free(dir[--nr_dir]);
 					break;
-				read_directory(fullname, fullname,
-					       baselen + len);
+				}
+				contents += subdir;
 				continue;
 			case DT_REG:
 			case DT_LNK:
 				break;
 			}
 			add_name(fullname, baselen + len);
+			contents++;
 		}
-		closedir(dir);
+		closedir(fdir);
 
 		pop_exclude_per_directory(exclude_stk);
 	}
+
+	return contents;
 }
 
 static int cmp_name(const void *p1, const void *p2)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.

 Hi,

Dear diary, on Sun, Mar 26, 2006 at 04:25:05PM CEST, I got a letter
where Petr Baudis [off-list ref] said that...

Dear diary, on Mon, Feb 27, 2006 at 03:43:32PM CET, I got a letter
where Jim MacBaine [off-list ref] said that...

quoted

Many packages put empty directories under /etc, and although only a
few of those directories are actually needed, the automatic removal of
those packages will fail if I remove the empty directories manually.  
Equally, the removal will fail, if I put a .placeholder file into
those direrectories and cg-add it.  Is there a simple way out?

  this is caused by git-ls-files behaviour - we now call it with
the --directory argument which is nice since it will show a non-empty
unknown directory as a single entry and won't list all its contents.
What is not so nice is the side-effect you are describing, and I tend
to agree that if the directory is empty, it should not be listed.

  it turned out that cg-clean depends on the original behaviour (and it
makes sense there, we want to purge even empty directories). Therefore
this patch will preserve the old behaviour but add an option
--no-empty-directory. When that gets propagated to Git releases, I will
use it in cg-status.

---

Without the --directory flag, git-ls-files wouldn't ever list directories,
producing no output for empty directories, which is good since they cannot
be added and they bear no content, even untracked one (if Git ever starts
tracking directories on their own, this should obviously change since the
content notion will change).

With the --directory flag however, git-ls-files would list even empty
directories. This may be good in some situations but sometimes you want to
prevent that. This patch adds a --no-empty-directory option which makes
git-ls-files omit empty directories.

Signed-off-by: Petr Baudis <redacted>
---

 Documentation/git-ls-files.txt |    3 +++
 ls-files.c                     |   33 +++++++++++++++++++++++++--------
 2 files changed, 28 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index e813f84..980c5c9 100644
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt

@@ -52,6 +52,9 @@ OPTIONS
 	If a whole directory is classified as "other", show just its
 	name (with a trailing slash) and not its whole contents.
 
+--no-empty-directory::
+	Do not list empty directories. Has no effect without --directory.
+
 -u|--unmerged::
 	Show unmerged files in the output (forces --stage)
 

diff --git a/ls-files.c b/ls-files.c
index e42119c..83b0a3b 100644
--- a/ls-files.c
+++ b/ls-files.c

@@ -20,6 +20,7 @@ static int show_unmerged = 0;
 static int show_modified = 0;
 static int show_killed = 0;
 static int show_other_directories = 0;
+static int hide_empty_directories = 0;
 static int show_valid_bit = 0;
 static int line_terminator = '\n';
 

@@ -258,11 +259,12 @@ static int dir_exists(const char *dirnam
  * Also, we ignore the name ".git" (even if it is not a directory).
  * That likely will not change.
  */
-static void read_directory(const char *path, const char *base, int baselen)
+static int read_directory(const char *path, const char *base, int baselen)
 {
-	DIR *dir = opendir(path);
+	DIR *fdir = opendir(path);
+	int contents = 0;
 
-	if (dir) {
+	if (fdir) {
 		int exclude_stk;
 		struct dirent *de;
 		char fullname[MAXPATHLEN + 1];

@@ -270,7 +272,7 @@ static void read_directory(const char *p
 
 		exclude_stk = push_exclude_per_directory(base, baselen);
 
-		while ((de = readdir(dir)) != NULL) {
+		while ((de = readdir(fdir)) != NULL) {
 			int len;
 
 			if ((de->d_name[0] == '.') &&

@@ -288,6 +290,7 @@ static void read_directory(const char *p
 
 			switch (DTYPE(de)) {
 			struct stat st;
+			int subdir, rewind_base;
 			default:
 				continue;
 			case DT_UNKNOWN:

@@ -301,22 +304,32 @@ static void read_directory(const char *p
 			case DT_DIR:
 				memcpy(fullname + baselen + len, "/", 2);
 				len++;
+				rewind_base = nr_dir;
+				subdir = read_directory(fullname, fullname,
+				                        baselen + len);
 				if (show_other_directories &&
-				    !dir_exists(fullname, baselen + len))
+				    (subdir || !hide_empty_directories) &&
+				    !dir_exists(fullname, baselen + len)) {
+					// Rewind the read subdirectory
+					while (nr_dir > rewind_base)
+						free(dir[--nr_dir]);
 					break;
-				read_directory(fullname, fullname,
-					       baselen + len);
+				}
+				contents += subdir;
 				continue;
 			case DT_REG:
 			case DT_LNK:
 				break;
 			}
 			add_name(fullname, baselen + len);
+			contents++;
 		}
-		closedir(dir);
+		closedir(fdir);
 
 		pop_exclude_per_directory(exclude_stk);
 	}
+
+	return contents;
 }
 
 static int cmp_name(const void *p1, const void *p2)

@@ -696,6 +709,10 @@ int main(int argc, const char **argv)
 			show_other_directories = 1;
 			continue;
 		}
+		if (!strcmp(arg, "--no-empty-directory")) {
+			hide_empty_directories = 1;
+			continue;
+		}
 		if (!strcmp(arg, "-u") || !strcmp(arg, "--unmerged")) {
 			/* There's no point in showing unmerged unless
 			 * you also show the stage information.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.

  Hi,

Dear diary, on Mon, Feb 27, 2006 at 03:43:32PM CET, I got a letter
where Jim MacBaine [off-list ref] said that...

Many packages put empty directories under /etc, and although only a
few of those directories are actually needed, the automatic removal of
those packages will fail if I remove the empty directories manually.  
Equally, the removal will fail, if I put a .placeholder file into
those direrectories and cg-add it.  Is there a simple way out?

  BTW, with Cogito-0.17.1 the simple way out should be cg-status -S
which restores the original behaviour.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.