Garbage in .git directories???

Hi,

I just did:

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

[dtor@anvil work]$ ls -la .git/
total 40
drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
-rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?���?E ???#V?C??ſl??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The similar garbage(?) shows when I clone git's repository.

Is this expected?

-- 
Dmitry

On Sat, 17 Sep 2005, Dmitry Torokhov wrote:

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?���?E ???#V?C??ſl??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
The similar garbage(?) shows when I clone git's repository.

Is this expected?

Absolutely not. And I don't see it when I try. What filesystem are you 
running this on? Looks like something did a "mkdir()" with an 
uninitialized pointer, but I don't see how that would happen.

		Linus

On Sunday 18 September 2005 12:10, Linus Torvalds wrote:

On Sat, 17 Sep 2005, Dmitry Torokhov wrote:

quoted

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?���?E ???#V?C??ſl??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
The similar garbage(?) shows when I clone git's repository.

Is this expected?

Absolutely not. And I don't see it when I try. What filesystem are you 
running this on? Looks like something did a "mkdir()" with an 
uninitialized pointer, but I don't see how that would happen.

		Linus

It is ext3 on 2.6.12-rc6.

-- 
Dmitry

On Sunday 18 September 2005 12:54, Dmitry Torokhov wrote:

On Sunday 18 September 2005 12:10, Linus Torvalds wrote:

quoted

On Sat, 17 Sep 2005, Dmitry Torokhov wrote:

quoted

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?���?E ???#V?C??ſl??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
The similar garbage(?) shows when I clone git's repository.

Is this expected?

Absolutely not. And I don't see it when I try. What filesystem are you 
running this on? Looks like something did a "mkdir()" with an 
uninitialized pointer, but I don't see how that would happen.

		Linus

It is ext3 on 2.6.12-rc6.

Couple more points - just using rsync to get the data (without using git)
does not not produce that wierd directory, only got clone does.

Btw, after doing a pull from git repository git cone now produces the
following:

sent 403 bytes  received 2164 bytes  570.44 bytes/sec
total size is 533  speedup is 0.21
rsync: link_stat "/scm/git/git.git/objects/info/alternates" (in pub) failed: No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(1173)


-- 
Dmitry

On Sun, 18 Sep 2005, Dmitry Torokhov wrote:

Couple more points - just using rsync to get the data (without using git)
does not not produce that wierd directory, only got clone does.

Does a plain "git-init-db" in a newly created empty directory do the same?  
There was a bug (a long time ago)  that created garbage instead of the
template files, I think.

Also, is there something inside of the garbage subdirectory?

Btw, after doing a pull from git repository git cone now produces the
following:

Ignore that one, looks like a silly error. Unrelated.

		Linus

On 9/19/05, Linus Torvalds [off-list ref] wrote:

On Sun, 18 Sep 2005, Dmitry Torokhov wrote:

quoted

Couple more points - just using rsync to get the data (without using git)
does not not produce that wierd directory, only got clone does.

Does a plain "git-init-db" in a newly created empty directory do the same?
There was a bug (a long time ago)  that created garbage instead of the
template files, I think.

Will test once get home...

Also, is there something inside of the garbage subdirectory?

No, they are always empty.

-- 
Dmitry

Dmitry Torokhov wrote:

git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git work

and it seems there is some garbage in .git directory:

[dtor@anvil work]$ ls -la .git/
total 40
drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
-rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?�?E ???#V?C??ſl??E#V?C??;H
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The similar garbage(?) shows when I clone git's repository.

FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
finding a directory in .git which consists of a single DEL character (\177).

	-hpa

Dear diary, on Mon, Sep 19, 2005 at 09:10:19PM CEST, I got a letter
where "H. Peter Anvin" [off-list ref] told me that...

Dmitry Torokhov wrote:

quoted

git clone 
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git 
work

and it seems there is some garbage in .git directory:

[dtor@anvil work]$ ls -la .git/
total 40
drwxrwxr-x    9 dtor dtor 4096 Sep 17 21:17 .
drwxrwxr-x    3 dtor dtor 4096 Sep 17 21:17 ..
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 branches
-rw-rw-r--    1 dtor dtor   58 Sep 17 21:17 description
lrwxrwxrwx    1 dtor dtor   17 Sep 17 21:17 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 info
drwxr-xr-x  260 dtor dtor 4096 Sep 17 17:41 objects
drwxrwxr-x    4 dtor dtor 4096 May  1 19:15 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:28 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 17 21:17 V?Cl?????E 
???#V?C????l??E#V?C??;H
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The similar garbage(?) shows when I clone git's repository.

FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
finding a directory in .git which consists of a single DEL character (\177).

Does cg-init produce it too? It really seems that this has to be a bug
in git-init-db. If that is the case, could you try temporarily renaming
the templates directory?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
If you want the holes in your knowledge showing up try teaching
someone.  -- Alan Cox

Petr Baudis wrote:

quoted

FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
finding a directory in .git which consists of a single DEL character (\177).

Does cg-init produce it too? It really seems that this has to be a bug
in git-init-db. If that is the case, could you try temporarily renaming
the templates directory?

cg-init does produce it, too.  I'm not sure what you mean with 
temporarily renaming the templates directory?

	-hpa

Dear diary, on Tue, Sep 20, 2005 at 01:16:14AM CEST, I got a letter
where "H. Peter Anvin" [off-list ref] told me that...

Petr Baudis wrote:

quoted

quoted

FWIW, using cg-clone (using git-core 0.99.6 and cogito 0.14.1) I keep 
finding a directory in .git which consists of a single DEL character 
(\177).

Does cg-init produce it too? It really seems that this has to be a bug
in git-init-db. If that is the case, could you try temporarily renaming
the templates directory?

cg-init does produce it, too.  I'm not sure what you mean with 
temporarily renaming the templates directory?

Preventing git-init-db to copy its stuff into your .git/ - I think the
bug is likely somewhere there. Actually, this can be prevented by
passing an argument to git-init-db:

	git-init-db --template=somethingnonexisting

Another idea - try this patch:

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c

@@ -154,6 +154,7 @@ static void copy_templates(const char *g
 	}
 
 	memcpy(path, git_dir, len);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Petr Baudis wrote:

This seems to have worked.

	-hpa

Dear diary, on Tue, Sep 20, 2005 at 01:58:52AM CEST, I got a letter
where "H. Peter Anvin" [off-list ref] told me that...

Petr Baudis wrote:

quoted

Dear diary, on Tue, Sep 20, 2005 at 01:16:14AM CEST, I got a letter

Another idea - try this patch:

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c

@@ -154,6 +154,7 @@ static void copy_templates(const char *g

	}

	memcpy(path, git_dir, len);
+	path[len] = 0;
	copy_templates_1(path, len,
			 template_path, template_len,
			 dir);

This seems to have worked.

Great. Resending a marginally better version with a cool log message and
everything.

--

The base target directory for the templates copying was initialized
to git_dir, but git_dir[len] is not zero but / at the time we do the
initialization. This is not what we want for our target directory string
since we pass it to mkdir(), so make it zero-terminated manually.

Signed-off-by: Petr Baudis <redacted>

---
commit dbe1fe2f0d6de58bc0c9f98e6cf9573874612890
tree 30398d409fa27e95e3e592c803f6cd5a93652234
parent 79d2a11ec497fe940ca9a1fd8e0b9f295d39c290
author Petr Baudis [off-list ref] Tue, 20 Sep 2005 01:55:53 +0200
committer Petr Baudis <xpasky@machine.(none)> Tue, 20 Sep 2005 01:55:53 +0200

 init-db.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c

@@ -153,7 +153,8 @@ static void copy_templates(const char *g
 		return;
 	}
 
-	memcpy(path, git_dir, len);
+	memcpy(path, git_dir, len-1);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Petr Baudis wrote:

Wrong!  You're not initializing path[len-1]!

	-hpa

Dear diary, on Tue, Sep 20, 2005 at 02:09:41AM CEST, I got a letter
where "H. Peter Anvin" [off-list ref] told me that...

Petr Baudis wrote:

quoted

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c

@@ -153,7 +153,8 @@ static void copy_templates(const char *g

		return;
	}

-	memcpy(path, git_dir, len);
+	memcpy(path, git_dir, len-1);
+	path[len] = 0;
	copy_templates_1(path, len,

Wrong!  You're not initializing path[len-1]!

Oops, sorry. That's what you get when you want to make things marginally
better. ;-) We indeed want to pass copy_templates_1() the trailing slash
as well.

Let's just settle with the original patch then.

--

The base target directory for the templates copying was initialized
to git_dir, but git_dir[len] is not zero but / at the time we do the
initialization. This is not what we want for our target directory string
since we pass it to mkdir(), so make it zero-terminated manually.

Signed-off-by: Petr Baudis <redacted>

---

diff --git a/init-db.c b/init-db.c
--- a/init-db.c
+++ b/init-db.c

@@ -154,6 +154,7 @@ static void copy_templates(const char *g
 	}
 
 	memcpy(path, git_dir, len);
+	path[len] = 0;
 	copy_templates_1(path, len,
 			 template_path, template_len,
 			 dir);

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

On Monday 19 September 2005 09:51, Linus Torvalds wrote:

On Sun, 18 Sep 2005, Dmitry Torokhov wrote:

quoted

Couple more points - just using rsync to get the data (without using git)
does not not produce that wierd directory, only got clone does.

Does a plain "git-init-db" in a newly created empty directory do the same?  
There was a bug (a long time ago)  that created garbage instead of the
template files, I think.

Yes, plain git-init-db produces the garbage.

[dtor@anvil tst]$ git-init-db
defaulting to local storage area
[dtor@anvil tst]$ ls -l .git/
total 32
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 branches
-rw-rw-r--    1 dtor dtor   58 Sep 19 19:08 description
lrwxrwxrwx    1 dtor dtor   17 Sep 19 19:08 HEAD -> refs/heads/master
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 hooks
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 info
drwxrwxr-x  260 dtor dtor 4096 Sep 19 19:08 objects
drwxrwxr-x    4 dtor dtor 4096 Sep 19 19:08 refs
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 remotes
drwxrwxr-x    2 dtor dtor 4096 Sep 19 19:08 V?Cl???E ???#V?C??տl??E#V?C??;H
[dtor@anvil tst]$ git --version
git version 0.99.7

(Actually this is last night's pull).


-- 
Dmitry

Petr Baudis wrote:

quoted

quoted

-	memcpy(path, git_dir, len);
+	memcpy(path, git_dir, len-1);
+	path[len] = 0;
copy_templates_1(path, len,

Wrong!  You're not initializing path[len-1]!

Oops, sorry. That's what you get when you want to make things marginally
better. ;-) We indeed want to pass copy_templates_1() the trailing slash
as well.

Let's just settle with the original patch then.

But if len is the index of the '/', then you're not.

Think about it: the memcpy(path, git_dir, len) copies bytes 0..len-1. 
Thus you need to use path[len] = 0 to terminate.

If you want to copy len-1 characters, then you need to use path[len-1] = 0.

	-hpa

On Mon, 19 Sep 2005, Dmitry Torokhov wrote:

Yes, plain git-init-db produces the garbage.

Hmm. It doesn't do it for me, but I suspect it will be obvious where it
happens if you send us the output from "strace git-init-db"

Thanks,

		Linus

Dear diary, on Tue, Sep 20, 2005 at 02:42:47AM CEST, I got a letter
where "H. Peter Anvin" [off-list ref] told me that...

Petr Baudis wrote:

quoted

quoted

quoted

-	memcpy(path, git_dir, len);
+	memcpy(path, git_dir, len-1);
+	path[len] = 0;
copy_templates_1(path, len,

Wrong!  You're not initializing path[len-1]!

Oops, sorry. That's what you get when you want to make things marginally
better. ;-) We indeed want to pass copy_templates_1() the trailing slash
as well.

Let's just settle with the original patch then.

But if len is the index of the '/', then you're not.

But fortunately len is not the index of the '/', len-1 is. len is the length
of the leading path fragment including the trailing slash.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Petr Baudis wrote:

quoted

But if len is the index of the '/', then you're not.

But fortunately len is not the index of the '/', len-1 is. len is the length
of the leading path fragment including the trailing slash.

Then you'd need len-1 both inside the memcpy() and the assignment, of 
course.

	-hpa

On Monday 19 September 2005 19:45, Linus Torvalds wrote:

On Mon, 19 Sep 2005, Dmitry Torokhov wrote:

quoted

Yes, plain git-init-db produces the garbage.

Hmm. It doesn't do it for me, but I suspect it will be obvious where it
happens if you send us the output from "strace git-init-db"

Peter's fix for copy_templates() fixes the problem for me.

-- 
Dmitry