Commit ID in exported Tar Ball

Hello,
Michae, a friend of mine, is in the phase of migrating from git to cvs.
He releases tar balls of his software project using gitweb. He would
love to have a way to have the commit-id of HEAD of the export contained
in one of the files he exported that way. Is there infrastructure in git
that makes that already possible or does he need to some kind of
gerneration tool by himself? Maybe it would be helpful if the
git-tar-tree would generate a file .commitid or something like that in
the generated tar tree.

        Thomas

On Thursday 17 May 2007, Thomas Glanzmann wrote:

Hello,
Michae, a friend of mine, is in the phase of migrating from git to cvs.

                                                         ^^^^^^^^^^^^^^^
Man... You are _definitely_ on the wrong list. ;)

He releases tar balls of his software project using gitweb. He would
love to have a way to have the commit-id of HEAD of the export contained
in one of the files he exported that way. Is there infrastructure in git
that makes that already possible or does he need to some kind of
gerneration tool by himself? Maybe it would be helpful if the
git-tar-tree would generate a file .commitid or something like that in
the generated tar tree.

Hmm, doesn't seem like git-tar-tree (or git-archive for that matter) 
supports this out of the box. Maybe it's possible to achieve in combination 
with the $Id$ construct?

I guess it depends on whether git-tar-tree/git-archive actually does a 
checkout from the repo from which the archive is made. If so, it should 
be possible to store "$Id$" in .commitid, and check it in, and it should 
automagically appear with the correct commit-id in your archive.

Of course, it all depends on whether the $Id$ conversion is triggered by 
git-archive...


Have fun!

...Johan

-- 
Johan Herland, [off-list ref]
www.herland.net

On 5/17/07, Thomas Glanzmann [off-list ref] wrote:

Hello,
Michae, a friend of mine, is in the phase of migrating from git to cvs.
He releases tar balls of his software project using gitweb. He would
love to have a way to have the commit-id of HEAD of the export contained
in one of the files he exported that way. Is there infrastructure in git
that makes that already possible or does he need to some kind of
gerneration tool by himself? Maybe it would be helpful if the
git-tar-tree would generate a file .commitid or something like that in
the generated tar tree.

Use git-get-tar-commit-id:

  $ gzip -cd snapshot.tar.gz | git-get-tar-commit-id

cheers,
Kristian

On Thu, May 17, 2007 at 06:57:22PM +0200, Johan Herland wrote:

Of course, it all depends on whether the $Id$ conversion is triggered by 
git-archive...

Another possibility might be to add a commandline switch to git-archive
so you can decide whether the commit id should be added as a header to
the tar file (which it already supports) or as a ordinary file (which
should be reasonable trivial to implement). The question if whether
it would be worth to add that feature. Don't know if there are many
other users out there that need it.

Gruesse,
-- 
Frank Lichtenheld [off-list ref]
www: http://www.djpig.de/

Hello,

 $ gzip -cd snapshot.tar.gz | git-get-tar-commit-id

not good enough. What he wants is:

        - user loads tar tree down from gitweb via the 'snapshot'
          button.

        - user extracts the tarball and types make

        - The output of the produced binary contains a unique
          identifier.

        Thomas

Hello,

Another possibility might be to add a commandline switch to
git-archive so you can decide whether the commit id should be added as
a header to the tar file (which it already supports) or as a ordinary
file (which should be reasonable trivial to implement). The question
if whether it would be worth to add that feature. Don't know if there
are many other users out there that need it.

that would be very good aproach I guess. At least from my point of view.
Because it doesn't break diffs, it concentrates on what matters and if
you don't like it you don't use it.

        Thomas

On Thursday 17 May 2007, Frank Lichtenheld wrote:

On Thu, May 17, 2007 at 06:57:22PM +0200, Johan Herland wrote:

quoted

Of course, it all depends on whether the $Id$ conversion is triggered by 
git-archive...

Another possibility might be to add a commandline switch to git-archive
so you can decide whether the commit id should be added as a header to
the tar file (which it already supports) or as a ordinary file (which
should be reasonable trivial to implement). The question if whether
it would be worth to add that feature. Don't know if there are many
other users out there that need it.

Although this efficiently solves Michael's problem, I still think the ideal 
solution would be for git-archive to do the same conversions/filters as a 
regular checkout would. Otherwise, we'll easily get into situations where 
a git-archive tree is different enough from a "regular" working tree to 
cause annoying differences in behaviour.


Have fun!

...Johan

-- 
Johan Herland, [off-list ref]
www.herland.net

On Thu, May 17, 2007 at 06:57:22PM +0200, Johan Herland wrote:

Hmm, doesn't seem like git-tar-tree (or git-archive for that matter) 
supports this out of the box. Maybe it's possible to achieve in combination 
with the $Id$ construct?

$Id$ contains the blob id, not the commit id.

Gruesse,
-- 
Frank Lichtenheld [off-list ref]
www: http://www.djpig.de/

On Thursday 17 May 2007, Frank Lichtenheld wrote:

On Thu, May 17, 2007 at 06:57:22PM +0200, Johan Herland wrote:

quoted

Hmm, doesn't seem like git-tar-tree (or git-archive for that matter) 
supports this out of the box. Maybe it's possible to achieve in combination 
with the $Id$ construct?

$Id$ contains the blob id, not the commit id.

Oops. My bad. In that case, your solution (to add another commandline switch 
to git-archive) is definitely what Michael needs.


...Johan

-- 
Johan Herland, [off-list ref]
www.herland.net

As noted by Johan Herland, git-archive is a kind of checkout and needs
to apply any checkout filters that might be configured.

This patch adds the convenience function convert_sha1_file which returns
a buffer containing the object's contents, after converting, if necessary
(i.e. it's a combination of read_sha1_file and convert_to_working_tree).
Direct calls to read_sha1_file in git-archive are then replaced by calls
to convert_sha1_file.

Since convert_sha1_file expects its path argument to be NUL-terminated --
a convention it inherits from convert_to_working_tree -- the patch also
changes the path handling in archive-tar.c to always NUL-terminate the
string.  It used to solely rely on the len field of struct strbuf before.

archive-zip.c already NUL-terminates the path and thus needs no such
change.

Signed-off-by: Rene Scharfe <redacted>

---
 archive-tar.c |   12 +++++++-----
 archive-zip.c |    2 +-
 cache.h       |    1 +
 convert.c     |   15 +++++++++++++++
 4 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/archive-tar.c b/archive-tar.c
index 56ff356..eb0abc7 100644
--- a/archive-tar.c
+++ b/archive-tar.c

@@ -83,11 +83,12 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
 	int slen = strlen(s);
 	int total = sb->len + slen;
 	if (total > sb->alloc) {
-		sb->buf = xrealloc(sb->buf, total);
-		sb->alloc = total;
+		sb->buf = xrealloc(sb->buf, total + 1);
+		sb->alloc = total + 1;
 	}
 	memcpy(sb->buf + sb->len, s, slen);
 	sb->len = total;
+	sb->buf[total] = '\0';
 }
 
 /*

@@ -272,18 +273,19 @@ static int write_tar_entry(const unsigned char *sha1,
 	}
 	if (path.alloc < baselen + filenamelen) {
 		free(path.buf);
-		path.buf = xmalloc(baselen + filenamelen);
-		path.alloc = baselen + filenamelen;
+		path.buf = xmalloc(baselen + filenamelen + 1);
+		path.alloc = baselen + filenamelen + 1;
 	}
 	memcpy(path.buf, base, baselen);
 	memcpy(path.buf + baselen, filename, filenamelen);
 	path.len = baselen + filenamelen;
+	path.buf[path.len] = '\0';
 	if (S_ISDIR(mode) || S_ISDIRLNK(mode)) {
 		strbuf_append_string(&path, "/");
 		buffer = NULL;
 		size = 0;
 	} else {
-		buffer = read_sha1_file(sha1, &type, &size);
+		buffer = convert_sha1_file(path.buf, sha1, mode, &type, &size);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 	}

diff --git a/archive-zip.c b/archive-zip.c
index 1eaf262..3cbf6bb 100644
--- a/archive-zip.c
+++ b/archive-zip.c

@@ -195,7 +195,7 @@ static int write_zip_entry(const unsigned char *sha1,
 		if (S_ISREG(mode) && zlib_compression_level != 0)
 			method = 8;
 		result = 0;
-		buffer = read_sha1_file(sha1, &type, &size);
+		buffer = convert_sha1_file(path, sha1, mode, &type, &size);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 		crc = crc32(crc, buffer, size);

diff --git a/cache.h b/cache.h
index aaeb04a..4204bc1 100644
--- a/cache.h
+++ b/cache.h

@@ -548,6 +548,7 @@ extern void trace_argv_printf(const char **argv, int count, const char *format,
 /* convert.c */
 extern char *convert_to_git(const char *path, const char *src, unsigned long *sizep);
 extern char *convert_to_working_tree(const char *path, const char *src, unsigned long *sizep);
+extern void *convert_sha1_file(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size);
 
 /* match-trees.c */
 void shift_tree(const unsigned char *, const unsigned char *, unsigned char *, int);

diff --git a/convert.c b/convert.c
index 12abdaf..c64880b 100644
--- a/convert.c
+++ b/convert.c

@@ -652,3 +652,18 @@ char *convert_to_working_tree(const char *path, const char *src, unsigned long *
 
 	return buf;
 }
+
+void *convert_sha1_file(const char *path, const unsigned char *sha1,
+                        unsigned int mode, enum object_type *type,
+                        unsigned long *size)
+{
+	void *buffer = read_sha1_file(sha1, type, size);
+	if (S_ISREG(mode) && buffer) {
+		void *converted = convert_to_working_tree(path, buffer, size);
+		if (converted) {
+			free(buffer);
+			buffer = converted;
+		}
+	}
+	return buffer;
+}

On Sat, 19 May 2007, René Scharfe wrote:

Conditional needs a "+ 1", too.

Same here.

	-Daniel
*This .sig left intentionally blank*

Daniel Barkalow schrieb:

Conditional needs a "+ 1", too.

[...]

Same here.

Thank you for spotting this.  Fix-up patch below.

Signed-off-by: Rene Scharfe <redacted>

---
Embarrassing.  I'm off to go to sleep now.

 archive-tar.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/archive-tar.c b/archive-tar.c
index eb0abc7..33e7657 100644
--- a/archive-tar.c
+++ b/archive-tar.c

@@ -82,7 +82,7 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
 {
 	int slen = strlen(s);
 	int total = sb->len + slen;
-	if (total > sb->alloc) {
+	if (total + 1 > sb->alloc) {
 		sb->buf = xrealloc(sb->buf, total + 1);
 		sb->alloc = total + 1;
 	}

@@ -271,7 +271,7 @@ static int write_tar_entry(const unsigned char *sha1,
 		path.alloc = PATH_MAX;
 		path.len = path.eof = 0;
 	}
-	if (path.alloc < baselen + filenamelen) {
+	if (path.alloc < baselen + filenamelen + 1) {
 		free(path.buf);
 		path.buf = xmalloc(baselen + filenamelen + 1);
 		path.alloc = baselen + filenamelen + 1;

Frank Lichtenheld schrieb:

On Thu, May 17, 2007 at 06:57:22PM +0200, Johan Herland wrote:

quoted

Of course, it all depends on whether the $Id$ conversion is triggered by 
git-archive...

Another possibility might be to add a commandline switch to git-archive
so you can decide whether the commit id should be added as a header to
the tar file (which it already supports) or as a ordinary file (which
should be reasonable trivial to implement). The question if whether
it would be worth to add that feature. Don't know if there are many
other users out there that need it.

Something like the following patch?  Since we're already embedding the
commit ID in a comment, we might as well offer creating a synthetic file
for it, too, if that solves a user's problem that might be difficult to
work around otherwise.

René


 Documentation/git-archive.txt |    4 ++++
 archive-tar.c                 |    7 +++++++
 archive-zip.c                 |    7 +++++++
 archive.h                     |    1 +
 builtin-archive.c             |   11 +++++++++++
 5 files changed, 30 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt
index 721e035..ac31aac 100644
--- a/Documentation/git-archive.txt
+++ b/Documentation/git-archive.txt

@@ -43,6 +43,10 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+--commit-id-file=<filename>::
+	Adds a file to the archive containing the commit ID.  This option
+	is ignored if <tree-ish> references a tree instead of a commit.
+
 <extra>::
 	This can be any options that the archiver backend understand.
 	See next section.

diff --git a/archive-tar.c b/archive-tar.c
index 33e7657..555850a 100644
--- a/archive-tar.c
+++ b/archive-tar.c

@@ -319,6 +319,13 @@ int write_tar_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_tar_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_tar_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_trailer();
 
 	return 0;

diff --git a/archive-zip.c b/archive-zip.c
index 3cbf6bb..88c5dfa 100644
--- a/archive-zip.c
+++ b/archive-zip.c

@@ -328,6 +328,13 @@ int write_zip_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_zip_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_zip_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_zip_trailer(args->commit_sha1);
 
 	free(zip_dir);

diff --git a/archive.h b/archive.h
index 6838dc7..020f82f 100644
--- a/archive.h
+++ b/archive.h

@@ -8,6 +8,7 @@ struct archiver_args {
 	const char *base;
 	struct tree *tree;
 	const unsigned char *commit_sha1;
+	const char *commit_sha1_file;
 	time_t time;
 	const char **pathspec;
 	unsigned int verbose : 1;

diff --git a/builtin-archive.c b/builtin-archive.c
index 7f4e409..e58ea16 100644
--- a/builtin-archive.c
+++ b/builtin-archive.c

@@ -151,6 +151,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	int extra_argc = 0;
 	const char *format = "tar";
 	const char *base = "";
+	const char *commit_sha1_file = NULL;
 	int verbose = 0;
 	int i;
 

@@ -174,6 +175,10 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 			base = arg + 9;
 			continue;
 		}
+		if (!prefixcmp(arg, "--commit-id-file=")) {
+			commit_sha1_file = arg + 17;
+			continue;
+		}
 		if (!strcmp(arg, "--")) {
 			i++;
 			break;

@@ -192,6 +197,11 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 		usage(archive_usage);
 	if (init_archiver(format, ar) < 0)
 		die("Unknown archive format '%s'", format);
+	if (commit_sha1_file) {
+		size_t namelen = strlen(commit_sha1_file);
+		if (namelen == 0 || commit_sha1_file[namelen - 1] == '/')
+			die("Invalid commit ID file name: %s", commit_sha1_file);
+	}
 
 	if (extra_argc) {
 		if (!ar->parse_extra)

@@ -201,6 +211,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	}
 	ar->args.verbose = verbose;
 	ar->args.base = base;
+	ar->args.commit_sha1_file = commit_sha1_file;
 
 	return i;
 }