Wishlist for a bundle-only transport mode

Hi *,

  I have to work in a bundle-only mode, but there are some problems
with it, namely:

1) git-clone does not accept a bundle file, even if git-fetch does.
I've made a patch to use git-fetch in git-clone for this.

2) The bundles created with "git bundle" does not record the HEAD,
they resolve the symbolic name to a branch name.

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
  url = /file/to/bundle
  fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

Thank you

Santi

Santi Béjar wrote:

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
  url = /file/to/bundle
  fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and
path meaning "local" protocol, i.e. git repository on the same
filesystem? "bundle = /file/to/bundle" perhaps...

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

On Nov 21, 2007 4:04 PM, Jakub Narebski [off-list ref] wrote:

Santi Béjar wrote:

quoted

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
url = /file/to/bundle
fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and
path meaning "local" protocol, i.e. git repository on the same
filesystem? "bundle = /file/to/bundle" perhaps...

Good point. The git repository must exist before pushing, but the bundle:

1) Exist and is a file, do nothing by default, maybe requiere a --overwrite
2) Does not exist, create the bundle, but maybe you've written badly the url, so
    the best thing would be to say explicitly that you wanted a bundle
like you said.

Santi

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

Santi B?jar wrote:

quoted

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
? url = /file/to/bundle
? fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and path 
meaning "local" protocol, i.e. git repository on the same filesystem? 

Maybe because the git repository is specified as an existing directory?  
The bundle is specified as a (possibly non-existing) file...

Hth,
Dscho

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

1) git-clone does not accept a bundle file, even if git-fetch does. I've 
made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over git-init, 
-remote and -fetch, is a really low hanging fruit.

Or maybe go the full nine yards and build it in.  Should be a breeze now, 
given parse_options() and run_command().

2) The bundles created with "git bundle" does not record the HEAD, they 
resolve the symbolic name to a branch name.

It imitates ls-remote output.

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
  url = /file/to/bundle
  fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

Does not seem to be too complicated to add it to transport.c IMHO.  
Something similar to rsync_transport_push().  And I'd definitely refuse to 
overwrite any existing file.

Ciao,
Dscho

On Nov 21, 2007 4:59 PM, Johannes Schindelin [off-list ref] wrote:

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

1) git-clone does not accept a bundle file, even if git-fetch does. I've
made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over git-init,
-remote and -fetch, is a really low hanging fruit.

For the basic/normal mode it can be a very thin wrapper but you have
to support --local, --shared, --reference...

quoted

2) The bundles created with "git bundle" does not record the HEAD, they
resolve the symbolic name to a branch name.

It imitates ls-remote output.

No, it does not.

With a newly created project with one commit:

$ git ls-remote git.git
b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master

$ git bundle create git.bdl HEAD master

$ git bundle list-heads git.bdl
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master

Santi

On Wed, 2007-11-21 at 15:59 +0000, Johannes Schindelin wrote:

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

1) git-clone does not accept a bundle file, even if git-fetch does. I've 
made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over git-init, 
-remote and -fetch, is a really low hanging fruit.

Or maybe go the full nine yards and build it in.  Should be a breeze now, 
given parse_options() and run_command().

I started doing this, which is pretty stupid as I'm pressed to find time
to finish up builtin-commit.  Nevertheless, could you elaborate on the
plan here?  How would you rewrite it to just use remote and fetch?  I
just finished the boilerplate option parsing stuff (patch below) and
started reading the core of git-clone.sh, but if most of this can be
replaced I'd like to hear about it :)

cheers,
Kristian

From ccfa6b7b6c0bad98201166db3f91760b423ab0c9 Mon Sep 17 00:00:00 2001

From: =?utf-8?q?Kristian=20H=C3=B8gsberg?= <redacted>
Date: Sun, 18 Nov 2007 18:48:04 -0500
Subject: [PATCH] builtin-clone: first steps, up to and including
git-init.

---
 Makefile                                      |    2 +-
 builtin-clone.c                               |  286
+++++++++++++++++++++++++
 builtin-init-db.c                             |   90 ++++----
 builtin.h                                     |    1 +
 cache.h                                       |    4 +
 git-clone.sh => contrib/examples/git-clone.sh |    0 
 git.c                                         |    1 +
 7 files changed, 341 insertions(+), 43 deletions(-)
 create mode 100644 builtin-clone.c
 rename git-clone.sh => contrib/examples/git-clone.sh (100%)

diff --git a/Makefile b/Makefile
index 4d23d12..f39b7e8 100644
--- a/Makefile
+++ b/Makefile

@@ -213,7 +213,6 @@ BASIC_LDFLAGS =
 
 SCRIPT_SH = \
 	git-bisect.sh git-checkout.sh \
-	git-clone.sh \
 	git-merge-one-file.sh git-mergetool.sh git-parse-remote.sh \
 	git-pull.sh git-rebase.sh git-rebase--interactive.sh \
 	git-repack.sh git-request-pull.sh \

@@ -327,6 +326,7 @@ BUILTIN_OBJS = \
 	builtin-checkout-index.o \
 	builtin-check-ref-format.o \
 	builtin-clean.o \
+	builtin-clone.o \
 	builtin-commit.o \
 	builtin-commit-tree.o \
 	builtin-count-objects.o \

diff --git a/builtin-clone.c b/builtin-clone.c
new file mode 100644
index 0000000..8a569e0
--- /dev/null
+++ b/builtin-clone.c

@@ -0,0 +1,286 @@
+/*
+ * Builtin "git clone"
+ *
+ * Copyright (c) 2007 Kristian Høgsberg <krh@redhat.com>
+ * Based on git-commit.sh by Junio C Hamano and Linus Torvalds
+ *
+ * Clone a repository into a different directory that does not yet

exist.
+ */
+
+#include "cache.h"
+#include "parse-options.h"
+
+/*
+ * Implementation notes:
+ *  - dropping use-separate-remote and no-separate-remote compatibility
+ *
+ */
+static const char * const builtin_clone_usage[] = {
+	"git-clone [options] [--] <repo> [<dir>]",
+	NULL
+};
+
+static int option_quiet, option_no_checkout, option_bare;
+static int option_local, option_no_hardlinks, option_shared,
option_depth;
+static char *option_template, *option_reference, *option_origin;
+static char *option_upload_pack;
+
+static struct option builtin_clone_options[] = {
+	OPT__QUIET(&option_quiet),
+	OPT_BOOLEAN('n', "no-checkout", &option_no_checkout,
+		    "don't create a checkout"),
+	OPT_BOOLEAN(0, "bare", &option_bare, "create a bare repository"),
+	OPT_BOOLEAN(0, "naked", &option_bare, "create a bare repository"),
+	OPT_BOOLEAN('l', "local", &option_local,
+		    "to clone from a local repository"),
+	OPT_BOOLEAN(0, "no-hardlinks", &option_no_hardlinks,
+		    "don't use local hardlinks, always copy"),
+	OPT_BOOLEAN('s', "shared", &option_shared,
+		    "setup as shared repository"),
+	OPT_STRING(0, "template", &option_template, "path",
+		   "path the template repository"),
+	OPT_STRING(0, "reference", &option_reference, "repo",
+		   "reference repository"),
+	OPT_STRING('o', "origin", &option_origin, "branch",
+		   "use <branch> instead or 'origin' to track upstream"),
+	OPT_STRING('u', "upload-pack", &option_upload_pack, "path",
+		   "path to git-upload-pack on the remote"),
+	OPT_INTEGER(0, "depth", &option_depth,
+		    "create a shallow clone of that depth"),
+
+	OPT_END()
+};
+
+static char *get_repo_path(const char *repo)
+{
+	const char *path;
+	struct stat buf;
+
+	path = mkpath("%s/.git", repo);
+	if (!stat(path, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(path));
+
+	path = mkpath("%s.git", repo);
+	if (!stat(path, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(path));
+
+	if (!stat(repo, &buf) && S_ISDIR(buf.st_mode))
+		return xstrdup(make_absolute_path(repo));
+	
+	return NULL;
+}
+
+static char *guess_dir_name(const char *repo)
+{
+	const char *p, *start, *end, *limit;
+	int after_slash_or_colon;
+
+	/* Guess dir name from repository: strip trailing '/',
+	 * strip trailing '[:/]*git', strip leading '.*[/:]'. */
+
+	after_slash_or_colon = 1;
+	limit = repo + strlen(repo);
+	start = repo;
+	end = limit;
+	for (p = repo; p < limit; p++) {
+		if (!prefixcmp(p, ".git")) {
+			if (!after_slash_or_colon)
+				end = p;
+			p += 3;
+		} else if (*p == '/' || *p == ':') {
+			if (end == limit)
+				end = p; 
+			after_slash_or_colon = 1;
+		} else if (after_slash_or_colon) {
+			start = p;
+			end = limit;
+			after_slash_or_colon = 0;
+		}
+	}
+
+	return xstrndup(start, end - start);
+}
+
+static void
+setup_reference(void)
+{
+	struct stat buf;
+	const char *ref_git, *alternates;
+	int fd;
+
+	if (!option_reference)
+		return;
+
+	if (!stat(mkpath("%s/.git/objects", option_reference), &buf) &&
+	    S_ISDIR(buf.st_mode))
+		ref_git = mkpath("%s/.git", option_reference);
+	else if (!stat(mkpath("%s/objects", option_reference), &buf) &&
+		 S_ISDIR(buf.st_mode))
+		ref_git = option_reference;
+	else
+		die("reference repository '%s' is not a local directory.",
+		    option_reference);
+
+	ref_git = make_absolute_path(ref_git);
+	
+	fd = open(git_path("objects/info/alternates"),
+		  O_WRONLY | O_CREAT, 0666);
+	if (fd < 0)
+		die("could not create alternates file\n");
+	alternates = mkpath("%s/objects\n", ref_git);
+	write_or_die(fd, alternates, strlen(alternates));
+	if (close(fd) < 0)
+		die("could not close alternates file\n");
+
+	/* This part is going to be tricky, since it deals with two
+	 * repositories.  Most code in git works on global state
+	 * assumed to be coming from the one active repository, but
+	 * here we want to do for-each-ref in one repository and say
+	 * update-ref in another.  We could just say set_git_dir() and
+	 * then say for-each-ref and store the refs in an array and
+	 * the say set_git_dir() again to switch to the repository
+	 * we're creating and then write out the refs using update-ref. */
+
+	/*
+
+	echo "$ref_git/objects" >"$GIT_DIR/objects/info/alternates"
+	(
+		GIT_DIR="$ref_git" git for-each-ref \
+			--format='%(objectname) %(*objectname)'
+	) |
+	while read a b
+	do
+		test -z "$a" ||
+		git update-ref "refs/reference-tmp/$a" "$a"
+		test -z "$b" ||
+		git update-ref "refs/reference-tmp/$b" "$b"
+	done
+
+	*/
+}
+
+static void
+cleanup_reference(void)
+{
+	/*
+	  test -d "$GIT_DIR/refs/reference-tmp" &&
+		rm -fr "$GIT_DIR/refs/reference-tmp"
+	*/
+}
+
+static void
+clone_local(const char *path)
+{
+	/* local hardlink magic */
+	printf("local, from %s\n", path);
+}
+
+static void
+clone_rsync(const char *repo)
+{
+	printf("rsync, from %s\n", repo);
+	if (option_depth > 0)
+		die("shallow over rsync not supported");
+}
+
+static void
+clone_native(const char *repo)
+{
+		printf("native git, from %s\n", repo);
+}
+
+static void
+clone_curl(const char *repo)
+{
+	printf("curl, from %s\n", repo);
+	if (option_depth > 0)
+		die("shallow over rsync not supported");
+}
+
+int cmd_clone(int argc, const char **argv, const char *prefix)
+{
+	int use_local_hardlinks = 1;
+	int use_separate_remote = 1;
+	struct stat buf;
+	const char *repo, *work_tree, *git_dir;
+	char *path, *dir;
+
+	argc = parse_options(argc, argv, builtin_clone_options,
+			     builtin_clone_usage, 0);
+
+	if (argc == 0)
+		die("You must specify a repository to clone.");
+
+	if (option_no_hardlinks)
+		use_local_hardlinks = 0;
+
+	if (option_bare) {
+		if (option_origin)
+			die("--bare and --origin %s options are incompatible.",
+			    option_origin);
+		option_no_checkout = 1;
+		use_separate_remote = 0;
+	}
+
+	if (!option_origin)
+		option_origin = "origin";
+
+	repo = argv[0];
+	path = get_repo_path(repo);
+
+	if (argc == 2) {
+		dir = xstrdup(argv[1]);
+	} else {
+		dir = guess_dir_name(repo);
+	}
+
+	printf("repo: %s, path: %s\n", repo, path);
+	printf("dir: %s\n", dir);
+
+	if (!stat(dir, &buf))
+		die("destination directory '%s' already exists.", dir);
+
+	if (option_bare)
+		work_tree = NULL;
+	else {
+		work_tree = getenv("GIT_WORK_TREE");
+		if (work_tree && !stat(work_tree, &buf))
+			die("working tree '%s' already exists.", work_tree);
+	}
+
+	if (mkdir(dir, 0755))
+		die("could not create repository dir '%s'.", dir);
+	if (work_tree && mkdir(work_tree, 0755))
+		die("could not create work tree dir '%s'.", work_tree);
+
+	if (option_bare || work_tree)
+		git_dir = xstrdup(dir); 
+	else
+		git_dir = xstrdup(mkpath("%s/.git", dir));
+
+	set_git_dir(make_absolute_path(git_dir));
+	init_db(git_dir, option_template, option_quiet ? INIT_DB_QUIET : 0);
+
+	if (option_bare)
+		git_config_set("core.bare", "true");
+
+	setup_reference();
+
+	if (path != NULL) {
+		clone_local(path);
+	} else if (!prefixcmp(repo, "rsync://")) {
+		clone_rsync(repo);
+	} else if (!prefixcmp(repo, "https://") ||
+		   !prefixcmp(repo, "http://") ||
+		   !prefixcmp(repo, "ftp://")) {
+		clone_curl(repo);
+	} else {
+		clone_native(repo);
+	}
+
+	cleanup_reference();
+
+
+
+	return 0;
+}

diff --git a/builtin-init-db.c b/builtin-init-db.c
index e1393b8..0ddfb46 100644
--- a/builtin-init-db.c
+++ b/builtin-init-db.c

@@ -330,49 +330,11 @@ static void guess_repository_type(const char

*git_dir)
 	return;
 }
 
-static const char init_db_usage[] =
-"git-init [-q | --quiet] [--template=<template-directory>] [--shared]";
-
-/*
- * If you want to, you can share the DB area with any number of
branches.
- * That has advantages: you can save space by sharing all the SHA1
objects.
- * On the other hand, it might just make lookup slower and messier. You
- * be the judge.  The default case is to have one DB per managed
directory.
- */
-int cmd_init_db(int argc, const char **argv, const char *prefix)
+int init_db(const char *git_dir, const char *template_dir, unsigned int
flags)
 {
-	const char *git_dir;
 	const char *sha1_dir;
-	const char *template_dir = NULL;
 	char *path;
-	int len, i, reinit;
-	int quiet = 0;
-
-	for (i = 1; i < argc; i++, argv++) {
-		const char *arg = argv[1];
-		if (!prefixcmp(arg, "--template="))
-			template_dir = arg+11;
-		else if (!strcmp(arg, "--shared"))
-			shared_repository = PERM_GROUP;
-		else if (!prefixcmp(arg, "--shared="))
-			shared_repository = git_config_perm("arg", arg+9);
-		else if (!strcmp(arg, "-q") || !strcmp(arg, "--quiet"))
-		        quiet = 1;
-		else
-			usage(init_db_usage);
-	}
-
-	/*
-	 * GIT_WORK_TREE makes sense only in conjunction with GIT_DIR
-	 * without --bare.  Catch the error early.
-	 */
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
-	if ((!git_dir || is_bare_repository_cfg == 1)
-	    && getenv(GIT_WORK_TREE_ENVIRONMENT))
-		die("%s (or --work-tree=<directory>) not allowed without "
-		    "specifying %s (or --git-dir=<directory>)",
-		    GIT_WORK_TREE_ENVIRONMENT,
-		    GIT_DIR_ENVIRONMENT);
+	int len, reinit;
 
 	guess_repository_type(git_dir);
 

@@ -388,7 +350,6 @@ int cmd_init_db(int argc, const char **argv, const

char *prefix)
 	/*
 	 * Set up the default .git directory contents
 	 */
-	git_dir = getenv(GIT_DIR_ENVIRONMENT);
 	if (!git_dir)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
 	safe_create_dir(git_dir, 0);

@@ -427,7 +388,7 @@ int cmd_init_db(int argc, const char **argv, const

char *prefix)
 		git_config_set("receive.denyNonFastforwards", "true");
 	}
 
-	if (!quiet)
+	if (!(flags & INIT_DB_QUIET))
 		printf("%s%s Git repository in %s/\n",
 		       reinit ? "Reinitialized existing" : "Initialized empty",
 		       shared_repository ? " shared" : "",

@@ -435,3 +396,48 @@ int cmd_init_db(int argc, const char **argv, const

char *prefix)
 
 	return 0;
 }
+
+static const char init_db_usage[] =
+"git-init [-q | --quiet] [--template=<template-directory>] [--shared]";
+
+/*
+ * If you want to, you can share the DB area with any number of
branches.
+ * That has advantages: you can save space by sharing all the SHA1
objects.
+ * On the other hand, it might just make lookup slower and messier. You
+ * be the judge.  The default case is to have one DB per managed
directory.
+ */
+int cmd_init_db(int argc, const char **argv, const char *prefix)
+{
+	const char *git_dir;
+	const char *template_dir = NULL;
+	unsigned int flags = 0;
+	int i;
+
+	for (i = 1; i < argc; i++, argv++) {
+		const char *arg = argv[1];
+		if (!prefixcmp(arg, "--template="))
+			template_dir = arg+11;
+		else if (!strcmp(arg, "--shared"))
+			shared_repository = PERM_GROUP;
+		else if (!prefixcmp(arg, "--shared="))
+			shared_repository = git_config_perm("arg", arg+9);
+		else if (!strcmp(arg, "-q") || !strcmp(arg, "--quiet"))
+		        flags |= INIT_DB_QUIET;
+		else
+			usage(init_db_usage);
+	}
+
+	/*
+	 * GIT_WORK_TREE makes sense only in conjunction with GIT_DIR
+	 * without --bare.  Catch the error early.
+	 */
+	git_dir = getenv(GIT_DIR_ENVIRONMENT);
+	if ((!git_dir || is_bare_repository_cfg == 1)
+	    && getenv(GIT_WORK_TREE_ENVIRONMENT))
+		die("%s (or --work-tree=<directory>) not allowed without "
+		    "specifying %s (or --git-dir=<directory>)",
+		    GIT_WORK_TREE_ENVIRONMENT,
+		    GIT_DIR_ENVIRONMENT);
+
+	return init_db(git_dir, template_dir, flags);
+}

diff --git a/builtin.h b/builtin.h
index 6f01e96..8e59778 100644
--- a/builtin.h
+++ b/builtin.h

@@ -24,6 +24,7 @@ extern int cmd_check_attr(int argc, const char **argv,

const char *prefix);
 extern int cmd_check_ref_format(int argc, const char **argv, const char
*prefix);
 extern int cmd_cherry(int argc, const char **argv, const char *prefix);
 extern int cmd_cherry_pick(int argc, const char **argv, const char
*prefix);
+extern int cmd_clone(int argc, const char **argv, const char *prefix);
 extern int cmd_clean(int argc, const char **argv, const char *prefix);
 extern int cmd_commit(int argc, const char **argv, const char *prefix);
 extern int cmd_commit_tree(int argc, const char **argv, const char
*prefix);

diff --git a/cache.h b/cache.h
index 510154b..8c5f6af 100644
--- a/cache.h
+++ b/cache.h

@@ -230,6 +230,10 @@ extern const char *prefix_filename(const char

*prefix, int len, const char *path
 extern void verify_filename(const char *prefix, const char *name);
 extern void verify_non_filename(const char *prefix, const char *name);
 
+#define INIT_DB_QUIET 0x0001
+
+extern int init_db(const char *git_dir, const char *template_dir,
unsigned int flags);
+
 #define alloc_nr(x) (((x)+16)*3/2)
 
 /*

diff --git a/git-clone.sh b/contrib/examples/git-clone.sh
similarity index 100%
rename from git-clone.sh
rename to contrib/examples/git-clone.sh
diff --git a/git.c b/git.c
index 9606937..9236833 100644
--- a/git.c
+++ b/git.c

@@ -293,6 +293,7 @@ static void handle_internal_command(int argc, const

char **argv)
 		{ "check-attr", cmd_check_attr, RUN_SETUP | NEED_WORK_TREE },
 		{ "cherry", cmd_cherry, RUN_SETUP },
 		{ "cherry-pick", cmd_cherry_pick, RUN_SETUP | NEED_WORK_TREE },
+		{ "clone", cmd_clone },
 		{ "clean", cmd_clean, RUN_SETUP | NEED_WORK_TREE },
 		{ "commit", cmd_commit, RUN_SETUP | NEED_WORK_TREE },
 		{ "commit-tree", cmd_commit_tree, RUN_SETUP },
-- 
1.5.3.4

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

On Nov 21, 2007 4:59 PM, Johannes Schindelin [off-list ref] wrote:

quoted

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

1) git-clone does not accept a bundle file, even if git-fetch does. 
I've made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over 
git-init, -remote and -fetch, is a really low hanging fruit.

For the basic/normal mode it can be a very thin wrapper but you have to 
support --local, --shared, --reference...

That is not all that difficult.

quoted

quoted

2) The bundles created with "git bundle" does not record the HEAD, 
they resolve the symbolic name to a branch name.

It imitates ls-remote output.

No, it does not.

With a newly created project with one commit:

$ git ls-remote git.git
b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master

$ git bundle create git.bdl HEAD master

$ git bundle list-heads git.bdl
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master

Ah, I misunderstood.  I thought you were expecting that the first line 
would read

	refs/heads/master	HEAD

Alas, this behaviour stems from dwim_ref() returning "refs/heads/master" 
as real ref.

I am not quite sure how to solve this, though.  Let me see what I can come 
up with.

Ciao,
Dscho
 

On Nov 21, 2007 5:36 PM, Johannes Schindelin [off-list ref] wrote:

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

On Nov 21, 2007 4:59 PM, Johannes Schindelin [off-list ref] wrote:

quoted

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

1) git-clone does not accept a bundle file, even if git-fetch does.
I've made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over
git-init, -remote and -fetch, is a really low hanging fruit.

For the basic/normal mode it can be a very thin wrapper but you have to
support --local, --shared, --reference...

That is not all that difficult.

I did not say it was difficult, I said it was not a very thin wrapper.

quoted

quoted

quoted

2) The bundles created with "git bundle" does not record the HEAD,
they resolve the symbolic name to a branch name.

It imitates ls-remote output.

No, it does not.

With a newly created project with one commit:

$ git ls-remote git.git
b71992713c17c3a29f9566e1b50e8cf402375faf        HEAD
b71992713c17c3a29f9566e1b50e8cf402375faf        refs/heads/master

$ git bundle create git.bdl HEAD master

$ git bundle list-heads git.bdl
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master
b71992713c17c3a29f9566e1b50e8cf402375faf refs/heads/master

Ah, I misunderstood.

Maybe because of my non-native english :-)

 I thought you were expecting that the first line
would read

        refs/heads/master       HEAD

or

ref: refs/heads/master        HEAD

That would be perfect, but it is a different story.

Alas, this behaviour stems from dwim_ref() returning "refs/heads/master"
as real ref.

I am not quite sure how to solve this, though.  Let me see what I can come
up with.

Thank you.

Santi

Johannes Schindelin wrote:

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Santi Bejar wrote:

quoted

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
	url = /file/to/bundle
	fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and path 
meaning "local" protocol, i.e. git repository on the same filesystem? 

Maybe because the git repository is specified as an existing directory?  
The bundle is specified as a (possibly non-existing) file...

That has the disadvantage of pushing to bundle when you make an error
in the lastpart of path to existing repository.

After thinking about it a bit, I think it would be better to use 
bundle:// pseudoprotocol for the URL including config (bundle://file,
bundle:///path/to/bundle) and --bundle option to git push for
commandline.

-- 
Jakub Narebski
Poland

When creating a bundle, symbolic refs used to be resolved to the
non-symbolic refs they point to before being written to the list
of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
would show something like

388afe7881b33102fada216dd07806728773c011        refs/heads/master
388afe7881b33102fada216dd07806728773c011        refs/heads/master

Introduce a special handling so that the symbolic refs are listed
with the names passed on the command line.

Noticed by Santi Béjar.

Signed-off-by: Johannes Schindelin <redacted>
---
 bundle.c |   13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/bundle.c b/bundle.c
index e4d60cd..5e8242f 100644
--- a/bundle.c
+++ b/bundle.c

@@ -6,6 +6,7 @@
 #include "revision.h"
 #include "list-objects.h"
 #include "run-command.h"
+#include "refs.h"
 
 static const char bundle_signature[] = "# v2 git bundle\n";
 

@@ -231,12 +232,17 @@ int create_bundle(struct bundle_header *header, const char *path,
 	for (i = 0; i < revs.pending.nr; i++) {
 		struct object_array_entry *e = revs.pending.objects + i;
 		unsigned char sha1[20];
-		char *ref;
+		const char *ref;
+		int flag;
 
 		if (e->item->flags & UNINTERESTING)
 			continue;
-		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
+		ref = resolve_ref(e->name, sha1, 1, &flag);
+		if (!ref)
 			continue;
+		if (flag & REF_ISSYMREF)
+			ref = e->name;
+
 		/*
 		 * Make sure the refs we wrote out is correct; --max-count and
 		 * other limiting options could have prevented all the tips

@@ -249,7 +255,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 		if (!(e->item->flags & SHOWN) && e->item->type == OBJ_COMMIT) {
 			warning("ref '%s' is excluded by the rev-list options",
 				e->name);
-			free(ref);
 			continue;
 		}
 		/*

@@ -280,7 +285,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 				obj->flags |= SHOWN;
 				add_pending_object(&revs, obj, e->name);
 			}
-			free(ref);
 			continue;
 		}
 

@@ -289,7 +293,6 @@ int create_bundle(struct bundle_header *header, const char *path,
 		write_or_die(bundle_fd, " ", 1);
 		write_or_die(bundle_fd, ref, strlen(ref));
 		write_or_die(bundle_fd, "\n", 1);
-		free(ref);
 	}
 	if (!ref_count)
 		die ("Refusing to create empty bundle.");

-- 
1.5.3.6.1977.g54d30

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

Johannes Schindelin wrote:

quoted

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Santi Bejar wrote:

quoted

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
	url = /file/to/bundle
	fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and path 
meaning "local" protocol, i.e. git repository on the same filesystem? 

Maybe because the git repository is specified as an existing directory?  
The bundle is specified as a (possibly non-existing) file...

That has the disadvantage of pushing to bundle when you make an error
in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing 
file.

After thinking about it a bit, I think it would be better to use 
bundle:// pseudoprotocol for the URL including config (bundle://file, 
bundle:///path/to/bundle) and --bundle option to git push for 
commandline.

I don't like that at all.

Specifying a non-existing file should be good enough.

Ciao,
Dscho

Hi,

On Wed, 21 Nov 2007, Kristian H?gsberg wrote:

On Wed, 2007-11-21 at 15:59 +0000, Johannes Schindelin wrote:

quoted

Hi,

On Wed, 21 Nov 2007, Santi B?jar wrote:

quoted

1) git-clone does not accept a bundle file, even if git-fetch does. I've 
made a patch to use git-fetch in git-clone for this.

This, along with rewriting git-clone as a very thin wrapper over git-init, 
-remote and -fetch, is a really low hanging fruit.

Or maybe go the full nine yards and build it in.  Should be a breeze now, 
given parse_options() and run_command().

I started doing this, which is pretty stupid as I'm pressed to find time 
to finish up builtin-commit.  Nevertheless, could you elaborate on the 
plan here?  How would you rewrite it to just use remote and fetch?  I 
just finished the boilerplate option parsing stuff (patch below) and 
started reading the core of git-clone.sh, but if most of this can be 
replaced I'd like to hear about it :)

After handling the options like --reference, and except for special 
handling of a local clone (not via file:// protocol), it should be as easy 
as

	git remote add -f origin $url &&
	case "$no_checkout" in
	t)
		;;
	*)
		git checkout -f -b master remotes/origin/HEAD
		;;
	esac

(Of course I'd use run_command() for this.)

I would avoid at all costs to reimplement the different methods for the 
different protocols.

Ciao,
Dscho

Johannes Schindelin wrote:

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Johannes Schindelin wrote:

quoted

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Santi Bejar wrote:

quoted

3) I can "git fetch" a bundle but I cannot "git push" a bundle, so if I have:

[remote "bundle"]
	url = /file/to/bundle
	fetch = "+refs/heads/*:refs/remotes/bundle/*"

$ git push bundle

would create a bundle in /file/to/bundle with the same branches as a
normal git push, but considering the remote branches as the local
remotes/bundle/*

And how you would differentiate between path meaning bundle, and path 
meaning "local" protocol, i.e. git repository on the same filesystem? 

Maybe because the git repository is specified as an existing directory?  
The bundle is specified as a (possibly non-existing) file...

That has the disadvantage of pushing to bundle when you make an error
in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing 
file.

Specifying a non-existing file should be good enough.

What I meant here that if you do "git push /some/path/to/rpeo.git", with
mistake in the last part of path to repository, you would end up with
a bundle, and you would have to really watch what happened to catch
the error.

I'd rather use "git push bundle:///some/path/to/bundle" or 
"git push --bundle bundlename" to catch errors better.

Besides it should be IMHO be possible to overwrite bundle if you are
doing fast-forward push...

-- 
Jakub Narebski
Poland

[Cc: Santi Béjar [off-list ref], git@vger.kernel.org]

Santi Béjar wrote:

1) git-clone does not accept a bundle file, even if git-fetch does.
I've made a patch to use git-fetch in git-clone for this.

[...] 

3) I can "git fetch" a bundle but I cannot "git push" a bundle

We would also need "git ls-remote <bundle>" and "git fsck <bundle>"
to demote git-bundle to porcelain status :-)

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

Johannes Schindelin wrote:

quoted

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

That has the disadvantage of pushing to bundle when you make an error 
in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing 
file.

quoted

Specifying a non-existing file should be good enough.

What I meant here that if you do "git push /some/path/to/rpeo.git", with 
mistake in the last part of path to repository, you would end up with a 
bundle, and you would have to really watch what happened to catch the 
error.

I use tab completion all the time, so this would not happen to me.  IMHO 
that is a lesser issue than to introduce a "protocol".

I'd rather use "git push bundle:///some/path/to/bundle" or "git push 
--bundle bundlename" to catch errors better.

Besides it should be IMHO be possible to overwrite bundle if you are 
doing fast-forward push...

Not as far as I can see.  A push there would see what the bundle has 
already, and put them into the new bundle as _prerequisites_.  So the 
bundle would lose information.

BTW this was my gripe (that I decided not to make public earlier) with 
Santi's proposal to begin with: a push would not have any way to specify 
what the other side has already.  So I think "git push <bundle>" is the 
wrong way of creating a bundle.

Except if we add some cunning strategy not to overwrite, ever, but to 
create <bundle>.<n> with an incrementing <n>.  But that might be too much.

Ciao,
Dscho

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

We would also need "git ls-remote <bundle>" and "git fsck <bundle>" to 
demote git-bundle to porcelain status :-)

git ls-remote <bundle> worked, last time I checked (15 seconds ago).  But 
git fsck?  Since when does something like "git fsck 
git://repo.or.cz/git.git" work?

Besides, "git bundle verify <bundle>" works quite well.

Ciao,
Dscho

On Wed, 21 Nov 2007, Johannes Schindelin wrote:

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Johannes Schindelin wrote:

quoted

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

That has the disadvantage of pushing to bundle when you make an error 
in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing 
file.

quoted

Specifying a non-existing file should be good enough.

What I meant here that if you do "git push /some/path/to/rpeo.git", with 
mistake in the last part of path to repository, you would end up with a 
bundle, and you would have to really watch what happened to catch the 
error.

I use tab completion all the time, so this would not happen to me.  IMHO 
that is a lesser issue than to introduce a "protocol".

When I copy'n'paste pathname I sometimes catch return / linefeed in
the middle, ending with incorect pathname (which would create bundle).
Tab completon is not always solution (tab completion can be slow,
or nonexistent).

[cut]

But I agree that because "git create bundle" needs refspecs _and revlist_,
while "git push" gets only refspecs because it calculates revlist it would
be better to have to use "git bundle create" to create bundle.

Contrary to "git clone <bundle>" which should just work IMHO.

-- 
Jakub Narebski
Poland

On Nov 21, 2007 6:26 PM, Johannes Schindelin [off-list ref] wrote:

Hi,

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

Johannes Schindelin wrote:

quoted

On Wed, 21 Nov 2007, Jakub Narebski wrote:

quoted

That has the disadvantage of pushing to bundle when you make an error
in the lastpart of path to existing repository.

As I wrote in another reply, I would not allow overwriting an existing
file.

quoted

Specifying a non-existing file should be good enough.

What I meant here that if you do "git push /some/path/to/rpeo.git", with
mistake in the last part of path to repository, you would end up with a
bundle, and you would have to really watch what happened to catch the
error.

I use tab completion all the time, so this would not happen to me.  IMHO
that is a lesser issue than to introduce a "protocol".

quoted

I'd rather use "git push bundle:///some/path/to/bundle" or "git push
--bundle bundlename" to catch errors better.

I would vote for the later, and a way to configure this in the config.

quoted

Besides it should be IMHO be possible to overwrite bundle if you are
doing fast-forward push...

Not as far as I can see.  A push there would see what the bundle has
already, and put them into the new bundle as _prerequisites_.  So the
bundle would lose information.

I prefer not to overwrite an existing bundle.

BTW this was my gripe (that I decided not to make public earlier) with
Santi's proposal to begin with: a push would not have any way to specify
what the other side has already.  So I think "git push <bundle>" is the
wrong way of creating a bundle.

Sorry but I do not understand this. I think this two lines could be equivalent:

git push --bundle bundle.bdl "refs/heads/master:refs/remotes/bundle/master"
git bundle create bundle.bdl refs/heads/master ^refs/remotes/bundle/master

Except if we add some cunning strategy not to overwrite, ever, but to
create <bundle>.<n> with an incrementing <n>.  But that might be too much.

That make sense.

Santi

Hi,

On Wed, 21 Nov 2007, Johannes Schindelin wrote:

When creating a bundle, symbolic refs used to be resolved to the
non-symbolic refs they point to before being written to the list
of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
would show something like

388afe7881b33102fada216dd07806728773c011        refs/heads/master
388afe7881b33102fada216dd07806728773c011        refs/heads/master

Aaargh.

My patch broke "git bundle create x master".  I somehow believed that 
resolve_ref () would turn "master" into "refs/heads/master", which it does 
not do.

So please ignore this patch.

Ciao,
Dscho

When creating a bundle, symbolic refs used to be resolved to the
non-symbolic refs they point to before being written to the list
of contained refs.  I.e. "git bundle create a1.bundle HEAD master"
would show something like

388afe7881b33102fada216dd07806728773c011        refs/heads/master
388afe7881b33102fada216dd07806728773c011        refs/heads/master

instead of

388afe7881b33102fada216dd07806728773c011        HEAD
388afe7881b33102fada216dd07806728773c011        refs/heads/master

Introduce a special handling so that the symbolic refs are listed
with the names passed on the command line.

Noticed by Santi Béjar.

Signed-off-by: Johannes Schindelin <redacted>
---

	... and this patch actually does not break the test suite.

 bundle.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/bundle.c b/bundle.c
index e4d60cd..9b9b916 100644
--- a/bundle.c
+++ b/bundle.c

@@ -6,6 +6,7 @@
 #include "revision.h"
 #include "list-objects.h"
 #include "run-command.h"
+#include "refs.h"
 
 static const char bundle_signature[] = "# v2 git bundle\n";
 

@@ -232,11 +233,17 @@ int create_bundle(struct bundle_header *header, const char *path,
 		struct object_array_entry *e = revs.pending.objects + i;
 		unsigned char sha1[20];
 		char *ref;
+		const char *display_ref;
+		int flag;
 
 		if (e->item->flags & UNINTERESTING)
 			continue;
 		if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
 			continue;
+		if (!resolve_ref(e->name, sha1, 1, &flag))
+			flag = 0;
+		display_ref = (flag & REF_ISSYMREF) ? e->name : ref;
+
 		/*
 		 * Make sure the refs we wrote out is correct; --max-count and
 		 * other limiting options could have prevented all the tips

@@ -287,7 +294,7 @@ int create_bundle(struct bundle_header *header, const char *path,
 		ref_count++;
 		write_or_die(bundle_fd, sha1_to_hex(e->item->sha1), 40);
 		write_or_die(bundle_fd, " ", 1);
-		write_or_die(bundle_fd, ref, strlen(ref));
+		write_or_die(bundle_fd, display_ref, strlen(display_ref));
 		write_or_die(bundle_fd, "\n", 1);
 		free(ref);
 	}

-- 
1.5.3.6.1977.g54d30