Question about .git/objects/info/alternates

6 messages, 3 authors, 2016-06-15 · open the first message on its own page

Question about .git/objects/info/alternates

From: Chris Packham <hidden>
Date: 2016-06-15 22:48:29

Hi All,

I've been using git clone --reference and git submodule update
--reference to reduce the amount of data transferred when I cloned a
repository that I already had an older copy of (that for one reason or
another I didn't want to touch or clone directly).

Now I'm finding that what I really want to do is change around what is
referencing what. I currently have the following.

  projecta.git
  base.git       # references project a
  projectb.git  # referenced base (which, now that I think about it,
was probably the wrong thing to do)

Ideally I'd want to end up with
  base.git      # has all objects
  projecta.git # uses base as a reference
  projectb.git # uses base as a reference also

I would like to have base somehow find the objects it doesn't have in
its object store and either download them or just copy them from the
object store of projecta. Then I can manually point projecta at base
and repack (as discussed in this thread [1]) to free up some space.
projectb should be fine as is because it already references base. Is
there any way to actually do this? A little googling found hints on
adding alternates after the fact but I'm actually interested in going
the other direction. From reading [2] I think 'rm
.git/objects/info/alternates && git repack -a' might do the trick but
I'm not sure.

[1] http://thread.gmane.org/gmane.comp.version-control.git/141161/focus=141199
[2] http://stackoverflow.com/questions/2248228/how-to-detach-alternates-after-git-clone-reference

Re: Question about .git/objects/info/alternates

From: Jonathan Nieder <hidden>
Date: 2016-06-15 22:48:29

Chris Packham wrote:
I would like to have base somehow find the objects it doesn't have in
its object store and either download them or just copy them from the
object store of projecta.
[...]
From reading [2] I think 'rm
.git/objects/info/alternates && git repack -a' might do the trick but
I'm not sure.
Almost.  Try ‘git repack -a && rm .git/objects/info/alternates’ instead. :)

(Please back up the repository or try with something less important
first, since I am not sure.)

Hope that helps,
Jonathan
[2] http://stackoverflow.com/questions/2248228/how-to-detach-alternates-after-git-clone-reference

Re: Question about .git/objects/info/alternates

From: Chris Packham <hidden>
Date: 2016-06-15 22:48:29

On Mon, Mar 22, 2010 at 7:42 PM, Jonathan Nieder [off-list ref] wrote:
Chris Packham wrote:
quoted
I would like to have base somehow find the objects it doesn't have in
its object store and either download them or just copy them from the
object store of projecta.
[...]
quoted
From reading [2] I think 'rm
.git/objects/info/alternates && git repack -a' might do the trick but
I'm not sure.
Almost.  Try ‘git repack -a && rm .git/objects/info/alternates’ instead. :)

(Please back up the repository or try with something less important
first, since I am not sure.)

Hope that helps,
Jonathan
git repack -a did the correct thing.

It occurs to me that the UI around alternates is a bit lacking i.e.
there isn't a git command to display the alternates in use or to add
them to an existing repository (or at least I couldn't find one
skimming the docs or googling). So here's my attempt to add a 'git
alternates' command which can display, add or remove an alternate. The
adding and removing could be done with standard shell commands but
I've found the recursive displaying quite useful and couldn't think of
a simple command line to achieve the same thing .

------8<------

From a5c64de20937da132376d717f19a1d52b54701d2 Mon Sep 17 00:00:00 2001
From: Chris Packham <redacted>
Date: Wed, 24 Mar 2010 11:34:11 -0700
Subject: [PATCH] Add git alternates command

Provides a friendlier UI for displaying and configuring alternates.

Signed-off-by: Chris Packham <redacted>
---

This patch assumes multiple alterates are possible. If this is not correct then
it could be simplfied as there would be no need for walk_alternates.

 Makefile          |    1 +
 git-alternates.sh |  159 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 160 insertions(+), 0 deletions(-)
 create mode 100755 git-alternates.sh
diff --git a/Makefile b/Makefile
index 3a6c6ea..1a7b084 100644
--- a/Makefile
+++ b/Makefile
@@ -334,6 +334,7 @@ TEST_PROGRAMS_NEED_X =
 unexport CDPATH

 SCRIPT_SH += git-am.sh
+SCRIPT_SH += git-alternates.sh
 SCRIPT_SH += git-bisect.sh
 SCRIPT_SH += git-difftool--helper.sh
 SCRIPT_SH += git-filter-branch.sh
diff --git a/git-alternates.sh b/git-alternates.sh
new file mode 100755
index 0000000..74ec707
--- /dev/null
+++ b/git-alternates.sh
@@ -0,0 +1,159 @@
+#!/bin/sh
+#
+# This file is licensed under the GPL v2
+#
+
+USAGE='[-r|--recursive] [-a|--add <dir>] [-f|--force -d|--delete <dir>]'
+
+. git-sh-setup
+
+#
+# Runs through the alternates file calling the callback function $1
+# with the name of the alternate as the first argument to the callback
+# any additional arguments are passed to the callback function.
+#
+walk_alternates()
+{
+    local alternates=$GIT_DIR/objects/info/alternates
+    local callback=$1
+    shift
+
+    if [ -e $alternates ]; then
+        while read line
+        do
+            $callback $line $*
+        done < $alternates
+    fi
+}
+
+#
+# Walk function to display one alternate object store and, if the user
+# has specified -r, recursively call show_alternates on the git
+# repository that the object store belongs to.
+#
+show_alternates_walk()
+{
+    say "Object store $1"
+    say "    referenced via $GIT_DIR"
+
+    local new_git_dir=${line%%/objects}
+    if [ "$recursive" == "true" -a "$GIT_DIR" != "$new_git_dir" ]
+    then
+        GIT_DIR=$new_git_dir show_alternates
+    fi
+}
+
+show_alternates()
+{
+    walk_alternates show_alternates_walk
+}
+
+#
+# Walk function to check that the specified alternate does not
+# already exist.
+#
+check_current_alternate_walk()
+{
+    if test "$1" = "$2"; then
+        die "fatal: Object store $2 is already used by $GIT_DIR"
+    fi
+}
+
+add_alternate()
+{
+    if test ! -d $dir; then
+        die "fatal: $dir is not a directory"
+    fi
+
+    walk_alternates check_current_alternate_walk $dir
+
+    # At this point we know that $dir is a directory that exists
+    # and that its not already being used as an alternate. We could
+    # go further and verify that $dir has valid objects.
+
+    # if we're still going we can safely add the alternate
+    touch $GIT_DIR/objects/info/alternates
+    echo "$(readlink -f $dir)" >> $GIT_DIR/objects/info/alternates
+    say "$dir added as an alternate"
+    say "     use 'git repack -adl' to remove duplicate objects"
+}
+
+rewrite_alternates()
+{
+    if test "$1" != "$2"; then
+        echo $2 >> $3
+    fi
+}
+
+del_alternate()
+{
+    if test ! $force = "true"; then
+        say "Not forced, use"
+        say "   'git repack -a' to fetch missing objects, then "
+        say "   '$dashless -f -d $dir' to remove the alternate"
+        die
+    fi
+
+    local alternates=$GIT_DIR/objects/info/alternates
+
+    new_alts_file=$(mktemp $alternates-XXXXXX)
+    touch $new_alts_file
+
+    walk_alternates rewrite_alternates $dir $new_alts_file
+    mv $new_alts_file $alternates
+
+    # save the git from repeatedly reading a 0 length file
+    if test $(stat -c "%s" $alternates) -eq 0; then
+        rm $alternates
+    fi
+}
+
+dir=""
+oper=""
+force="false"
+
+# Option parsing
+while test $# != 0
+do
+    case "$1" in
+        -r|--recursive)
+            recursive="true"
+            ;;
+        -a|--add)
+            if test ! -z "$oper"; then
+                usage
+            fi
+            oper="add"
+            case "$#,$1" in
+                1,*) usage ;;
+                *)   dir=$2; shift ;;
+            esac
+            ;;
+        -d|--delete)
+            if test ! -z "$oper"; then
+                usage
+            fi
+            oper="del"
+            case "$#,$1" in
+                1,*) usage ;;
+                *)   dir=$2; shift ;;
+            esac
+            ;;
+        -f|--force)
+            force="true"
+            ;;
+        -*)
+            usage
+            ;;
+        *)
+            ;;
+    esac
+    shift
+done
+
+# Now go and do it
+case $oper in
+    add) add_alternate ;;
+    del) del_alternate ;;
+    *)   show_alternates ;;
+esac
-- 
1.7.0.3

Re: Question about .git/objects/info/alternates

From: Jonathan Nieder <hidden>
Date: 2016-06-15 22:48:29

Chris Packham wrote:
It occurs to me that the UI around alternates is a bit lacking i.e.
there isn't a git command to display the alternates in use or to add
them to an existing repository (or at least I couldn't find one
skimming the docs or googling). So here's my attempt to add a 'git
alternates' command which can display, add or remove an alternate.
Quite welcome!

Something like this explanation probably belongs in the commit
message.  That way, years down the line, people don’t need to trawl
the list archives to see what your goal was.

Comments:
From a5c64de20937da132376d717f19a1d52b54701d2 Mon Sep 17 00:00:00 2001
From: Chris Packham <redacted>
Date: Wed, 24 Mar 2010 11:34:11 -0700
Redundant next to the email header.  It is useful to be able to include
fields that differ from the e-mail header (often Subject:,
sometimes Date:, sometimes From:), but aside from that, this metadata
should be omitted when sending patches to the git mailing list.
+#
+# Runs through the alternates file calling the callback function $1
+# with the name of the alternate as the first argument to the callback
+# any additional arguments are passed to the callback function.
+#
+walk_alternates()
+{
+    local alternates=$GIT_DIR/objects/info/alternates
+    local callback=$1
I couldn’t find any other uses of “local” in-tree.  I assume old shells
don’t support it.

Will this be a problem for recursion?  Maybe the callback should be
called in a subshell.
+    shift
+
+    if [ -e $alternates ]; then
+        while read line
+        do
+            $callback $line $*
Probably "$line" "$@" instead of $line $* would be more flexible.
+#
+# Walk function to display one alternate object store and, if the user
+# has specified -r, recursively call show_alternates on the git
+# repository that the object store belongs to.
+#
+show_alternates_walk()
+{
+    say "Object store $1"
+    say "    referenced via $GIT_DIR"
+
+    local new_git_dir=${line%%/objects}
use of local.
+    if [ "$recursive" == "true" -a "$GIT_DIR" != "$new_git_dir" ]
Should use = instead of == (portability).  Also git scripts tend to
spell out 'test' and avoid the -a and -o operators:

	if test "$recursive" = true && test "$GIT_DIR" != "$new_git-dir"

though that is not a hard and fast rule.
+add_alternate()
+{
[...]
+    touch $GIT_DIR/objects/info/alternates
Necessary?
+    echo "$(readlink -f $dir)" >> $GIT_DIR/objects/info/alternates
Maybe

	readlink -f "$dir" >> $GIT_DIR/objects/info/alternates

would be simpler.
+rewrite_alternates()
+{
+    if test "$1" != "$2"; then
+        echo $2 >> $3
+    fi
+}
What does this function do?  (Could use a comment.)
+del_alternate()
+{
[...]
+    local alternates=$GIT_DIR/objects/info/alternates
use of local.
+
+    new_alts_file=$(mktemp $alternates-XXXXXX)
Not used elsewhere in git.  Is this needed?  Maybe a single
$GIT_DIR/objects/info/new-alternates.tmp or similar would be good
enough.
+    # save the git from repeatedly reading a 0 length file
+    if test $(stat -c "%s" $alternates) -eq 0; then
Not used elsewhere in core git.  test -s can help.
+# Option parsing
See OPTIONS_SPEC in git-repack.sh for an example of how to simplify
this.
+# Now go and do it
+case $oper in
+    add) add_alternate ;;
+    del) del_alternate ;;
+    *)   show_alternates ;;
+esac
Thank you for your excellent work!  Looks very useful.

Regards,
Jonathan

Re: Question about .git/objects/info/alternates

From: Stephen Boyd <hidden>
Date: 2016-06-15 22:48:29

On Wed, Mar 24, 2010 at 11:53 AM, Chris Packham [off-list ref] wrote:
+USAGE='[-r|--recursive] [-a|--add <dir>] [-f|--force -d|--delete <dir>]'
[...]
+case $oper in
+    add) add_alternate ;;
+    del) del_alternate ;;
+    *)   show_alternates ;;
+esac
From a very high-level this should probably be more like git remote
and git notes. 'add' and 'delete' would be subcommands instead of
options. Plus you might have an explicit subcommand for show (or
list?). Something like

git alternates [show]  [-r|--recursive]
git alternates add <dir>
git alternates delete  [-f|--force] <dir>

Re: Question about .git/objects/info/alternates

From: Chris Packham <hidden>
Date: 2016-06-15 22:48:29

On Wed, Mar 24, 2010 at 1:16 PM, Stephen Boyd [off-list ref] wrote:
On Wed, Mar 24, 2010 at 11:53 AM, Chris Packham [off-list ref] wrote:
quoted
+USAGE='[-r|--recursive] [-a|--add <dir>] [-f|--force -d|--delete <dir>]'
[...]
quoted
+case $oper in
+    add) add_alternate ;;
+    del) del_alternate ;;
+    *)   show_alternates ;;
+esac
From a very high-level this should probably be more like git remote
and git notes. 'add' and 'delete' would be subcommands instead of
options. Plus you might have an explicit subcommand for show (or
list?). Something like

git alternates [show]  [-r|--recursive]
git alternates add <dir>
git alternates delete  [-f|--force] <dir>
I like that suggestion. I'd have to figure out the option parsing but
should be doable. This is of course assuming it remains its own
command set and isn't rolled into something else.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help