From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
This patch series was inspired by several issues dealing with
multiple remotes and submodules. The tests added as the last
patch demonstrate the specific use pattern to be supported,
and that does not work currently.
1 - user A creates a project on server "frotz" with a submodule.
2 - user B clones the project, forks it, adds a submodule, and
publishes the fork on server "fork"
user A adds the fork to his working tree, via git remote add +
git fetch. The desire is that:
1 - git submodule init + git submodule update include the new
submdule added in the fork.
2 - The top-level project and all submodules use the same
remote nickname to refer to the same server.
Breakage in 1.5.4 without this patch series is in several places:
1 - git submodule does not follow the top-level project's
branch.<name>.remote in fetching, so does not try to clone
the new submodule from remoe "fork" where the fork exists.
2 - If 1 were fixed, git submodule still invokes clone without "-o
<remotenick>" when cloning, so each submodule would have "origin"
point to a different remote.
3 - If 2 were fixed, submodule update leaves submodules on detached
heads, so the remotes machinery cannot supply a remote name, and thus
"origin" is used for further updates. As origin may not be defined,
(e.g., for the submodule fetched from "fork" this also fails.)
The underlying design issue is the assumption that all projects
revolve around a single server, so that *the* server can be called
"origin" without confusion. Given a project with multiple servers and
no one server being *the* server, origin cannot be usefully defined.
Absent submodules, the remotes machinery largely masks this: origin is
referred to only as the default remote and it is easy to avoid using
the default remote. With submodules, this becomes more of an issue.
Following this patch series, the following is true:
1) The existing workflow of top-level and all sub-modules always
cloned from and interacting with a single default remote
nick-named "origin" is unchanged.
2) Via clone -o <nick> (or git confit core.origin <nick>), the
default remote is changed from origin to <nick>, and that default
is used instead of origin everywhere.
3) git-submodule uses the top-level's branch.<name>.remote to
choose which remote to fetch from, uses that in all submodules,
including defining that in existing submodules whose url is
relative (../* or ./*) indicating the submodule is maintained on
same server as top-level.
The penultimate patch addresses a basic usability issue, allowing
addition of a submodule in place in a project, rather than requiring
it be cloned from a remote. This hole in the porcelain became evident
in creating a test (t7401) for this series.
submodules except by cloning).
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
This records the users choice of default remote name (by default "origin")
as given by the -o option.
Signed-off-by: Mark Levedahl <redacted>
---
Documentation/git-clone.txt | 3 ++-
git-clone.sh | 1 +
2 files changed, 3 insertions(+), 1 deletions(-)
@@ -103,7 +103,8 @@ OPTIONS --origin <name>:: -o <name>:: Instead of using the remote name 'origin' to keep track- of the upstream repository, use <name> instead.+ of the upstream repository, use <name> instead. The name+ is recorded in the core.origin config variable. --upload-pack <upload-pack>:: -u <upload-pack>::
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
This adds a sequence of tests to assure that the following two sequences
work:
git clone -o frotz <someurl> foo
cd foo
git submodule init
git submodule update
This should result in the master and subproject having "frotz" as the
name of the default remote (and origin undefined).
Then, in the same working directory
git remote add fork <some url>
git fetch fork
git checkout --track -b fork fork/<somebranch>
git submodule init
git submodule update
will retrive new submodules from remote "fork", and define fork in the
existing modules. Origin remains undefined.
Note: this latter case is a clear motivation for overriding "origin": after
the second test, the various submodules would have different ideas of
remote "origin": these would point to different servers. This would
entirely prevent the top-level branch.<name>.remote machinery from
controlling the project as there is no uniform naming of remotes.
Signed-off-by: Mark Levedahl <redacted>
---
t/t7401-submodule-remote.sh | 104 +++++++++++++++++++++++++++++++++++++++++++
1 files changed, 104 insertions(+), 0 deletions(-)
create mode 100755 t/t7401-submodule-remote.sh
@@ -0,0 +1,104 @@+#!/bin/sh++test_description='Porcelainsupportforsubmoduleswithmultipleremotes++Thistestverifiesoperationofsubmodulesusingmultipleremotesand+differingremotepertop-levelbranch.Thisincludesabilitytonamethe+defaultsomethingotherthanorigin,tofollowthetop-level+remote.<branch>,andtopropagatedefinitionofnewremotesdownto+submodulesasneeded.+'++../test-lib.sh++# the standard tests all work with one repo, but we need several..+rm-rf.git++test_expect_success'Prepare master repository with 1 submodule''+(+mkdirmaster&&+cdmaster&&+gitinit&&+echo"on master">master.txt&&+gitaddmaster.txt&&+gitcommit-m"Add master.txt"&&+mkdirsubmod1&&+cdsubmod1&&+gitinit&&+echo"submod1">submod1.txt&&+gitaddsubmod1.txt&&+gitcommit-m"Added submod1.txt"&&+cd..&&+gitsubmoduleadd./submod1submod1&&+gitcommit-m"Added submodule submod1"+)+'++test_expect_success'Clone master as fork''+(+gitclonemasterfork&&+cdfork&&+test"$(gitremote)"="origin"&&+gitsubmoduleinit&&+gitsubmoduleupdate&&+test-esubmod1/.git+)+'++test_expect_success'Add second submodule in fork''+(+cdfork&&+mkdirsubmod2&&+cdsubmod2&&+gitinit&&+echo"submod2">submod2.txt&&+gitaddsubmod2.txt&&+gitcommit-m"Added submod2.txt"&&+cd..&&+gitsubmoduleadd./submod2submod2&&+gitcommit-m"Added submodule submod2 on fork"+)+'++test_expect_success'Clone master using frotz instead of origin''+(+gitclone-ofrotzmasterworker&&+cdworker&&+test"$(gitremote)"="frotz"+)+'++test_expect_success'Get submodules using frotz instead of origin''+(+cdworker&&+gitsubmoduleinit&&+gitsubmoduleupdate&&+test-esubmod1/.git&&+cdsubmod1&&+test"$(gitremote)"="frotz"+)+'++test_expect_success'Update using fork to get additional submodule''+(+cdworker&&+gitremoteaddfork$(pwd)/../fork&&+gitfetchfork&&+gitcheckout--track-bfork_masterfork/master&&+gitsubmoduleinit&&+gitsubmoduleupdate&&+test-esubmod2/.git&&+cdsubmod2&&+test"$(gitremote)"="fork"&&+cd../submod1&&+remotes1=$(gitremote)&&+case$remotes1in+fork*frotz|frotz*fork)+true;;+*)+false;;+esac+)+'++test_done
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
This introduces a new configuration variable, core.origin, that
defines the name of the default remote to be used. Traditionally, this
is "origin", and could be overridden for a given branch. This change
introduces a way to redefine the default as desired and have that honored
regardless of the currently checked out head (e.g., core.origin is
used when on a detached head or any other non-tracking branch).
Signed-off-by: Mark Levedahl <redacted>
---
Documentation/config.txt | 6 ++++++
git-parse-remote.sh | 5 +++--
remote.c | 11 ++++++++++-
3 files changed, 19 insertions(+), 3 deletions(-)
@@ -291,6 +291,12 @@ core.editor:: `GIT_EDITOR` environment, `core.editor`, `VISUAL` and `EDITOR` environment variables and then finally `vi`.+core.origin::+ The name of the remote used by default for fetch / pull. If unset,+ origin is assumed. This value is used whenever the current branch+ has no corresponding branch.<name>.remote, such as when working on+ a detached head.+ core.pager:: The command that git will use to paginate output. Can be overridden with the `GIT_PAGER` environment variable.
@@ -291,7 +297,6 @@ static void read_config(void)intflag;if(default_remote_name)// did this alreadyreturn;-default_remote_name=xstrdup("origin");current_branch=NULL;head_ref=resolve_ref("HEAD",sha1,0,&flag);if(head_ref&&(flag&REF_ISSYMREF)&&
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
Modules that are defined using relative urls to the master project are
assumed to be completely owned by the project. When running
"submodule update" from the top level, it is reasonable that the entire
project exists at the remote given by top-level branch.<name>.remote.
Using the remote machinery, this remote can be different for each
branch and can be different than the current defaults in each submodule.
This teaches submodule to:
1) Possibly define the current master's remote in each submodule, using
the same relative url used by submodule init.
2) Fetch each submodule's updates from the master's remote.
Submodules defined using absolute urls (not relative to the parent) are
not touched by this logic. Such modules are assumed to be independent
of the master project so submodule can do no better than to fetch from
their currently defined default remotes as already done.
Signed-off-by: Mark Levedahl <redacted>
---
git-submodule.sh | 20 +++++++++++++++++++-
1 files changed, 19 insertions(+), 1 deletions(-)
@@ -255,6 +255,8 @@ cmd_init()## Update each submodule path to correct revision, using clone and checkout as needed+# For owned submodules (defined using relative url), we use master project's remote+# and define that in each submodule if not already there## $@ = requested paths (default to all)#
@@ -306,9 +309,24 @@ cmd_update()die"Unable to find current revision in submodule path '$path'"fi+baseurl="$(GIT_CONFIG=.gitmodulesgitconfigsubmodule."$name".url)"+case"$baseurl"in+./*|../*)+fetch_remote=$master_remote+(unsetGIT_DIR;cd"$path"&&gitconfigremote."$fetch_remote".url>nul)||+(+absurl="$(resolve_relative_url$baseurl)"+unsetGIT_DIR;cd"$path"&&gitremoteadd"$master_remote""$absurl"+)||die"Unable to define remote '$fetch_remote' in submodule path '$path'"+;;+*)+fetch_remote=+;;+esac+iftest"$subsha1"!="$sha1"then-(unsetGIT_DIR;cd"$path"&&git-fetch&&+(unsetGIT_DIR;cd"$path"&&git-fetch$fetch_remote&&git-checkout-q"$sha1")||die"Unable to checkout '$sha1' in submodule path '$path'"
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
For submodules defined relative to their parent, it is likely that the
parent's defined default remote is correct for the child as well. This
allows use of remote names other than "origin", important as managed
submodules are typically checked out on a detached head and therefore
submodule-update invokes git-fetch using the default remote. Without this
change, submodules effectively had to have a default remote of "origin."
Signed-off-by: Mark Levedahl <redacted>
---
Documentation/git-submodule.txt | 8 +++++---
git-submodule.sh | 20 ++++++++++++++------
2 files changed, 19 insertions(+), 9 deletions(-)
@@ -37,9 +37,11 @@ status:: init:: Initialize the submodules, i.e. register in .git/config each submodule- name and url found in .gitmodules. The key used in .git/config is- `submodule.$name.url`. This command does not alter existing information- in .git/config.+ name and url found in .gitmodules, along with the default remote origin.+ For submodules using a relative url, the default remote is inherited+ from the parent project, for absolute urls the default "origin" is used.+ The key used in .git/config is submodule.$name.url`. This command does+ not alter existing information in .git/config. update:: Update the registered submodules, i.e. clone missing submodules and
@@ -40,9 +41,7 @@ get_repo_base() {# Resolve relative url by appending to parent's url resolve_relative_url(){-branch="$(gitsymbolic-refHEAD2>/dev/null)"-remote="$(gitconfigbranch.${branch#refs/heads/}.remote)"-remote="${remote:-origin}"+remote="$(get_default_remote)"remoteurl="$(gitconfigremote.$remote.url)"||die"remote ($remote) does not have a url in .git/config"url="$1"
@@ -92,6 +91,7 @@ module_clone(){path=$1url=$2+origin=$3# If there already is a directory at the submodule path,# expect it to be empty (since that is the default checkout
@@ -107,7 +107,7 @@ module_clone()test-e"$path"&&die"A file already exist at path '$path'"-git-clone-n"$url""$path"||+git-clone-n-o"$origin""$url""$path"||die"Clone of '$url' into submodule path '$path' failed"}
@@ -156,10 +156,12 @@ cmd_add()case"$repo"in./*|../*)# dereference source url relative to parent's url+origin=$(get_default_remote)realrepo="$(resolve_relative_url$repo)";;*)# Turn the source into an absolute path if# it is local+origin=originifbase=$(get_repo_base"$repo");thenrepo="$base"fi
@@ -180,7 +182,7 @@ cmd_add()gitls-files--error-unmatch"$path">/dev/null2>&1&&die"'$path' already exists in the index"-module_clone"$path""$realrepo"||exit+module_clone"$path""$realrepo""$origin"||exit(unsetGIT_DIR;cd"$path"&&gitcheckout-q${branch:+-b "$branch""origin/$branch"})||die"Unable to checkout submodule '$path'"gitadd"$path"||
@@ -236,9 +238,14 @@ cmd_init()case"$url"in./*|../*)url="$(resolve_relative_url"$url")"+origin=$(get_default_remote)+;;+*)+origin=origin;;esac+gitconfigsubmodule."$name".origin"$origin"&&gitconfigsubmodule."$name".url"$url"||die"Failed to register url for submodule path '$path'"
@@ -287,10 +294,11 @@ cmd_update()say"Submodule path '$path' not initialized"continuefi+origin=$(gitconfigsubmodule."$name".origin||echoorigin)if!test-d"$path"/.gitthen-module_clone"$path""$url"||exit+module_clone"$path""$url""$origin"||exitsubsha1=elsesubsha1=$(unsetGIT_DIR;cd"$path"&&
From: Mark Levedahl <hidden> Date: 2016-06-15 22:44:09
When working in top-level directory, it is natural to create a new
submodule in a subdirectory, then add that submodule to top-level in
place. This allows "git submodule add <intended url> subdir" to add
the existing subdir to the current project. The presumption is the user
will later push / clone the subdir to the <intended url> so that future
submodule init / updates will work.
Absent this patch, "git submodule add" insists upon cloning the subdir
from the remote, which is fine for adding an existing project in but
less useful when adding a new submodule from scratch to an existing
project. This functionality remains, and the clone is attempted if the
subdir does not already exist as a valid git repo.
Signed-off-by: Mark Levedahl <redacted>
fix submodule again
---
Documentation/git-submodule.txt | 5 ++-
git-submodule.sh | 56 +++++++++++++++++++++++---------------
2 files changed, 37 insertions(+), 24 deletions(-)
@@ -18,8 +18,9 @@ COMMANDS -------- add:: Add the given repository as a submodule at the given path- to the changeset to be committed next. In particular, the- repository is cloned at the specified path, added to the+ to the changeset to be committed next. If path is a valid+ repository within the project, it is added as is. Otherwise,+ repository is cloned at the specified path. path is added to the changeset and registered in .gitmodules. If no path is specified, the path is deduced from the repository specification. If the repository url begins with ./ or ../, it is stored as
@@ -44,6 +44,7 @@ resolve_relative_url ()remote="$(get_default_remote)"remoteurl="$(gitconfigremote.$remote.url)"||die"remote ($remote) does not have a url in .git/config"+remoteurl="${remoteurl%/.git}"url="$1"whiletest-n"$url"do
@@ -153,22 +154,6 @@ cmd_add()usagefi-case"$repo"in-./*|../*)-# dereference source url relative to parent's url-origin=$(get_default_remote)-realrepo="$(resolve_relative_url$repo)";;-*)-# Turn the source into an absolute path if-# it is local-origin=origin-ifbase=$(get_repo_base"$repo");then-repo="$base"-fi-realrepo=$repo-;;-esac-# Guess path from repo if not specified or strip trailing slashesiftest-z"$path";thenpath=$(echo"$repo"|sed-e's|/*$||'-e's|:*/*\.git$||'-e's|.*[/:]||g')
@@ -176,15 +161,42 @@ cmd_add()path=$(echo"$path"|sed-e's|/*$||')fi-test-e"$path"&&-die"'$path' already exists"-gitls-files--error-unmatch"$path">/dev/null2>&1&&die"'$path' already exists in the index"-module_clone"$path""$realrepo""$origin"||exit-(unsetGIT_DIR;cd"$path"&&gitcheckout-q${branch:+-b "$branch""origin/$branch"})||-die"Unable to checkout submodule '$path'"+# perhaps the path exists and is already a git repo, else clone it+iftest-e"$path"+then+iftest-d"$path/.git"&&+test"$(unsetGIT_DIR;cd$path;gitrev-parse--git-dir)"=".git"+then+echo"Adding existing repo at '$path' to the index"+else+die"'$path' already exists and is not a valid git repo"+fi+else+case"$repo"in+./*|../*)+# dereference source url relative to parent's url+origin=$(get_default_remote)+realrepo="$(resolve_relative_url$repo)";;+*)+# Turn the source into an absolute path if+# it is local+origin=origin+ifbase=$(get_repo_base"$repo")+then+repo="$base"+fi+realrepo=$repo+;;+esac++module_clone"$path""$realrepo""$origin"||exit+(unsetGIT_DIR;cd"$path"&&gitcheckout-q${branch:+-b "$branch""origin/$branch"})||+die"Unable to checkout submodule '$path'"+fi+gitadd"$path"||die"Failed to add submodule '$path'"