This is a follow-up to the conversation at #git-devel Standup today.
We encountered the following situation. Our normal clone practice is to use
SSH URIs. So, the clone is straightforward with the main (8 year old) repo
of the style:
git@bitbucket.org:project/repo.git
with submodules referenced as:
git@bitbucket.org:project/module.git
When in SSH mode, clones are simple with --recurse-submodules doing what we
want. However, we had to clone on a system where SSH was locked down and we
could only use HTTPS. The form of the URIs changed rather radically:
https://user@bitbucket.org/project/repo.git
which caused the submodule clone not to work. Fortunately, the number of
submodules was low, so a manual edit of .gitmodules (with update-index
--assume-unchanged) and .git/modules and .git/config was not onerous.
However, we should be able to redirect the configuration without having to
do that. Our directory references are all relative to the main repo, so
those did not have to change. We still use SSH for most work with the repo
but needed to get source onto a test system for debugging.
I would like to put together a best practice section in
Documentation/gitsubmodules.txt on the subject, but really don't think I did
this *temporary* change the most effective way. I'm looking for better
practices than I used - I'm sure there is at least one. There may be code
changes involved, particularly since .gitmodules is tracked and contains the
full URI (maybe it should not). One option we also explored was explicitly
specifying the URI for each submodule using set-url, which works but it
seems like there should be something in clone that redirects the part of the
URI relative to the main repo - although that would only work in the special
case where the submodules have the same host and project as the main repo.
-Randall
From: Robert Coup <hidden> Date: 2021-12-06 23:54:00
Hi Randall,
On Mon, 6 Dec 2021 at 23:10, [off-list ref] wrote:
git@bitbucket.org:project/module.git
When in SSH mode, clones are simple with --recurse-submodules doing what we
want. However, we had to clone on a system where SSH was locked down and we
could only use HTTPS. The form of the URIs changed rather radically:
https://user@bitbucket.org/project/repo.git
I'm looking for better practices than I used - I'm sure there is at least one.
AFAIK the existing `url.<base>.insteadOf` config option[1] deals with this...
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-urlltbasegtinsteadOf
$ git config --global url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
If you don't want to set it globally (not a throwaway CI environment?)
then you can do it as a one-off:
$ git -c url."https://user@bitbucket.org/".insteadOf="git@bitbucket.org:"
clone --recurse-submodules git@bitbucket.org:repo/project.git
But it isn't persisted into your repo config then, so subsequent
fetches won't work. You'd need to persist it using something like:
$ git config url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
$ git submodule foreach --recursive 'git config
url."https://user@bitbucket.org/".insteadOf "git@bitbucket.org:" '
Maybe there's an opportunity to make that part easier?
Rob :)
On Mon, 6 Dec 2021 at 23:10, [off-list ref] wrote:
quoted
git@bitbucket.org:project/module.git
When in SSH mode, clones are simple with --recurse-submodules doing
what we want. However, we had to clone on a system where SSH was
locked down and we could only use HTTPS. The form of the URIs changed
AFAIK the existing `url.<base>.insteadOf` config option[1] deals with this...
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-
urlltbasegtinsteadOf
$ git config --global url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
If you don't want to set it globally (not a throwaway CI environment?) then
you can do it as a one-off:
$ git -c url."https://user@bitbucket.org/".insteadOf="git@bitbucket.org:"
clone --recurse-submodules git@bitbucket.org:repo/project.git
But it isn't persisted into your repo config then, so subsequent fetches won't
work. You'd need to persist it using something like:
$ git config url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
$ git submodule foreach --recursive 'git config
url."https://user@bitbucket.org/".insteadOf "git@bitbucket.org:" '
Maybe there's an opportunity to make that part easier?
Well, I gave this a shot. The mapping did not appear to work - I tried a few combinations. I was left with the original URIs. Note that I also had to do
$ git submodule init
which reported the wrong URIs
$ git submodule update
which obviously failed since they were still using SSH.
I think there may be something not working correctly with the insteadOf operator. This is easily reproduceable.
-Randall
On 07.12.2021 09:50, rsbecker@nexbridge.com wrote:
On December 6, 2021 6:54 PM, Robert Coup wrote:
quoted
On Mon, 6 Dec 2021 at 23:10, [off-list ref] wrote:
quoted
git@bitbucket.org:project/module.git
When in SSH mode, clones are simple with --recurse-submodules doing
what we want. However, we had to clone on a system where SSH was
locked down and we could only use HTTPS. The form of the URIs changed
AFAIK the existing `url.<base>.insteadOf` config option[1] deals with this...
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-
urlltbasegtinsteadOf
$ git config --global url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
If you don't want to set it globally (not a throwaway CI environment?) then
you can do it as a one-off:
$ git -c url."https://user@bitbucket.org/".insteadOf="git@bitbucket.org:"
clone --recurse-submodules git@bitbucket.org:repo/project.git
But it isn't persisted into your repo config then, so subsequent fetches won't
work. You'd need to persist it using something like:
$ git config url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
$ git submodule foreach --recursive 'git config
url."https://user@bitbucket.org/".insteadOf "git@bitbucket.org:" '
Maybe there's an opportunity to make that part easier?
Well, I gave this a shot. The mapping did not appear to work - I tried a few combinations. I was left with the original URIs. Note that I also had to do
$ git submodule init
which reported the wrong URIs
$ git submodule update
which obviously failed since they were still using SSH.
I think there may be something not working correctly with the insteadOf operator. This is easily reproduceable.
On December 8, 2021 7:52 AM, Fabian Stelzer wrote:
On 07.12.2021 09:50, rsbecker@nexbridge.com wrote:
quoted
On December 6, 2021 6:54 PM, Robert Coup wrote:
quoted
On Mon, 6 Dec 2021 at 23:10, [off-list ref] wrote:
quoted
git@bitbucket.org:project/module.git
When in SSH mode, clones are simple with --recurse-submodules doing
what we want. However, we had to clone on a system where SSH was
locked down and we could only use HTTPS. The form of the URIs
changed
clone --recurse-submodules git@bitbucket.org:repo/project.git
But it isn't persisted into your repo config then, so subsequent
fetches won't work. You'd need to persist it using something like:
$ git config url."https://user@bitbucket.org/".insteadOf
"git@bitbucket.org:"
$ git submodule foreach --recursive 'git config
url."https://user@bitbucket.org/".insteadOf "git@bitbucket.org:" '
Maybe there's an opportunity to make that part easier?
Well, I gave this a shot. The mapping did not appear to work - I tried
a few combinations. I was left with the original URIs. Note that I also
had to do
$ git submodule init
which reported the wrong URIs
$ git submodule update
which obviously failed since they were still using SSH.
I think there may be something not working correctly with the insteadOf
operator. This is easily reproduceable.
quoted
I ran into this as well some time ago. For submodules this only works if you
put it into your global config since the parent projects config is not relevant to
the submodule.
There's some discussion in the archive about this as well:
https://lore.kernel.org/git/404d109f-e5a7-85a3-e64c-
ab1b21c3045d@durchholz.org/
The minimal procedure that I got to work is:
$ git config --global url."https://user@bitbucket.org/".insteadOf "git@bitbucket.org:"
$ git clone --recurse-submodules https://user@bitbucket.org/project/repo.git
This leaves .gitmodules untouched and contains the original URIs, so the status is clean and there is no real risk of someone pushing an unwanted URI change back to the upstream repo. I don't see a major downside to leaving the redirect in global as this would be useful for subsequent clones. Any scripting that might depend on .gitmodules being correct would break - not in my case though - to that should probably be noted.
I think I have enough to document this consideration in Documentation/gitsubmodules.txt. Thanks everyone.
-Randall