Stefan Beller [off-list ref] writes:
quoted
I may be missing the subtleties, but if you are serving others from
a non-bare repository with submodules, I do not think you would want
to expose the in-tree version of the submodule in the first place.
Well I would imagine that is the exact point.
If I was not trying to expose my state, I could ask you to
obtain your copy from $(git remote get-url origin) just as I did.
That wasn't what I had in mind, but if the cloner cloned from your
repository with a working tree, the cloner would discover submodules
you use from your .gitmodules file, which would record the location
you cloned them from, so something like that may come into the
picture. What I had in mind was more like this one you mentioned
below:
$GIT_DIR_SUPER_PROJECT/modules/$MODULE_NAME
...
Right instead of cloning $WORKTREE/sub/.git you rather want
$GITDIR/module/sub
So currently the protocol doesn't allow to even specify the submodules
directories.
Depends on what you exactly mean by "the protocol", but the
networking protocol is about accessing a single repository. It is
up to you to decide where to go next after learning what you can
learn from the result, typically by following what appears in
the .gitmodules file.
The only special case is when .gitmodules file records the URL in a
relative form, I would think. Traditionally (i.e. when it was
considered sane to clone only from bare repositories) I think people
expected a layout like this:
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
top.git/sub.git/
top.git/sub.git/refs/{heads,tags,...}/...
top.git/sub.git/objects/...
and refer to ./sub.git from .gitmodules recorded in top.git. It
still would be norm for common distribution sites (i.e. the original
place Yaroslav likely has cloned things from) to be bare, and with
or without $GIT_DIR/modules/, the relative path of submodule seen
by its superproject would (have to) be different between a bare and
a non-bare repository.
I'd imagine that people could agree on a common layout like this
even for a forest of bare repositories:
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
top.git/modules/sub.git/
top.git/modules/sub.git/refs/{heads,tags,...}/...
top.git/modules/sub.git/objects/...
which would probably make the "relative" relationship between the
supermodule and its submodules the same between bare and non-bare
repositories, but I didn't think it too deeply.
On Wed, Apr 20, 2016 at 2:27 PM, Junio C Hamano [off-list ref] wrote:
Stefan Beller [off-list ref] writes:
quoted
quoted
I may be missing the subtleties, but if you are serving others from
a non-bare repository with submodules, I do not think you would want
to expose the in-tree version of the submodule in the first place.
Well I would imagine that is the exact point.
If I was not trying to expose my state, I could ask you to
obtain your copy from $(git remote get-url origin) just as I did.
That wasn't what I had in mind, but if the cloner cloned from your
repository with a working tree, the cloner would discover submodules
you use from your .gitmodules file, which would record the location
you cloned them from, so something like that may come into the
picture. What I had in mind was more like this one you mentioned
below:
quoted
$GIT_DIR_SUPER_PROJECT/modules/$MODULE_NAME
...
Right instead of cloning $WORKTREE/sub/.git you rather want
$GITDIR/module/sub
quoted
So currently the protocol doesn't allow to even specify the submodules
directories.
Depends on what you exactly mean by "the protocol", but the
networking protocol is about accessing a single repository. It is
up to you to decide where to go next after learning what you can
learn from the result, typically by following what appears in
the .gitmodules file.
Right. But the .gitmodules file is not sufficient.
If I clone from a bare hosting location, the .gitmodules file
is the best we can do and the .gitmodules file works as intended.
But in the non bare I case I think we would want to get the submodule
from that location as well.
So in git clone (which calls out to git submodule update, which uses
submodule--helper update_clone for cloning submodules) we'd want to see
if remote was bare:
do as usual (obtain URL from .gitmodules file)
else
take URL=$NON_BARE_REMOTE/module/submodule
The only special case is when .gitmodules file records the URL in a
relative form, I would think. Traditionally (i.e. when it was
considered sane to clone only from bare repositories) I think people
expected a layout like this:
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
top.git/sub.git/
top.git/sub.git/refs/{heads,tags,...}/...
top.git/sub.git/objects/...
which could also be referred to as
top
without the .git suffix as someone thought this was an optimization?
Relative paths for submodules I have seen so far (on github,
googlesource, eclipse)
start with a ../ such that we'd have
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
sub.git/
sub.git/refs/{heads,tags,...}/...
sub.git/objects/...
and the .git suffix omission works as we only need to check for the last
for characters and not somewhere in between. The sub.git is a standalone
repository, and you cannot tell it is a submodule (except by its contents)
and refer to ./sub.git from .gitmodules recorded in top.git. It
still would be norm for common distribution sites (i.e. the original
place Yaroslav likely has cloned things from) to be bare, and with
or without $GIT_DIR/modules/, the relative path of submodule seen
by its superproject would (have to) be different between a bare and
a non-bare repository.
I think on a hosting site they could even coexist when having the
layout as above.
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
sub.git/
sub.git/refs/{heads,tags,...}/...
sub.git/objects/...
# the following only exist in non bare:
top.git/modules/sub.git/
top.git/modules/sub.git/refs/{heads,tags,...}/...
top.git/modules/sub.git/objects/...
The later files would be more reflective of what you *really*
want if you clone from top.git.
Traditionally (when cloning was done from bare repos only),
the .gitmodules file provides a very good way to indicate what
the intent of the superproject is as the recorded sha1 in the tree
doesn't tell you anything and tracking the remote for the submodule
out of tree is cumbersome, so an in tree solution makes perfect sense.
If we have a non bare repo, it is safe to assume that the cloner actually
meant to get the whole state from the remote (including submodules)?
I am trying to think of reasons why you would not want to get that copy
from the remote. One (weak) reason is that the submodule may be a
well known library, which you can obtain faster from a well known git
hosting site rather than $remote.
I'd imagine that people could agree on a common layout like this
even for a forest of bare repositories:
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
top.git/modules/sub.git/
top.git/modules/sub.git/refs/{heads,tags,...}/...
top.git/modules/sub.git/objects/...
which would probably make the "relative" relationship between the
supermodule and its submodules the same between bare and non-bare
repositories, but I didn't think it too deeply.
Forrests as of now are handled as a flat level thing, e.g.
git clone git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git
will produce a superproject with 25 submodules, all of them
are either at ../ or at ../../ such that it would follow
projects/top.git/
projects/top.git/refs/{heads,tags,...}/...
projects/top.git/objects/...
projects/sub.git/
projects/sub.git/refs/{heads,tags,...}/...
projects/sub.git/objects/...
libs/sub2.git
libs/sub2.git/refs/{heads,tags,...}/...
libs/sub2.git/objects/...
Looking at our internal code search there is no .gitmodules file
whose url starts with "./", they all start with ../ or are absolute.
NB Thank you for the lively discussion!
On Wed, 20 Apr 2016, Stefan Beller wrote:
quoted
quoted
So currently the protocol doesn't allow to even specify the submodules
directories.
quoted
Depends on what you exactly mean by "the protocol", but the
networking protocol is about accessing a single repository. It is
up to you to decide where to go next after learning what you can
learn from the result, typically by following what appears in
the .gitmodules file.
Right. But the .gitmodules file is not sufficient.
why?
quoted
...<
I think on a hosting site they could even coexist when having the
layout as above.
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
sub.git/
sub.git/refs/{heads,tags,...}/...
sub.git/objects/...
# the following only exist in non bare:
top.git/modules/sub.git/
top.git/modules/sub.git/refs/{heads,tags,...}/...
top.git/modules/sub.git/objects/...
The later files would be more reflective of what you *really*
want if you clone from top.git.
may be there is no need for assumptions and .gitmodules should be
sufficient?
- absolute url in .gitmodules provides absolute URL/path to the
submodule of interest, regardless either submodule is present in
originating repository as updated submodule. Either cloning it
instead of original repository would be more efficient is already a
heuristic which might fail miserably (may be I have a faster
connection to the original repository pointed by the absolute
url than to this particular repository)
- relative url in .gitmodules provides relative location to the location
of the "top" repository, and that is only when that submodule "absolute"
url should be resolved relative to the one of the "top" repository
NB I will consider it a separate issue either relative paths
without '../' prefix are having any sense in bare repositories.
or have I missed the point?
--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
On Wed, Apr 20, 2016 at 8:14 PM, Yaroslav Halchenko [off-list ref] wrote:
NB Thank you for the lively discussion!
On Wed, 20 Apr 2016, Stefan Beller wrote:
quoted
quoted
quoted
So currently the protocol doesn't allow to even specify the submodules
directories.
quoted
quoted
Depends on what you exactly mean by "the protocol", but the
networking protocol is about accessing a single repository. It is
up to you to decide where to go next after learning what you can
learn from the result, typically by following what appears in
the .gitmodules file.
quoted
Right. But the .gitmodules file is not sufficient.
why?
What do you expect from cloning a repo with submodules?
In case of a bare repo:
Get the repo from the specified remote and get the submodules
from "somewhere" (and .gitmodules helps you guessing where
"somewhere" is).
This has been the traditional way, and the .gitmodules file
is just a helper for a best guess where to get a submodule sha1
from. (The repo pointed at from the .gitmodules file may not exist
any more; or it may have forgot the wanted commit)
In case of non bare:
Get the repo and all its submodules from the specified remote.
(As the submodule is right there, that's the best guess to get it from,
no need to get it from somewhere else. The submodule at the remote
is the closest match you can get for replicating the superproject with
its submodules.)
This way is heavy underutilized as it wasn't exercised as often I would
guess, so the "wrong" default (to obtain the submodule information from
.gitmodules instead of from the remote directly) was not pointed out before.
Now that the client wants to make a decision where to get the
submodules from, based on the bare-ness of the remote, it may
require changes in the wire protocol, such that the remote simply
advertises it is a (non-)bare repository when you clone the superproject
from it. Then the client can make a better decision where to get the
submodules from.
quoted
quoted
...<
quoted
I think on a hosting site they could even coexist when having the
layout as above.
quoted
top.git/
top.git/refs/{heads,tags,...}/...
top.git/objects/...
sub.git/
sub.git/refs/{heads,tags,...}/...
sub.git/objects/...
quoted
# the following only exist in non bare:
top.git/modules/sub.git/
top.git/modules/sub.git/refs/{heads,tags,...}/...
top.git/modules/sub.git/objects/...
quoted
The later files would be more reflective of what you *really*
want if you clone from top.git.
may be there is no need for assumptions and .gitmodules should be
sufficient?
- absolute url in .gitmodules provides absolute URL/path to the
submodule of interest, regardless either submodule is present in
originating repository as updated submodule. Either cloning it
instead of original repository would be more efficient is already a
heuristic which might fail miserably (may be I have a faster
connection to the original repository pointed by the absolute
url than to this particular repository)
- relative url in .gitmodules provides relative location to the location
of the "top" repository, and that is only when that submodule "absolute"
url should be resolved relative to the one of the "top" repository
I think the .gitmodules file is not sufficient for the following reason:
* As a "downstream" user you cannot change remote locations without
altering the history. Maybe you just want to have a mirror of some cool
open source project without the hassle to always merge and maintain changes
in your local submodules configuration. (c.f. git config url.<base>.insteadOf
for repos, just for submodule specific)
NB I will consider it a separate issue either relative paths
without '../' prefix are having any sense in bare repositories.
I guess it is a separate issue.
or have I missed the point?
--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik