From: Junio C Hamano <hidden> Date: 2016-08-24 05:44:56
Jakub Narębski [off-list ref] writes:
The point is that submodule has it's own object database. It might
be the same as superproject's, but you need to handle submodule objects
being in separate submodule repository anyway. Common repository is
just a special case.
By the way, this also means that proposed "extended extended SHA1"
syntax would be useful to user's of submodules...
Not really.
I think that you gave a prime example why <treeish>:<path1>//<path2>
is not a useful thing for submodules. When the syntax resolves to a
40-hex object name, that object name by itself is not useful.
You also need to carry an additional piece of information that lets
you identify the location of the repository, in which the object
name is valid, in the current user's context (i.e. somewhere in the
superproject where the submodule lives). In other words, you'd need
to carry <treeish>:<path1> around anyway for the object name to be
useful, so there is no good reason why anybody should insist that
the plumbing level resolve <treeish>:<path1>//<path2> directly to an
object name in the first place.
From: Jakub Narębski <hidden> Date: 2016-08-24 13:17:06
W dniu 24.08.2016 o 07:36, Junio C Hamano pisze:
Jakub Narębski [off-list ref] writes:
quoted
The point is that submodule has it's own object database. It might
be the same as superproject's, but you need to handle submodule objects
being in separate submodule repository anyway. Common repository is
just a special case.
By the way, this also means that proposed "extended extended SHA1"
syntax would be useful to user's of submodules...
Not really.
I think that you gave a prime example why <treeish>:<path1>//<path2>
is not a useful thing for submodules. When the syntax resolves to a
40-hex object name, that object name by itself is not useful.
You also need to carry an additional piece of information that lets
you identify the location of the repository, in which the object
name is valid, in the current user's context (i.e. somewhere in the
superproject where the submodule lives). In other words, you'd need
to carry <treeish>:<path1> around anyway for the object name to be
useful, so there is no good reason why anybody should insist that
the plumbing level resolve <treeish>:<path1>//<path2> directly to an
object name in the first place.
Not really.
The above means only that the support for new syntax would be not
as easy as adding it to 'git rev-parse' (and it's built-in equivalent),
except for the case where submodule uses the same object database as
supermodule.
So it wouldn't be as easy (on conceptual level) as adding support
for ':/<text>' or '<commit>^{/<text>}'. It would be at least as
hard, if not harder, as adding support for '@{-1}' and its '-'
shortcut.
Josh, what was the reason behind proposing this feature? Was it
conceived as adding completeness to gitrevisions syntax, a low-hanging
fruit? It isn't (the latter). Or was it some problem with submodule
handling that you would want to use this syntax for?
As for usefulness: this fills the hole in accessing submodules, one
that could be handled by combining plumbing-level commands. Namely,
there are 5 states of submodule (as I understand it)
* recorded in ref / commit in supermodule
* recorded in the index in supermodule
- recorded in ref / commit in submodule
- recorded in the index in submodule
- state of worktree in submodule
The last three can be easyly acessed by cd-ing to submodule. The first
two are not easy to get, AFAIUC.
--
Jakub Narębski
On Wed, Aug 24, 2016 at 03:16:56PM +0200, Jakub Narębski wrote:
W dniu 24.08.2016 o 07:36, Junio C Hamano pisze:
quoted
Jakub Narębski [off-list ref] writes:
quoted
The point is that submodule has it's own object database. It might
be the same as superproject's, but you need to handle submodule objects
being in separate submodule repository anyway. Common repository is
just a special case.
By the way, this also means that proposed "extended extended SHA1"
syntax would be useful to user's of submodules...
Not really.
I think that you gave a prime example why <treeish>:<path1>//<path2>
is not a useful thing for submodules. When the syntax resolves to a
40-hex object name, that object name by itself is not useful.
You also need to carry an additional piece of information that lets
you identify the location of the repository, in which the object
name is valid, in the current user's context (i.e. somewhere in the
superproject where the submodule lives). In other words, you'd need
to carry <treeish>:<path1> around anyway for the object name to be
useful, so there is no good reason why anybody should insist that
the plumbing level resolve <treeish>:<path1>//<path2> directly to an
object name in the first place.
Not really.
The above means only that the support for new syntax would be not
as easy as adding it to 'git rev-parse' (and it's built-in equivalent),
except for the case where submodule uses the same object database as
supermodule.
So it wouldn't be as easy (on conceptual level) as adding support
for ':/<text>' or '<commit>^{/<text>}'. It would be at least as
hard, if not harder, as adding support for '@{-1}' and its '-'
shortcut.
Depends on which cases you want to handle. In the most general case,
you'd need to find and process the applicable .gitmodules file, which
would only work if you started from the top-level tree, not a random
treeish. On the other hand, in the most general case, you don't
necessarily even have the module you need, because .git/modules only
contains the modules the *current* version needed, not every past
version.
As an alternate approach (pun intended): treat every module in
.git/modules as an alternate and just look up the object by hash. Or,
teach git-submodule to store all the objects for submodules in the
supermodule's .git/objects (and teach git's reachability algorithm to
respect refs in .git/modules, or store their refs in
.git/refs/submodules/ or in a namespace).
Josh, what was the reason behind proposing this feature? Was it
conceived as adding completeness to gitrevisions syntax, a low-hanging
fruit? It isn't (the latter). Or was it some problem with submodule
handling that you would want to use this syntax for?
This wasn't an abstract/theoretical completeness issue. I specifically
wanted this syntax for practical use with actual trees containing
gitlinks, motivated by having a tool that creates and uses such
gitlinks. :)
As for usefulness: this fills the hole in accessing submodules, one
that could be handled by combining plumbing-level commands. Namely,
there are 5 states of submodule (as I understand it)
* recorded in ref / commit in supermodule
* recorded in the index in supermodule
- recorded in ref / commit in submodule
- recorded in the index in submodule
- state of worktree in submodule
The last three can be easyly acessed by cd-ing to submodule. The first
two are not easy to get, AFAIUC.
Right. I primarily care about those first two cases, especially the
first one: given a commit containing a gitlink, how can I easily dig
into the linked commit?
From: Stefan Beller <hidden> Date: 2016-08-24 16:26:38
On Wed, Aug 24, 2016 at 7:20 AM, Josh Triplett [off-list ref] wrote:
Depends on which cases you want to handle. In the most general case,
you'd need to find and process the applicable .gitmodules file, which
would only work if you started from the top-level tree, not a random
treeish. On the other hand, in the most general case, you don't
necessarily even have the module you need, because .git/modules only
contains the modules the *current* version needed, not every past
version.
The code in submodule-config.{c,h} allows exactly that:
submodule_from_path(commit_sha1, path)
returns information about the submodule recorded in a .gitmodules
file of a specific revision of the superproject.
Ideally the .git/modules contains all the modules that existed, ever.
Well "ideally" is the wrong word, but it is at least possible as the
submodules git dir is kept even when you remove the outdated
submodules working dir. (That's why the git dir is in the superprojects
git dir in the first place).
But as you say, it is possible of not having the submodule available.
As an alternate approach (pun intended): treat every module in
.git/modules as an alternate and just look up the object by hash. Or,
teach git-submodule to store all the objects for submodules in the
supermodule's .git/objects (and teach git's reachability algorithm to
respect refs in .git/modules, or store their refs in
.git/refs/submodules/ or in a namespace).
This is a sensible thing to do no matter the outcome of this discussion.
quoted
* recorded in ref / commit in supermodule
* recorded in the index in supermodule
- recorded in ref / commit in submodule
- recorded in the index in submodule
- state of worktree in submodule
The last three can be easyly acessed by cd-ing to submodule. The first
two are not easy to get, AFAIUC.
Right. I primarily care about those first two cases, especially the
first one: given a commit containing a gitlink, how can I easily dig
into the linked commit?
What do you exactly need? (What is digging here?)
See for example the series, that Jake Keller currently tries to land:
"submodule inline diff format"
https://public-inbox.org/git/20160822234344.22797-1-jacob.e.keller@intel.com
That would enhance all the log/diff/show things.
Reading the original message, do you want to create patches in
the submodule from the superproject?
Thanks,
Stefan
From: Jakub Narębski <hidden> Date: 2016-08-24 17:05:28
W dniu 24.08.2016 o 16:20, Josh Triplett pisze:
On Wed, Aug 24, 2016 at 03:16:56PM +0200, Jakub Narębski wrote:
[...]
quoted
Not really.
The above means only that the support for new syntax would be not
as easy as adding it to 'git rev-parse' (and it's built-in equivalent),
except for the case where submodule uses the same object database as
supermodule.
So it wouldn't be as easy (on conceptual level) as adding support
for ':/<text>' or '<commit>^{/<text>}'. It would be at least as
hard, if not harder, as adding support for '@{-1}' and its '-'
shortcut.
Depends on which cases you want to handle. In the most general case,
you'd need to find and process the applicable .gitmodules file, which
would only work if you started from the top-level tree, not a random
treeish. On the other hand, in the most general case, you don't
necessarily even have the module you need, because .git/modules only
contains the modules the *current* version needed, not every past
version.
There is an additional problem, namely that directory with submodule
can be renamed.
I don't know if there is an existing API, but assuming modern
git-submodule (with repository in .git/modules) you would have to
do the following steps for <revision>:<path/to/submodule>//<path>:
* look up <revision>:.gitmodules for module which 'path'
is <path/to/submodule>; let's say it is named <submodule>
* check if <revision>:<path/to/submodule> commit object
is present in .git/modules/<submodule>
* look up this object
In the case of legacy submodule setup, with submodule repository
in the supermodule working directory, you would need:
* look up <revision>:.gitmodules for module which 'path'
is <path/to/submodule>; let's say it is named <submodule>
* look up current .gitmodules for current path of submodule
named <submodule>; let's say it is <new/path/submodule>
* check of <revision>:</path/to/submodule> commit object
is present in :(top)<new/path/submodule>/.git repository
* look up this object
You could also check if the submodule repository (as stored
in config) is a path, and use it if it is... but that might
be going to far.
BTW. all that reminds me that gitweb should handle submodules
better.
As an alternate approach (pun intended): treat every module in
.git/modules as an alternate and just look up the object by hash.
This could be a good fallback, to search through all submodules.
Or, teach git-submodule to store all the objects for submodules in the
supermodule's .git/objects (and teach git's reachability algorithm to
respect refs in .git/modules, or store their refs in
.git/refs/submodules/ or in a namespace).
And fallback to this fallback could be searching through supermodule
object repository.
Storing all objects in single repository is counter to the design
decision of submodules (though I don't remember what it was), but
it might be done. Still, Git needs to be able to deal with legacy
situations anyway.
quoted
Josh, what was the reason behind proposing this feature? Was it
conceived as adding completeness to gitrevisions syntax, a low-hanging
fruit? It isn't (the latter). Or was it some problem with submodule
handling that you would want to use this syntax for?
This wasn't an abstract/theoretical completeness issue. I specifically
wanted this syntax for practical use with actual trees containing
gitlinks, motivated by having a tool that creates and uses such
gitlinks. :)
Could you explain what you need in more detail? Is it a fragment
of history of submodule, a contents of a file at given point of
superproject history, diff between file-in-submodule and something
else, or what?
quoted
As for usefulness: this fills the hole in accessing submodules, one
that could be handled by combining plumbing-level commands. Namely,
there are 5 states of submodule (as I understand it)
* recorded in ref / commit in supermodule
* recorded in the index in supermodule
- recorded in ref / commit in submodule
- recorded in the index in submodule
- state of worktree in submodule
The last three can be easyly acessed by cd-ing to submodule. The first
two are not easy to get, AFAIUC.
Right. I primarily care about those first two cases, especially the
first one: given a commit containing a gitlink, how can I easily dig
into the linked commit?
All right.
Though you can cobble it with plumbing... just saying.
--
Jakub Narębski
On Wed, Aug 24, 2016 at 07:05:17PM +0200, Jakub Narębski wrote:
W dniu 24.08.2016 o 16:20, Josh Triplett pisze:
quoted
On Wed, Aug 24, 2016 at 03:16:56PM +0200, Jakub Narębski wrote:
[...]
quoted
quoted
Not really.
The above means only that the support for new syntax would be not
as easy as adding it to 'git rev-parse' (and it's built-in equivalent),
except for the case where submodule uses the same object database as
supermodule.
So it wouldn't be as easy (on conceptual level) as adding support
for ':/<text>' or '<commit>^{/<text>}'. It would be at least as
hard, if not harder, as adding support for '@{-1}' and its '-'
shortcut.
Depends on which cases you want to handle. In the most general case,
you'd need to find and process the applicable .gitmodules file, which
would only work if you started from the top-level tree, not a random
treeish. On the other hand, in the most general case, you don't
necessarily even have the module you need, because .git/modules only
contains the modules the *current* version needed, not every past
version.
There is an additional problem, namely that directory with submodule
can be renamed.
I don't know if there is an existing API, but assuming modern
git-submodule (with repository in .git/modules) you would have to
do the following steps for <revision>:<path/to/submodule>//<path>:
* look up <revision>:.gitmodules for module which 'path'
is <path/to/submodule>; let's say it is named <submodule>
* check if <revision>:<path/to/submodule> commit object
is present in .git/modules/<submodule>
* look up this object
This also assumes your lookup started with a <committish> and not an
intermediate <treeish>, but that'll work in many cases.
quoted
As an alternate approach (pun intended): treat every module in
.git/modules as an alternate and just look up the object by hash.
This could be a good fallback, to search through all submodules.
quoted
Or, teach git-submodule to store all the objects for submodules in the
supermodule's .git/objects (and teach git's reachability algorithm to
respect refs in .git/modules, or store their refs in
.git/refs/submodules/ or in a namespace).
And fallback to this fallback could be searching through supermodule
object repository.
I'd flip those around: first search registered .gitmodules, then look up
the object in the superproject (since you have it at hand), and then
maybe search every submodule.
quoted
quoted
Josh, what was the reason behind proposing this feature? Was it
conceived as adding completeness to gitrevisions syntax, a low-hanging
fruit? It isn't (the latter). Or was it some problem with submodule
handling that you would want to use this syntax for?
This wasn't an abstract/theoretical completeness issue. I specifically
wanted this syntax for practical use with actual trees containing
gitlinks, motivated by having a tool that creates and uses such
gitlinks. :)
Could you explain what you need in more detail? Is it a fragment
of history of submodule, a contents of a file at given point of
superproject history, diff between file-in-submodule and something
else, or what?
As part of git-series, I have commits, whose trees contain various
gitlinks, such as "series" and "base". Those gitlinks point to commits
in the same repository. I'd like to use those gitlinks everywhere I
could use any other committish, such as a branch name. In particular,
I'd like to write things like some_feature:series:path/to/file ("what
does path/to/file look like in the current version of some_feature"),
some_feature:series^ ("what's the second-to-last commit in
some_feature"), some_feature~5:series:path/to/file ("what did
path/to/file look like in an older version of some_feature"), or
some_feature~5:base..some_feature~5:series~2 ("all but the last two
patches in some_feature~5"). Those should work with show, diff,
format-patch, log, etc.
quoted
quoted
As for usefulness: this fills the hole in accessing submodules, one
that could be handled by combining plumbing-level commands. Namely,
there are 5 states of submodule (as I understand it)
* recorded in ref / commit in supermodule
* recorded in the index in supermodule
- recorded in ref / commit in submodule
- recorded in the index in submodule
- state of worktree in submodule
The last three can be easyly acessed by cd-ing to submodule. The first
two are not easy to get, AFAIUC.
Right. I primarily care about those first two cases, especially the
first one: given a commit containing a gitlink, how can I easily dig
into the linked commit?
All right.
Though you can cobble it with plumbing... just saying.
Sure, but that makes the expression much more complex.