Re: RFC: Subprojects

9 messages, 5 authors, 2016-06-15 · open the first message on its own page

Re: RFC: Subprojects

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:42:16

Daniel Barkalow [off-list ref] writes:
So why not use the "bind" approach for the "index vs working tree" part, 
but write out "gitlink"-style tree objects?
I said "index vs working tree" as a mere example, and never said
"gitlink" is easier (or at least as easy as "bind") for "tree
object vs index" or "tree object vs working tree through index".
In fact I suspect those parts also need to be changed fairly
heavily, and to be honest, I am not very much looking forward to
investigating the details.
In any case, I think it would be good to track where the subprojects are 
in some core state, and probably the right solution is to have special 
index entries for them, in addition to having their contents in the index. 
Actually, the "special entry" was what I found out to be quite a
pain, if you mean to have "linux-2.6/" in the index and have it
used in some meaningful way.  Further hacking and prototyping
_might_ convince me otherwise, but I am not so optimistic at
this moment.
I'm not seeing a clear way to get from commit objects with "bind" lines to 
an index with the appropriate things read and back otherwise.
Here again I am thinking aloud, remembering the earlier example
of an embedded linux project that ships with linux-2.6 and
gcc-4.0, along with its own README and Makefile at the toplevel
and src/ for its own sources.  The tools at the tip of "pu"
should be able to let you do the following:

	$ git cat-file commit $such_toplevel_commit
	tree $tree
        parent $parent
        bind $primarysub /
        bind $linuxsub linux-2.6/
        bind $gccsub gcc-4.0/
	author A U Thor [off-list ref] 1137392543 -0800
	commmitter A U Thor [off-list ref] 1137392543 -0800

        An example.

where $tree is the object name of the whole tree (no "gitlink"
object), $primarysub and $linuxsub are the object names of
commit objects for the primary subproject (which sits at the
rootlevel) and another subproject (which sits at linux-2.6/
subdirectory).

To make sure there is no misunderstanding:

	* "git-ls-tree $tree" would show the object name of
          $linuxsub^{tree} at path "linux-2.6/" because
          "tree" line of a commit describes the whole tree,
          including subprojects.

	* "git-ls-tree $primarysub" would show README,
          Makefile and src/ directories but not linux-2.6/ nor
          gcc-4.0/.

	* "git-ls-tree $linuxsub" would show COPYING, Makefile
          etc., not linux-2.6/COPYING.

Reading such a commit is easy:

	$ git-read-tree $tree ;# ;-)

But that is cheating.  Constructing such an index can be done by:

	$ git-read-tree $primarysub
        $ git-read-tree --prefix=linux-2.6/ $linuxsub
        $ git-read-tree --prefix=gcc-4.0/ $gccsub

When you have such an index, writing out various trees are:

	$ git-write-tree ;# $tree
	$ git-write-tree --prefix=linux-2.6/ ;# $linuxsub^{tree}
	$ git-write-tree --prefix=gcc-4.0/ ;# $gccsub^{tree}
	$ git-write-tree \
          --bound=linux-2.6/ --bound=gcc-4.0/ ;# $primarysub^{tree}

The decision to use what --prefix and --bound and what tree(s)
to write out must come from somewhere, and as you say it would
be nice if we _could_ stick them in the index as "special
entries", but for the purpose of prototyping I am assuming I
keep that somewhere in $GIT_DIR/ (the "mtab" in the previous
message.  Maybe "$GIT_DIR/bind" is a good name?).

Re: RFC: Subprojects

From: Petr Baudis <hidden>
Date: 2016-06-15 22:42:16

Dear diary, on Tue, Jan 17, 2006 at 07:18:47AM CET, I got a letter
where Junio C Hamano [off-list ref] said that...
Here again I am thinking aloud, remembering the earlier example
of an embedded linux project that ships with linux-2.6 and
gcc-4.0, along with its own README and Makefile at the toplevel
and src/ for its own sources.  The tools at the tip of "pu"
should be able to let you do the following:

	$ git cat-file commit $such_toplevel_commit
	tree $tree
        parent $parent
        bind $primarysub /
        bind $linuxsub linux-2.6/
        bind $gccsub gcc-4.0/
	author A U Thor [off-list ref] 1137392543 -0800
	commmitter A U Thor [off-list ref] 1137392543 -0800

        An example.

where $tree is the object name of the whole tree (no "gitlink"
object), $primarysub and $linuxsub are the object names of
commit objects for the primary subproject (which sits at the
rootlevel) and another subproject (which sits at linux-2.6/
subdirectory).
I perhaps missed this in the thread, but is it really so useful to bind
the subprojects to specific commits? If you care about reproducing
specific configuration, all you have to do is tag and seek recursively -
and even having a separate tiny git branch tracking just a single file
listing the commit ids of subprojects seems more elegant to me than just
forcing the specific commit ids. In the general case, I think it most
usually goes "this project#branch, the latest commit you can get", so
I'm not really convinced that you are optimizing for the right case at
all.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: RFC: Subprojects

From: Daniel Barkalow <hidden>
Date: 2016-06-15 22:42:16

On Tue, 17 Jan 2006, Petr Baudis wrote:
I perhaps missed this in the thread, but is it really so useful to bind
the subprojects to specific commits? If you care about reproducing
specific configuration, all you have to do is tag and seek recursively -
and even having a separate tiny git branch tracking just a single file
listing the commit ids of subprojects seems more elegant to me than just
forcing the specific commit ids. In the general case, I think it most
usually goes "this project#branch, the latest commit you can get", so
I'm not really convinced that you are optimizing for the right case at
all.
Think from a debugging standpoint. You know that the main project worked 
with a particular commit of the superproject. The bug you've found is 
related to the behavior of one of the subprojects in the the context of 
your superproject, but you don't know this. In order to reproduce the 
working version and search for the change that broken things, you need to 
be able to identify which commits of subprojects were used in each commit 
of the superproject; these are almost certainly not the latest commits on 
any branch of the subproject. And if you're going to want to debug things 
later, no commit of the superproject can just say to use the latest in the 
subproject. You don't know what you make a commit whether it will turn out 
to be a configuration that you'll want to recreate later.

Now it may be useful to have a tool to update all of the subprojects to 
the latest versions, similar in end-user usage to pulling repositories, 
but you still need to generate new commits when you do this, rather than 
reinterpreting the old commits with new content, so that you keep the 
history immutable. You also want to know when you've done this, so that 
you don't clone a tree that's working fine and build it only to find that 
the clone has fetched a new kernel version with different behavior without 
letting you know that anything has changed.

	-Daniel
*This .sig left intentionally blank*

Re: RFC: Subprojects

From: Craig Schlenter <hidden>
Date: 2016-06-15 22:42:16

Hi

For reference, here is what subversion does:

http://svnbook.red-bean.com/nightly/en/svn.advanced.externals.html

--Craig

Re: RFC: Subprojects

From: Linus Torvalds <torvalds@osdl.org>
Date: 2016-06-15 22:42:16


On Tue, 17 Jan 2006, Daniel Barkalow wrote:
Think from a debugging standpoint. You know that the main project worked 
with a particular commit of the superproject.
Yes, there are real advantages to being able to tag a very specific 
version of a tree.

You can do it manually (ie tag the versions of everything used), but 
there's a real convenience to being able to say "I want the tree to look 
exactly as it looked for our internal test-release that we shipped as a 
pre-view to customer so-and-so".

You can do it with ad-hoc build rules inside a company, but the likelihood 
that they don't work all the time is pretty high. Somebody forgot to 
follow the right procedure, and had updated a sub-tree without marking it, 
and now you can't reproduce the problem that a customer has with a debug 
build, because you have no way to reproduce the exact binary...

It's why people tag every file for huge trees under CVS for a release, and 
accept why building a release may take hours. It's crazy, yes, but there 
are other projects than just the BSD's that have that "World" mentality, 
where they want every single program under _one_ umbrella, so that they 
can tag them all together.

Me, I think it's crazy engineering ("if you can't reproduce it with 
individual projects, you're not doing programming, you're doing Voodoo"), 
but it's something that some organizations simply require.

Now, it might be enough with a cogito approach of ".git/subprojects", and 
just _version-control_ it in the top-level project, but then you'd need to 
make sure that all the tools automatically update the version when they do 
a "pull" or a "commit" on a subproject. But then it almost boils down to 
"gitlink"s after all.

		Linus

Re: RFC: Subprojects

From: Daniel Barkalow <hidden>
Date: 2016-06-15 22:42:16

On Mon, 16 Jan 2006, Junio C Hamano wrote:
Daniel Barkalow [off-list ref] writes:
quoted
So why not use the "bind" approach for the "index vs working tree" part, 
but write out "gitlink"-style tree objects?
I said "index vs working tree" as a mere example, and never said
"gitlink" is easier (or at least as easy as "bind") for "tree
object vs index" or "tree object vs working tree through index".
In fact I suspect those parts also need to be changed fairly
heavily, and to be honest, I am not very much looking forward to
investigating the details.
I suspect that it's just as easy, except that you get confronted 
immediately with the issues that you haven't dealt with in the bind 
approach (mentioned below). If you had parse_tree_buffer() just ignore 
them, and had write-tree take a list of bind lines, that would match the 
status of your "bind" implementation, I think (except for the part you say 
is cheating).

Incidentally, I don't think we'd want "gitlink" objects with the "gitlink" 
approach; we'd want trees to contain commit objects for subprojects. The 
"gitlink" thing that corresponds to ".git/HEAD" isn't an object, it's a 
tree entry, which, like ".git/HEAD" (or, more appropriately, 
".git/refs/heads/something") maps a name to the hash of a commit object.
quoted
In any case, I think it would be good to track where the subprojects are 
in some core state, and probably the right solution is to have special 
index entries for them, in addition to having their contents in the index. 
Actually, the "special entry" was what I found out to be quite a
pain, if you mean to have "linux-2.6/" in the index and have it
used in some meaningful way.  Further hacking and prototyping
_might_ convince me otherwise, but I am not so optimistic at
this moment.
Hmm... maybe libification should go ahead of subprojects. If access to the 
index weren't so often open-coded, it would just be a matter of having 
these entries in the data structure, but not actually returned by any 
current call, and it would be just like they were in some other structure. 

Actually, it should be easy to have them in the index file but not in the 
main index data structure, by skipping over them in the for loop near the 
end of read_cache(). Put them in a separate structure, and write them back 
to the file in write_cache(), and have a different method entirely for 
changing them, and they shouldn't affect the normal use of the index.
quoted
I'm not seeing a clear way to get from commit objects with "bind" lines to 
an index with the appropriate things read and back otherwise.
Here again I am thinking aloud, remembering the earlier example
of an embedded linux project that ships with linux-2.6 and
gcc-4.0, along with its own README and Makefile at the toplevel
and src/ for its own sources.  The tools at the tip of "pu"
should be able to let you do the following:

	$ git cat-file commit $such_toplevel_commit
	tree $tree
        parent $parent
        bind $primarysub /
        bind $linuxsub linux-2.6/
        bind $gccsub gcc-4.0/
	author A U Thor [off-list ref] 1137392543 -0800
	commmitter A U Thor [off-list ref] 1137392543 -0800

        An example.

where $tree is the object name of the whole tree (no "gitlink"
object), $primarysub and $linuxsub are the object names of
commit objects for the primary subproject (which sits at the
rootlevel) and another subproject (which sits at linux-2.6/
subdirectory).

To make sure there is no misunderstanding:

	* "git-ls-tree $tree" would show the object name of
          $linuxsub^{tree} at path "linux-2.6/" because
          "tree" line of a commit describes the whole tree,
          including subprojects.

	* "git-ls-tree $primarysub" would show README,
          Makefile and src/ directories but not linux-2.6/ nor
          gcc-4.0/.

	* "git-ls-tree $linuxsub" would show COPYING, Makefile
          etc., not linux-2.6/COPYING.
Side issue here: this implies that the kernel objects are in the 
superproject's repository, or at least accessible from it. So prune has to 
not remove them. So, if you've committed changes to a subproject but not 
yet committed the fact that you want to use the changed subproject into 
the superproject, fsck-objects has to find them somewhere.
Reading such a commit is easy:

	$ git-read-tree $tree ;# ;-)

But that is cheating.  
This is for backwards compatibility, I assume?
Constructing such an index can be done by:

	$ git-read-tree $primarysub
        $ git-read-tree --prefix=linux-2.6/ $linuxsub
        $ git-read-tree --prefix=gcc-4.0/ $gccsub

When you have such an index, writing out various trees are:

	$ git-write-tree ;# $tree
	$ git-write-tree --prefix=linux-2.6/ ;# $linuxsub^{tree}
	$ git-write-tree --prefix=gcc-4.0/ ;# $gccsub^{tree}
	$ git-write-tree \
          --bound=linux-2.6/ --bound=gcc-4.0/ ;# $primarysub^{tree}

The decision to use what --prefix and --bound and what tree(s)
to write out must come from somewhere, and as you say it would
be nice if we _could_ stick them in the index as "special
entries", but for the purpose of prototyping I am assuming I
keep that somewhere in $GIT_DIR/ (the "mtab" in the previous
message.  Maybe "$GIT_DIR/bind" is a good name?).
The hard thing here is getting the commits for the trees. The bind lines 
need commits, which means either identifying that we already have the 
correct commit object, because we didn't change anything in the 
subproject, or generating a new commit object with some message and the 
right parent. And we want to use commit objects, not tree objects, in the 
bind lines, so that, once we track a problem to the change of which commit 
is bound, we can treat the subproject as a project and debug it with 
bisect, rather than just having one tree that works and one that doesn't.

	-Daniel
*This .sig left intentionally blank*

Re: RFC: Subprojects

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:42:16

Daniel Barkalow [off-list ref] writes:
Incidentally, I don't think we'd want "gitlink" objects with the "gitlink" 
approach; we'd want trees to contain commit objects for subprojects. The 
"gitlink" thing that corresponds to ".git/HEAD" isn't an object, it's a 
tree entry, which, like ".git/HEAD" (or, more appropriately, 
".git/refs/heads/something") maps a name to the hash of a commit object.
Hmm... maybe libification should go ahead of subprojects. If access to the 
index weren't so often open-coded, it would just be a matter of having 
these entries in the data structure, but not actually returned by any 
current call, and it would be just like they were in some other structure. 
And libification has been waiting for the core to settle ;-) We
have to start somewhere.
Actually, it should be easy to have them in the index file but not in the 
main index data structure, by skipping over them in the for loop near the 
end of read_cache()....
Yeah, I guess I was vaguely thinking along those lines while I
was driving to work this morning.  I appreciate your spelling it
out to make things clearer.
Side issue here: this implies that the kernel objects are in the 
superproject's repository, or at least accessible from it. So prune has to 
not remove them. So, if you've committed changes to a subproject but not 
yet committed the fact that you want to use the changed subproject into 
the superproject, fsck-objects has to find them somewhere.
Yes.  I was planning to have "$GIT_DIR/bind" that says:

	master kernel=linux-2.6/ gcc=gcc-4.0/

meaning:

	The project kept track by "master" branch binds the
	project kept track by "kernel" branch as its subproject
	at its linux-2.6/ subdirectory.

or something like that, so when you make a commit, you update
those other branches as needed.  You already raised that issue
at the end of your message, and I will explain how I think that
can/should be done as a response to that part later.
quoted
Reading such a commit is easy:

	$ git-read-tree $tree ;# ;-)

But that is cheating.  
This is for backwards compatibility, I assume?
This is done more for not having to touch *anything* that does
"index vs working file", "tree vs index" and "tree vs working
file via index".  It also is the easiest way to keep the "a
commit object name can be used in place of the tree object name
of the tree it contains" invariant.  Also I suspect this
organization might help recursive subprojects, but if it is the
case, that is just a byproduct, not a design goal.
quoted
When you have such an index, writing out various trees are:

	$ git-write-tree ;# $tree
	$ git-write-tree --prefix=linux-2.6/ ;# $linuxsub^{tree}
	$ git-write-tree --prefix=gcc-4.0/ ;# $gccsub^{tree}
	$ git-write-tree \
          --bound=linux-2.6/ --bound=gcc-4.0/ ;# $primarysub^{tree}
The hard thing here is getting the commits for the trees. The bind lines 
need commits, which means either identifying that we already have the 
correct commit object, because we didn't change anything in the 
subproject, or generating a new commit object with some message and the 
right parent. And we want to use commit objects, not tree objects, in the 
bind lines, so that, once we track a problem to the change of which commit 
is bound, we can treat the subproject as a project and debug it with 
bisect, rather than just having one tree that works and one that doesn't.
Your wording "get the commit" is a bit misleading.  Even when
the tree for a subproject happens to match a commit in the
subproject in a distant past, we would not want to use it unless
the user explicitly asked for it.  IOW, we do not actively go
and look for a commit.

Our subproject tree either matches the subproject branch head,
in which case we just reuse it, or we make a new commit on top
of that ourselves.

Let's say my project breaks with the latest kernel, and I
suspect that it would work with v2.6.13 sources.  To test that
theory, I could:

        $ git branch -f kernel v2.6.13 ;# rewind

	$ git ls-files linux-2.6/ |
          xargs git update-index --force-remove
        $ git read-tree --prefix=linux-2.6/ -u kernel

to construct such a tree.  Maybe the latter two-command sequence
"ls-files & read-tree --prefix" sequence deserves to become a
command, "git update-subproject kernel" [*1*].

The result may work as-is, or I may need to do some further
futzing in linux-2.6/ directory before the result works.  Once
the result starts working, I'd want to make a commit:

 - I compare the result of write-tree for linux-2.6/ portion and
   the tree object name contained in the head commit of the
   "kernel" branch.  If they match, then the current "kernel"
   branch head commit is what I'll place on the "bind" line in
   my commit; I do not have to make a new commit in the "kernel"
   subproject in this case.

 - If the tree object does not match the "kernel" head, that
   means I have tweaked the kernel part further, on top of
   v2.6.13.  So I make a commit for the kernel subproject (whose
   parent is obviously v2.6.13), update the kernel branch head
   with that commit, and then record that tip-of-the-tree commit
   for the subproject on the "bind" line in my commit for the
   toplevel.

Or let's say my project builds with the latest kernel (IOW, I
did not do the branch -f kernel in the above), and I made some
custom tweaks in the kernel area.  The above precedure would
result in a new commit on top of the latest kernel, update the
"kernel" branch head, and make a commit for the toplevel that
records the updated "kernel" branch head on its "bind" line.

Note that the above procedure did not use the commit object name
recorded on the "bind" line at all in either case.  From the
mechanism point of view, it is the right thing to do.  From the
usability point of view, however, we may want to take notice
that "bind" line commit and the bound branch head do not match,
and remind/warn the user about it.  If the reason why they are
different is because the user rewound the bound branch to use a
known working version, or made fixes in the subproject and
pulled the result into the bound branch (in which case there is
no funny rewinding involved), then this warning is
extraneous. But in the normal case of keep reusing the same
vintage of subprojects (and maybe making necessary adjustments
to subprojects while working on the main project), the commit
object on the "bind" line of the HEAD commit and bound branch
head should match.


[Footnote]

*1* One could also do a forward development on the kernel branch
in a separate working tree and fetch from there.  For example,
if our example "superproject" is in embed/ directory, and there
is a linux/ directory next to it to house a kernel repository,
we could:

        $ cd ../linux/
        $ edit && compile && test 
        $ git commit -m 'Fix for upstream, not just for embed'

to make an upstream fix, and then:

        $ cd ../embed/
        $ git fetch ../linux/ master:kernel

to update the "kernel" subproject branch head.  In such a case:

	$ git update-subproject kernel

would bring the subproject working tree and index up to date
with respect to the updated kernel branch.

Re: RFC: Subprojects

From: Daniel Barkalow <hidden>
Date: 2016-06-15 22:42:16

On Tue, 17 Jan 2006, Junio C Hamano wrote:
Daniel Barkalow [off-list ref] writes:
quoted
Incidentally, I don't think we'd want "gitlink" objects with the "gitlink" 
approach; we'd want trees to contain commit objects for subprojects. The 
"gitlink" thing that corresponds to ".git/HEAD" isn't an object, it's a 
tree entry, which, like ".git/HEAD" (or, more appropriately, 
".git/refs/heads/something") maps a name to the hash of a commit object.
quoted
Hmm... maybe libification should go ahead of subprojects. If access to the 
index weren't so often open-coded, it would just be a matter of having 
these entries in the data structure, but not actually returned by any 
current call, and it would be just like they were in some other structure. 
And libification has been waiting for the core to settle ;-) We
have to start somewhere.
Well, we could do a pass at cleaning, organizing, and documenting the 
internals, which is sort of the start to each of them.
quoted
Side issue here: this implies that the kernel objects are in the 
superproject's repository, or at least accessible from it. So prune has to 
not remove them. So, if you've committed changes to a subproject but not 
yet committed the fact that you want to use the changed subproject into 
the superproject, fsck-objects has to find them somewhere.
Yes.  I was planning to have "$GIT_DIR/bind" that says:

	master kernel=linux-2.6/ gcc=gcc-4.0/

meaning:

	The project kept track by "master" branch binds the
	project kept track by "kernel" branch as its subproject
	at its linux-2.6/ subdirectory.

or something like that, so when you make a commit, you update
those other branches as needed.  You already raised that issue
at the end of your message, and I will explain how I think that
can/should be done as a response to that part later.
Okay, so you're using additional branch heads in the superproject to track 
the current state of the subprojects. That makes sense, although I think 
it would confuse people less if they were held separately. IIRC, 
refs/subprojects/kernel/heads/master is a perfectly good ref name these 
days, so that might be a good idea. That would also mean that 
refs/tags/v2.6.14 and refs/tags/v2.7.2.3 wouldn't get confused (being 
linux and gcc tags, respectively), because they'd be under the appropriate 
subprojects.

I assume these get updated by checkout when you check out the commit with 
them as bind lines?
quoted
quoted
Reading such a commit is easy:

	$ git-read-tree $tree ;# ;-)

But that is cheating.  
This is for backwards compatibility, I assume?
This is done more for not having to touch *anything* that does
"index vs working file", "tree vs index" and "tree vs working
file via index".  It also is the easiest way to keep the "a
commit object name can be used in place of the tree object name
of the tree it contains" invariant.  Also I suspect this
organization might help recursive subprojects, but if it is the
case, that is just a byproduct, not a design goal.
Ah, okay, so it's cheating for checkout, because checkout is supposed to 
understand everything, but not cheating for other things. I thought we 
decided that the stuff that doesn't know about subprojects sees them as 
opaque, rather than as their contents, so your toplevel git diff doesn't 
show you a millions lines when you switch from linux-2.6.14 to 15.
quoted
quoted
When you have such an index, writing out various trees are:

	$ git-write-tree ;# $tree
	$ git-write-tree --prefix=linux-2.6/ ;# $linuxsub^{tree}
	$ git-write-tree --prefix=gcc-4.0/ ;# $gccsub^{tree}
	$ git-write-tree \
          --bound=linux-2.6/ --bound=gcc-4.0/ ;# $primarysub^{tree}
The hard thing here is getting the commits for the trees. The bind lines 
need commits, which means either identifying that we already have the 
correct commit object, because we didn't change anything in the 
subproject, or generating a new commit object with some message and the 
right parent. And we want to use commit objects, not tree objects, in the 
bind lines, so that, once we track a problem to the change of which commit 
is bound, we can treat the subproject as a project and debug it with 
bisect, rather than just having one tree that works and one that doesn't.
Your wording "get the commit" is a bit misleading.  Even when
the tree for a subproject happens to match a commit in the
subproject in a distant past, we would not want to use it unless
the user explicitly asked for it.  IOW, we do not actively go
and look for a commit.
We don't search the history for just any commit, but we have to look 
somewhere...
Our subproject tree either matches the subproject branch head,
in which case we just reuse it, or we make a new commit on top
of that ourselves.
I hadn't realized that the subprojects had branch heads. It makes much 
more sense that you'd expect to be able to just write out bind lines if 
you've got that information.

I thought we decided that committing the superproject wouldn't commit the 
subprojects. If this is what we decided, then the subproject tree is 
required to match the branch head, because we must have committed the 
subproject already (which is good, because otherwise the user will get 
confused about which commit message to write when).
Let's say my project breaks with the latest kernel, and I
suspect that it would work with v2.6.13 sources.  To test that
theory, I could:

        $ git branch -f kernel v2.6.13 ;# rewind

	$ git ls-files linux-2.6/ |
          xargs git update-index --force-remove
        $ git read-tree --prefix=linux-2.6/ -u kernel

to construct such a tree.  Maybe the latter two-command sequence
"ls-files & read-tree --prefix" sequence deserves to become a
command, "git update-subproject kernel" [*1*].
Shouldn't "git read-tree --prefix=linux-2.6/ -u kernel" remove everything 
else in the index in linux-2.6 itself, making the "git update-index 
--force-remove" unnecessary?
The result may work as-is, or I may need to do some further
futzing in linux-2.6/ directory before the result works.  Once
the result starts working, I'd want to make a commit:

 - I compare the result of write-tree for linux-2.6/ portion and
   the tree object name contained in the head commit of the
   "kernel" branch.  If they match, then the current "kernel"
   branch head commit is what I'll place on the "bind" line in
   my commit; I do not have to make a new commit in the "kernel"
   subproject in this case.

 - If the tree object does not match the "kernel" head, that
   means I have tweaked the kernel part further, on top of
   v2.6.13.  So I make a commit for the kernel subproject (whose
   parent is obviously v2.6.13), update the kernel branch head
   with that commit, and then record that tip-of-the-tree commit
   for the subproject on the "bind" line in my commit for the
   toplevel.
Equivalently, you make a commit for the kernel subproject, update the 
branch head, and start over; the first case should apply now.
Or let's say my project builds with the latest kernel (IOW, I
did not do the branch -f kernel in the above), and I made some
custom tweaks in the kernel area.  The above precedure would
result in a new commit on top of the latest kernel, update the
"kernel" branch head, and make a commit for the toplevel that
records the updated "kernel" branch head on its "bind" line.

Note that the above procedure did not use the commit object name
recorded on the "bind" line at all in either case.  From the
mechanism point of view, it is the right thing to do.  From the
usability point of view, however, we may want to take notice
that "bind" line commit and the bound branch head do not match,
and remind/warn the user about it.  If the reason why they are
different is because the user rewound the bound branch to use a
known working version, or made fixes in the subproject and
pulled the result into the bound branch (in which case there is
no funny rewinding involved), then this warning is
extraneous. But in the normal case of keep reusing the same
vintage of subprojects (and maybe making necessary adjustments
to subprojects while working on the main project), the commit
object on the "bind" line of the HEAD commit and bound branch
head should match.
I hope people will want to prepare their commits to the kernel subproject 
as would be suitable for pushing to Linus, which would suggest that they'd 
tend to do a commit in the kernel subproject embedded in their 
superproject separately from doing the commit in the superproject, and 
so the branch head would match the index but not the bind line when they 
got to committing the superproject.

	-Daniel
*This .sig left intentionally blank*

Re: RFC: Subprojects

From: Petr Baudis <hidden>
Date: 2016-06-15 22:42:17

Dear diary, on Wed, Jan 18, 2006 at 07:21:59PM CET, I got a letter
where Daniel Barkalow [off-list ref] said that...
Okay, so you're using additional branch heads in the superproject to track 
the current state of the subprojects. That makes sense, although I think 
it would confuse people less if they were held separately. IIRC, 
refs/subprojects/kernel/heads/master is a perfectly good ref name these 
days, so that might be a good idea. That would also mean that 
refs/tags/v2.6.14 and refs/tags/v2.7.2.3 wouldn't get confused (being 
linux and gcc tags, respectively), because they'd be under the appropriate 
subprojects.
I passionately agree - this is the only thing I do not like on the
current Junio's proposal (besides that top-level subproject confusion).
The way it is proposed, you are mixing different projects in a single
refs namespace and I think that's *really* confusing.

Besides, you are going to get a lot of complications since to do merging
properly you need two heads per subproject (its 'master' and 'origin'
heads; and it's useful to have e.g. all the upstream heads called
'origin' since then you can say cg-fetch -r origin in the superproject
and have all the subproject origins fetched as well) and you might want
to have other subproject heads as well. Now, for different superproject
heads, you want separate set of subproject heads. You can see the
downward spiral from here, I guess... And multiply all that by two since
you also have tags.

It actually took me a short while to realize that keeping separate
subproject/.git/refs makes no sense precisely because for different
superproject heads, you want a different set of subproject refs.
So in line with Daniel's proposal, I'd propose:

	refs/subprojects/<superhead>/<subid>/heads/master

<superhead> is the name of the current HEAD (${#refs/heads/}). <subid>
is a little more tricky - this should be the part after the equal sign
in .git/mtab (or .git/binds or .git/subprojects or whichever is the name
of the day). Obviously, you can just figure out something, but I'd like
to assign this automagically.

OTOH, in Cogito I might as well just default to sha1 of something random
(e.g.  the path+commitid+time()) since I do not expect this to be
normally referenced by a human; I just intend to switch from refs/ to
refs/subprojects/<superhead>/<subid>/ when dealing with the subproject
exclusively. ($GIT_REF_DIR (by default $GIT_DIR/refs) would come useful;
I'll probably whip up a patch when I get to finally need it.)
I hope people will want to prepare their commits to the kernel subproject 
as would be suitable for pushing to Linus, which would suggest that they'd 
tend to do a commit in the kernel subproject embedded in their 
superproject separately from doing the commit in the superproject, and 
so the branch head would match the index but not the bind line when they 
got to committing the superproject.
FWIW, my idea is that it should be "a seamless experience for the user"
(tm) to do development in a subproject of another project, and I can see
no reason why should that be hard to do in any way.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help