Hi all
[Jens, I was hoping you'd have some insight, but of course everyone is
welcome to join in. I honestly never thought I'd get into the
submodule business so shortly after gittogether...]
Out of $DAYJOB issues with tracking exercises and exams, I had a
little brainstorming session with a fellow assistant yesterday. It
seems none of this has been done before? Is anyone else interested
(mainly in getting the requirements right so that the next poor soul
won't have to roll their own)?
The scenario is that we have a bunch of exercise questions stored in
one or several files, each living in a directory. Then, we assemble
those into exercise sheets (super-repos) spanning a group of questions
(submodules). We would like to track this in such a way that changes
to the questions propagate to other sheets (possibly in other
lectures) that use them.
Training everyone in full git usage is probably not an option, so any
solution would have to have some frontend that can be explained
easily.
The requirements we came up with are roughly:
1) commit across all sub-repos at the same time (atomicity nice but
optional)
1b) tag across all sub-repos
2) do recursive clone/update of one super-repo easily
3) never need to be aware of repo boundaries or manipulate sub-repo
4) sanely cope with branches etc. in case the user wants them
A sample workflow might be:
foo clone git@example.com/some/super/repo
# time passes
cd repo
foo update
# work
foo status
foo diff
foo commit -m 'one message fits all'
# possibly, but this could also be commit --no-push instead
foo push
Merges are expected to be rare but would happen every so often.
It seems repo can do (2) and (4) but violates (3) [requires use of
git-commit and others in the sub-repo]. git-subtree can do (1) and
(2) but probably violates (3) [need to rebase changes to sub-repo
branch] and (4). Submodules can do (2) and (4) but violate (3) and
currently have no support for (1). I think svn externals could do
(1)-(3) but violate (4) and probably (1b).
Has this been done before? Or do we need to hack up a new wrapper
around submodules?
--
Thomas Rast
trast@{inf,student}.ethz.ch
In message [off-list ref], Thomas Rast writes:
The requirements we came up with are roughly:
1) commit across all sub-repos at the same time (atomicity nice but
optional)
1b) tag across all sub-repos
2) do recursive clone/update of one super-repo easily
3) never need to be aware of repo boundaries or manipulate sub-repo
4) sanely cope with branches etc. in case the user wants them
This is exactly the sort of situation I wrote gitslave for.
http://gitslave.sf.net I have a superproject of many subprojects where
I want to perform the same git operation on all of them at the same
time as if it was one giant repository (without the drawbacks of a
giant repository while losing as few as the benefits as possible).
It does not perform atomic commits across sub-repos, but otherwise
handles 1 and 1b. 2 and 4 are handled as well. 3 is mostly handled,
and by mostly, I mean if you get, for instance, a merge conflict in
the superproject or the subproject, you would need to resolve that
using the normal git commands and then typically retry the gitslave
command. gitslave does not prevent you from using any git command in
any way you might desire, it simply allows you to use "gits" instead
of "git" when you want your command to run over all repos.
A sample workflow might be:
foo clone git@example.com/some/super/repo
# time passes
cd repo
foo update
# work
foo status
foo diff
foo commit -m 'one message fits all'
# possibly, but this could also be commit --no-push instead
foo push
In the gitslave world, this would be done:
git clone git@example.com/some/super/repo
cd repo
gits checkout master
# time passes
gits pull
# work
gits status
gits diff
gits commit -a -m 'one message fits none'
# I am not familiar with the "git commit --no-push" option
gits push
Merges are expected to be rare but would happen every so often.
gits checkout featurebranch
# work
gits commit -a -m 'features are misunderstood'
gits checkout master
gits merge featurebranch
gits tag v1.0 -a -m "feature available"
gits push
gits push --tags
Has this been done before? Or do we need to hack up a new wrapper
around submodules?
Hopefully gitslave will work for you (or could be modified to work for
you). It is not perfect but has been working well enough for a team
of developers for over two years.
-Seth Robertson
Hi Thomas
Am 18.11.2010 11:09, schrieb Thomas Rast:
The scenario is that we have a bunch of exercise questions stored in
one or several files, each living in a directory. Then, we assemble1
those into exercise sheets (super-repos) spanning a group of questions
(submodules).
That sounds like each directory is maintained by a different group of
people and then there is another bunch of people choosing the content
of the next exercise sheets, right?
We would like to track this in such a way that changes
to the questions propagate to other sheets (possibly in other
lectures) that use them.
This could mean you would want an 'always-tip' mode for the
submodules, or am I misunderstanding you here?
Training everyone in full git usage is probably not an option, so any
solution would have to have some frontend that can be explained
easily.
Yup, makes sense (at least until something goes wrong, see 3). ;-)
The requirements we came up with are roughly:
1) commit across all sub-repos at the same time (atomicity nice but
optional)
1b) tag across all sub-repos
"git commit" and "git tag" could be taught the "--recurse-submodule"
option (but the commit part only makes sense when "git branch" can
do that too, so you have something to commit on. And I think you
want to enable a - yet to be implemented - file-based recursive diff
and status too, to see what changes your next commit will include).
And until all that materializes for submodules, a script would have
to do that.
2) do recursive clone/update of one super-repo easily
That will be handled by recursive checkout and is already achieved
by "git clone --recursive" (but at least in the first version both
don't support an "always-tip" mode).
3) never need to be aware of repo boundaries or manipulate sub-repo
I think that this requirement is the hardest for any solution I know
of or can imagine, as you hit these boundaries sooner or later either
when you want to commit, push and/or when you have to resolve merge
conflicts.
4) sanely cope with branches etc. in case the user wants them
A "--recurse-submodules" option to "git-branch" might be what you
want here, but as this isn't there yet a script will have to do that
for now.
A sample workflow might be:
foo clone git@example.com/some/super/repo
# time passes
cd repo
foo update
# work
foo status
foo diff
foo commit -m 'one message fits all'
# possibly, but this could also be commit --no-push instead
foo push
Merges are expected to be rare but would happen every so often.
It seems repo can do (2) and (4) but violates (3) [requires use of
git-commit and others in the sub-repo]. git-subtree can do (1) and
(2) but probably violates (3) [need to rebase changes to sub-repo
branch] and (4). Submodules can do (2) and (4) but violate (3) and
currently have no support for (1). I think svn externals could do
(1)-(3) but violate (4) and probably (1b).
Has this been done before? Or do we need to hack up a new wrapper
around submodules?
If you would base that on submodule functionality, you would have
to hack up a wrapper script for the foreseeable future because the
"fully integrated" world view you seem to need is not worked on yet
(and I didn't see anyone coming forward to do that).
I took a cursory glance at "gitslave" Seth mentioned, it might do
what you want, but I can't tell for sure as I never used it myself.
Jens