From: Junio C Hamano <hidden> Date: 2016-06-15 22:43:44
Theodore Tso [off-list ref] writes:
I assume PU is what you used to build your proposed-update branch?
I'm actually trying to use it, and it is useful, although it hasn't
been working for me completely perfectly. I've still been trying to
figure out if the reason why it hasn't been working quite right is due
to my not understanding how to use it correctly, or whether you don't
use it these days.
Ah, these days I almost always do:
git checkout pu
git reset --hard next
and then merge the topics that haven't been merged anywhere by
hand, using output from
Meta/git-topic.perl
as the guide.
One question which I have had about the WC script is that if I
manually add a commit to the next branch, it ends up showing up in all
of the topic branches as a commit that was part of that topic branch
which is in next...
Well, the policy is never to commit directly on top of next
(iow, only merge other topics and nothing else). Otherwise it
becomes hard to allow individual topics graduate to 'master'
independently.
On Mon, Oct 22, 2007 at 06:29:59PM -0700, Junio C Hamano wrote:
Well, the policy is never to commit directly on top of next
(iow, only merge other topics and nothing else). Otherwise it
becomes hard to allow individual topics graduate to 'master'
independently.
I see. So if it's non-trivial enough that you want it to "cook" in
next for a cycle, you'll create a topic branch for it (based off of
'master'), and then force a merge into 'next'?
- Ted
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:43:44
Theodore Tso [off-list ref] wrote:
On Mon, Oct 22, 2007 at 06:29:59PM -0700, Junio C Hamano wrote:
quoted
Well, the policy is never to commit directly on top of next
(iow, only merge other topics and nothing else). Otherwise it
becomes hard to allow individual topics graduate to 'master'
independently.
I see. So if it's non-trivial enough that you want it to "cook" in
next for a cycle, you'll create a topic branch for it (based off of
'master'), and then force a merge into 'next'?
Yes. Because 'next' always has commits in it that never appear in
'master'. So any topic forked from master must merge into next.
It can't be a fast-forward. No forced merging required.
Of course this isn't true for a new project. That first topic
that forked from master to *create* next will be a fast-forward
as it creates next. But that's no big deal. The second topic will
merge into next, and that first topic can still be merged back into
master without merging next (or the second topic).
I was also doing the same thing Junio already explained to manage
next and pu while he was away. Except I shortcut his:
git checkout pu
git reset --hard next
as:
git branch -f pu next
git checkout pu
as I'm was usually already sitting on next. This saved my poor
little laptop from a second of IO chugging as it slewed around
between the two versions. There were no files to update as it
switched from next to pu, and pu was already setup for merging
the proposed topics. :-)
--
Shawn.
On Tue, Oct 23, 2007 at 12:05:22AM -0400, Shawn O. Pearce wrote:
Yes. Because 'next' always has commits in it that never appear in
'master'. So any topic forked from master must merge into next.
It can't be a fast-forward. No forced merging required.
Why is it the case that 'next' always commits that never appear in
'master'. So far in how I've been doing things that hasn't been the
case. When I do a "git checkout master; git merge next", it's always
been a fast-forward merge.
Oh, I see. That's because if you put some trivial changes in
'master', and then pull those changes into next, there will be merge
commits in 'next' that will never be in 'master. Is that it?
I had been trying to avoid that case by always putting new commits,
even trivial ones, into 'next', and then having them drop into
'master' at the next cycle; so 'master' was always trailing 'next',
but they were always the same commit string (i.e., 'master' was always
a subset of 'next').
Aside from the WC script not working right, are there other
disadvantages to my doing things that way as opposed to the way the
Junio has been running the git repository?
- Ted
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:43:44
Theodore Tso [off-list ref] wrote:
On Tue, Oct 23, 2007 at 12:05:22AM -0400, Shawn O. Pearce wrote:
quoted
Yes. Because 'next' always has commits in it that never appear in
'master'. So any topic forked from master must merge into next.
It can't be a fast-forward. No forced merging required.
Why is it the case that 'next' always commits that never appear in
'master'. So far in how I've been doing things that hasn't been the
case. When I do a "git checkout master; git merge next", it's always
been a fast-forward merge.
Oh, I see. That's because if you put some trivial changes in
'master', and then pull those changes into next, there will be merge
commits in 'next' that will never be in 'master. Is that it?
Exactly. This of course means that next has been growing in distance
from master for quite some time. It has well over 1000 commits now
in git.git that aren't in master. Most of those will never merge
there either.
I had been trying to avoid that case by always putting new commits,
even trivial ones, into 'next', and then having them drop into
'master' at the next cycle; so 'master' was always trailing 'next',
but they were always the same commit string (i.e., 'master' was always
a subset of 'next').
Aside from the WC script not working right, are there other
disadvantages to my doing things that way as opposed to the way the
Junio has been running the git repository?
The reason Junio does what he does is flexibility.
By merging only individual topics forked from master into next you
can merge those individual topics into master at different points
in time. For example db/fetch-pack has been in next for many weeks
and hasn't yet merged into master, yet jc/am-quiet was forked after
db/fetch-pack started and has already merged into master.
Your way would make jc/am-quiet wait until db/fetch-pack was ready.
That's a big risk in the sense that your tree is "blocked" and even
simple changes are held up by ones that suddenly became a lot more
complex then you originally thought they were going to be.
--
Shawn.
On Tue, Oct 23, 2007 at 12:46:57AM -0400, Shawn O. Pearce wrote:
By merging only individual topics forked from master into next you
can merge those individual topics into master at different points
in time. For example db/fetch-pack has been in next for many weeks
and hasn't yet merged into master, yet jc/am-quiet was forked after
db/fetch-pack started and has already merged into master.
Your way would make jc/am-quiet wait until db/fetch-pack was ready.
That's a big risk in the sense that your tree is "blocked" and even
simple changes are held up by ones that suddenly became a lot more
complex then you originally thought they were going to be.
Yes, true. Alternatively, what I've been doing is that if I wasn't
sure that a particular topic was ready to go to 'master' very shortly
after it went into 'next', I would never let it go into 'next', but
rather keep it in 'pu' (which is OK, because pu is constantly getting
rewound). But I guess the downside of that is you might get fewer
testers for the code, because fewer people are probably tracking and
testing 'pu' as compared to 'next'.
Right?
- Ted
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:43:44
Theodore Tso [off-list ref] wrote:
On Tue, Oct 23, 2007 at 12:46:57AM -0400, Shawn O. Pearce wrote:
quoted
By merging only individual topics forked from master into next you
can merge those individual topics into master at different points
in time. For example db/fetch-pack has been in next for many weeks
and hasn't yet merged into master, yet jc/am-quiet was forked after
db/fetch-pack started and has already merged into master.
Your way would make jc/am-quiet wait until db/fetch-pack was ready.
That's a big risk in the sense that your tree is "blocked" and even
simple changes are held up by ones that suddenly became a lot more
complex then you originally thought they were going to be.
Yes, true. Alternatively, what I've been doing is that if I wasn't
sure that a particular topic was ready to go to 'master' very shortly
after it went into 'next', I would never let it go into 'next', but
rather keep it in 'pu' (which is OK, because pu is constantly getting
rewound). But I guess the downside of that is you might get fewer
testers for the code, because fewer people are probably tracking and
testing 'pu' as compared to 'next'.
Right?
Yes, that's a good point.
I think in Git part of the reason less people track pu is because
its very volatile. Not because of the rewind policy, but becuase
sometimes the code there doesn't work properly so using it for real
"production" work is pretty risky. On the other hand most of the
code that merges into next has been reasonbly well reviewed and
tested, so following it for "production" work is not as risky.
Junio has in the past proposed rewinding next, especially after a
significant release (e.g. 1.5.3). A bunch of folks (myself included
if I recall correctly) didn't want to do this, as we create topic
branches locally from things in next and sometimes make commits
over them to improve the topic further. But I also make topic
branches for things in pu, so I might as well just shut up and
not complain. :-)
Of course another thought that just came to mind is it is very easy
for me to review next with a
git log -p --reverse origin/next..build-next
just before merging it into my build branch and compiling it locally.
If next rewound frequently (as pu does) this would be more difficult.
--
Shawn.
On Tue, Oct 23, 2007 at 01:07:26AM -0400, Shawn O. Pearce wrote:
Junio has in the past proposed rewinding next, especially after a
significant release (e.g. 1.5.3).
Hmm, yes. I think I'd want to rewind next after a while; the thought
of next drifting hundreds or thousands of commits away from master
just gives me the heebee-jeebies. I'm sure it mostly works, but it
just feels wrong. :-)
A bunch of folks (myself included if I recall correctly) didn't want
to do this, as we create topic branches locally from things in next
and sometimes make commits over them to improve the topic further.
I guess I don't see why this would be a hardship; would a quick rebase
on the topic branches more or less take care of the problem?
I guess that brings up another question; I've been regularly rebasing
the topics branches as master and next advances... probably more out
of superstition than anything else. Is that a bad idea for any reason?
Hmm... I guess some of this would be really good to get into the Howto
section of the user guide when talking about git workflows!
- Ted
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:43:44
Theodore Tso [off-list ref] wrote:
On Tue, Oct 23, 2007 at 01:07:26AM -0400, Shawn O. Pearce wrote:
quoted
Junio has in the past proposed rewinding next, especially after a
significant release (e.g. 1.5.3).
Hmm, yes. I think I'd want to rewind next after a while; the thought
of next drifting hundreds or thousands of commits away from master
just gives me the heebee-jeebies. I'm sure it mostly works, but it
just feels wrong. :-)
There's been a couple of times in git history where Junio has basically
done this to whack next back into line:
git checkout next
git diff next master | git apply --index
git commit -m "Whack next back in line"
Because we've found a change or two lurking in there that shouldn't
have been there after a while. I think it was related to a merge
conflict that happened in next but didn't in master or something
like that. But usually this difference exists as there's usually
always something cooking in next.
quoted
A bunch of folks (myself included if I recall correctly) didn't want
to do this, as we create topic branches locally from things in next
and sometimes make commits over them to improve the topic further.
I guess I don't see why this would be a hardship; would a quick rebase
on the topic branches more or less take care of the problem?
Yes. But you need the prior value of the branch so you can do
something easy like:
git checkout yourtopic
git rebase --onto $newtopic $oldtopic
which means you probably need to look through the logs for not just
pu but also pu@{1}. A script to break out the topic branches from
pu post fetch and store them as proper tracking branches would make
this easier, but that much. If you plan ahead you can save that
$oldtopic point so you can do something like this:
git log pu ; # find $newtopic
git checkout yourtopic
git rebase --onto $newtopic base-yourtopic
git tag -f base-yourtopic $newtopic
I guess that brings up another question; I've been regularly rebasing
the topics branches as master and next advances... probably more out
of superstition than anything else. Is that a bad idea for any reason?
It keeps the history shorter in gitk. But otherwise it isn't bad.
Unless you are running into a lot of conflicts every time you rebase
and its wasting your time. ;-)
I prefer to rebase the topics until they've merged to an integration
branch that doesn't rewind (e.g. master or next in git.git).
That way they have the shortest line possible in gitk between the
final merge and the start point.
There are good reasons why there's an "author" and a "committer"
field in commits. Rebasing will change the committer field's
timestamp, but not the author field. And author comes from the
email, to preserve the original date of development.
Hmm... I guess some of this would be really good to get into the Howto
section of the user guide when talking about git workflows!
Yea, I think so too. We've adopted this model in git.git because
it works for our community. A lot of other communities aren't
too far away, as we have a lot of crossover in members. E.g. we
learned a lot from the kernel community.
--
Shawn.
On Tue, Oct 23, 2007 at 01:42:38AM -0400, Shawn O. Pearce wrote:
Yes. But you need the prior value of the branch so you can do
something easy like:
git checkout yourtopic
git rebase --onto $newtopic $oldtopic
which means you probably need to look through the logs for not just
pu but also pu@{1}. A script to break out the topic branches from
pu post fetch and store them as proper tracking branches would make
this easier, but that much. If you plan ahead you can save that
$oldtopic point so you can do something like this:
git log pu ; # find $newtopic
git checkout yourtopic
git rebase --onto $newtopic base-yourtopic
git tag -f base-yourtopic $newtopic
Yeah, I had thought about writing a little script that would take my
project's topic branches, and then push them out into the public
repository under topics/ad/extents-testcases or
topics/tt/badblocks-cleanup. That would make it easy to find the head
of your topic, and once you find that, the base of your-topic isn't
that hard to find, since it would just be the result of "git-rev-list
topic ^master | tail -1".
One of the reasons I was thinking the above is because most of the
patches are coming into my end as emailed patch series, and I end up
tweaking them a lot as I carry them around in the topics branch. So
if other people want to see what I've done to a branch after I've done
a git rebase --interactive, it's easier if they can get access to the
individual topics branch, so they can extract out the patch series
while it's being tweaked by me (and possibly others).
This is probably because my view of git has been colored by kernel
community practices, where patches are normally perfected and get
rebased a lot (normally in a sub-maintainer or maintainer's tree)
before they get pushed to Linus, and in my mental model a topic branch
represents the maintainer's git tree in the central repository.
The extreme end of this would be the classical BitKeeper model, where
Larry McVoy once argued to me that he didn't like history to *ever* be
rewound/rewritten, since not only did this interfere with other people
trees once they had been pushed, but it causes development history to
be lost, which is always valuable. (Of course, in the end he did
write "bk fold", which squashes the last N commits into 1, mainly due
to customer pressure.) The kernel viewpoint is to rebase all the
time, because the history is so huge that we don't *want* to see the
development history of the rough drafts of features before they get
merged into mainline.
It keeps the history shorter in gitk. But otherwise it isn't bad.
Unless you are running into a lot of conflicts every time you rebase
and its wasting your time. ;-)
It sounds like what you are saying here is that the git.git tree takes
a viewpoint which is slightly between the extremes of the kernel model
(which does involve resolving rebasing a lot and resolving lots of
conflicts, but heh, that's not Linus's problem, that's been pushed out
to the leafs of the developer community, and besides, it strongly
encourages topics to get merged into the mainline fast), and the
classical Bitkeeper model, which says that philosophical goodness
means you should keep *all* development history once it enters the BK SCM.
With git.git, we are essentially throwing away development history
while it is in 'pu', but once a commit graduates to 'next', we do keep
the development history forever. The downside to this is that
development 'crud' can build up in next; even if all substantive
commits in 'next' end up graduating to 'master', there will still be
lots of merge commits that will only be in 'next'.
I have an emotional bias which tends to treat that excess history as
toxic waste to be avoided at all costs, but that's probably because
when you have a git tree as huge as the kernel, life is easier if the
history is kept as clean as possible.
Which I suppose is easy enough to do in the git.git model; if you
throw away the 'next' branch and then rewind it so it is forked off of
'master' all of that history essentially gets flushed. The downside
is that people maintaining topics branches which were forked against
the old 'next' will need to do some grotty work to rebase their
patches, so any attempt to rewind next would probably require the
central maintainer to give plenty of notice, and then on the flag day,
save 'next' as 'old-next' before rewinding to allow the other
developers to more easily rebase any private branches they might have.
Hmm, interesting. A lot of this is quite subtle, or at least the
impacts of different choices in the git workflow really didn't become
obvious to me until I started trying I stepped into the central
maintainer role for a project using git!
- Ted
From: Daniel Barkalow <hidden> Date: 2016-06-15 22:43:44
I keep thinking that there should be a better mechanism to use for "pu"
than a branch. Normally what you see in "pu" is a sequential merge of
"next" and a number of topic branches, where the series of merges is
either entirely uninteresting or only interesting in a schematic sense
(that is, it is interesting what topics appear, and in what order, but the
snapshot of each topic's head when it got merged isn't interesting).
That is, the work which "pu" consists of, and therefore the history is a
sequence of steps, each of which is one or more of: "add this topic",
"update this topic", "remove this topic", "update to a new next". And we
don't keep a record of this history, but it's not what's discarded by
rewinding anyway.
I think that, if we actually care about this sort of thing, we'd want to
make "pu" a series of commits, each with the previous "pu" as the sole
parent, with a series object given in a new header. The series object
would start with a "next" commit, and then list the topics merged by name
and head-as-merged. Of course, the "pu" commits would contain the
resulting tree as normal, so that people without a git that understands
this would see "pu" as consisting of a straight line of commits, each of
which simply shows the net effect of the changes. Or something like that.
I suppose "pu" could also be represented as a superproject where each
subproject is "next" or a topic branch, if we really want to avoid
introducing new objects, but that seems unweildy somehow.
Is this worth doing? It might be; I bet it would make debugging -mm a
whole lot nicer. (First bisect through -mm to find the action Andrew took
that accepted the breakage, then bisect the history within that action.) I
bet the status quo is a real pain when the feature that broke is only in
-mm and later in Andrew's list than the tree whose change triggered the
failure (i.e., -mm4 works; -mm5 doesn't work, and everything in -mm5 is
either broken or untestable).
And, of course, "origin/pu[db/builtin-fetch]" would be an easy thing to
build into git, and it could even generate extra magic, where it knows
what the topic was last rebased on, so git could lead people through
"rebase everything in a pu collection on the new base for the collection"
and "make my local topic branch agree with my topic branch as present in
origin/pu".
-Daniel
*This .sig left intentionally blank*