Re: Trying to sync two svn repositories with git-svn (repost)

On Wed, Apr 29, 2009 at 12:01 PM, Josef Wolf [off-list ref] wrote:

On Tue, Apr 28, 2009 at 11:19:51PM -0400, Avery Pennarun wrote:

quoted

Okay, I think I'm following you.  And I think the difficulty of your
solution will depend on how important it is to cherry-pick each
individual commit from each repo vs. just merging everything as a
batch.

I've already done the cherry-picking.

So you're saying that from now on, *all* changes from *both* branches
need to be integrated in both directions?  If so, you're done with
cherry-picking.  If not, you're not.

 Basically, I've done this:

 # first, move patches from second-svn to first-svn
 git checkout first-svn
 git svn rebase
 git cherry-pick sha1 # repeat as needed
 git merge -s ours second-svn

 # Now, the other way around
 git checkout second-svn
 git svn rebase
 git cherry-pick sha1 # repeat as needed
 git merge -s ours first-svn

The first git-svn-rebase after the merge causes all the (already picked)
commits from the other branch to be pulled into the current branch.
Adding the --no-ff option does not help.  Omitting the cherry-picking
does not help, either.

Hmm, I don't see any 'git svn dcommit' in there.  The steps I listed
referred to dcommit, but explicitly left out calls to 'git svn
rebase'.

I think it's likely that your problems stem from this. The git svn
documentation refers to the 'git svn rebase' operation a lot, but it's
only really useful for one thing: linearizing history to make it look
(to svn) like git was never involved. This is handy for people who
want to use git at work without their boss knowing about it, but it
*loses information* and will mess up future merges.

In general, 'git svn rebase' should be avoided for all the same
reasons that 'git rebase' should be avoided.  They're both great when
used carefully, but they shouldn't be your main day-to-day activity.
Unfortunately git svn encourages you to use rebase in your day-to-day
activity... but the workflow I'm talking about actually avoids this
problem completely.  What you want most of the time is really just
'git svn fetch'. and 'git svn dcommit'.

I think I was also a bit too offhand in my previous email when
expanding my suggestion to work with multiple svn hosts.  The clearest
way to do this is with three branches:

  - 1 remote branch: git-svn-1
  - 1 remote branch: git-svn-2
  - 1 local branch: master

So the steps are something like this.  (Again, WARNING: I'm not
running these as I type them, so I could be screwing up just about
anything.)

Getting started:

  git checkout master
  ... Use 'git svn fetch' to update git-svn-1 and git-svn-2 ...
  ... git merge/cherry-pick what you want from git-svn-1 and
git-svn-2.  ALWAYS use --no-ff if using git merge
  git merge --no-ff -s ours git-svn-1
  git merge --no-ff -s ours git-svn-2
    # now master has everything from both svn repositories

From now on:

  # Update git-svn-1 with the latest master
  git checkout git-svn-1
    # since git-svn-1 is a remote branch, you now have a detached HEAD
  git merge --no-ff master
  git svn dcommit

  # Update git-svn-2 with the latest master
  git checkout git-svn-2
    # since git-svn-2 is a remote branch, you now have a *different*
detached HEAD
  git merge --no-ff master
  git svn dcommit

  # Update master with the latest svn
  git checkout master
    # HEAD is now attached to master
  git merge --no-ff git-svn-1
  git merge --no-ff git-svn-2
    # no need for '-s ours' in the above merge, as no rebasing means
no merge history was lost

quoted

At Versabanq, we're using git for a bunch of stuff including our
autobuilder (http://github.com/apenwarr/gitbuilder) and my own

Interesting project.  One question: the README mentions that the
gitbuilder as capable to update itself.  But I can not actually see
this functionality in the scripts.  Is that just a typo or am I missing
something?

I guess you're reading the line that says, "Now that your gitbuilder
is working, you probably want to have it continue
to update itself automatically."  This is actually talking about
*running* itself automatically, as in "upating the build results to
the latest copy of your project."  I can see how it's a very unclear
word to use there.  Thanks for the feedback.

quoted

As long as you "git config merge.summary true" (to make the merge
commit list all the commits it's merging)

How does this option influence the merge operation?  Or is this meant
to provide additional information to the person who does the next merge?

When you *merge* (as opposed to rebase or cherry-pick) into an svn
branch, you only create a *single* svn commit that contains *all* the
changes.  The above config setting just makes the merge commit contain
a list of all the commits it contains.

quoted

Now, your problem is a little more complex, because it sounds like
people are checking in two types of things on both sides: private
things and public things.  So if you want *only* the private things,

I want both.  The difference is that I (usually) want to pull the public
things unmodified, while I want to generalize/localize the private things.
So when merging the private part, I would not want to pick the specific
entries.  But I still want to pick the _structure_ (possibly removing or
modifying the localized entries).

If you're going to be mangling things so thoroughly, then you might
just have to resort to cherry-picking everything one by one from one
branch to the other.  It doesn't sound very fun, but if other people
are being so uncooperative by mixing public and private stuff in their
repositories, there's no way I can see to automate it anyhow.

If you're using cherry-pick for everything, there's no reason to use
tricks like 'merge -s ours'.  Just leave out the merging entirely and
don't pretend that what you're doing is merging; it isn't.  (You still
don't need 'git svn rebase' for anything.  Just checkout the branch
you want to change, cherry-pick stuff into it, and 'git svn dcommit'
if appropriate.)

If the situation ever changes, you can always do one last 'merge -s
ours' and mark the histories as combined.  Then future merges will
bring in any future changes.

Good luck.

Avery