Re: git-svn set-tree bug

Eric Wong [off-list ref] writes:

If dcommit detects a merge commit when doing rev-list When looking at
commit objects, is it safe to assume that the first parent is always the
"mainline" and that parents after it are the ones to merge from?

So if I saw:

commit $X
parent $A
parent $B

I'd basically do:
  reset --hard $A
  merge --squash $B

And resulting in $C which would have the same tree as $X,
then, when dcommit-ting, $D would be created with two parents:
  $D~1 (svn), $B (git), but not $A

I am not sure what you mean by "mainline", but I assume that you
mean "SVN is the main and we are tracking it while taking
advantage of more efficient and merge-capable git in guerrilla
fashion".  Because the tip of the current branch is what the
user is pushing back to SVN via dcommit, I would say it is safe
to assume that the first parent of such a merge is the line that
corresponds to the SVN branch you are keeping track.

Junio C Hamano [off-list ref] wrote:

Eric Wong [off-list ref] writes:

quoted

If dcommit detects a merge commit when doing rev-list When looking at
commit objects, is it safe to assume that the first parent is always the
"mainline" and that parents after it are the ones to merge from?

So if I saw:

commit $X
parent $A
parent $B

I'd basically do:
  reset --hard $A
  merge --squash $B

And resulting in $C which would have the same tree as $X,
then, when dcommit-ting, $D would be created with two parents:
  $D~1 (svn), $B (git), but not $A

I am not sure what you mean by "mainline", but I assume that you
mean "SVN is the main and we are tracking it while taking
advantage of more efficient and merge-capable git in guerrilla
fashion".  Because the tip of the current branch is what the
user is pushing back to SVN via dcommit, I would say it is safe
to assume that the first parent of such a merge is the line that
corresponds to the SVN branch you are keeping track.

Yes, "mainline" meaning the history that would be committed to SVN if
history were linear.

I've gotten the following patch working for Joakim's second test script
(with dcommit before merge).  However, without the dcommit before merge
in the first test script, git-svn has trouble figuring out which history
to follow.  It'll take more work to figure out what to do in this
situation, and how to deal with more complex history...

Subject: git-svn: Allow dcommit to handle certain single-parent merge commits

This only works if a merge is the first commit to be committed
in a chain of commits.
---
 git-svn.perl |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 0ae8d70..6b3e021 100755
--- a/git-svn.perl
+++ b/git-svn.perl

@@ -403,6 +403,9 @@ sub cmd_dcommit {
 			                svn_path => '');
 			if (!SVN::Git::Editor->new(\%ed_opts)->apply_diff) {
 				print "No changes\n$d~1 == $d\n";
+			} elsif (my $merge_parent = verify_ref("$d^2")) {
+				$gs->{inject_parents}->{$last_rev} =
+				                                 $merge_parent;
 			}
 		}
 	}

-- 
Eric Wong

On Tue, 2007-06-12 at 01:39 -0700, Eric Wong wrote:

Junio C Hamano [off-list ref] wrote:

quoted

Eric Wong [off-list ref] writes:

quoted

If dcommit detects a merge commit when doing rev-list When looking at
commit objects, is it safe to assume that the first parent is always the
"mainline" and that parents after it are the ones to merge from?

So if I saw:

commit $X
parent $A
parent $B

I'd basically do:
  reset --hard $A
  merge --squash $B

And resulting in $C which would have the same tree as $X,
then, when dcommit-ting, $D would be created with two parents:
  $D~1 (svn), $B (git), but not $A

I am not sure what you mean by "mainline", but I assume that you
mean "SVN is the main and we are tracking it while taking
advantage of more efficient and merge-capable git in guerrilla
fashion".  Because the tip of the current branch is what the
user is pushing back to SVN via dcommit, I would say it is safe
to assume that the first parent of such a merge is the line that
corresponds to the SVN branch you are keeping track.

Yes, "mainline" meaning the history that would be committed to SVN if
history were linear.

I've gotten the following patch working for Joakim's second test script
(with dcommit before merge).  However, without the dcommit before merge
in the first test script, git-svn has trouble figuring out which history
to follow.  It'll take more work to figure out what to do in this
situation, and how to deal with more complex history...

Subject: git-svn: Allow dcommit to handle certain single-parent merge commits

This only works if a merge is the first commit to be committed
in a chain of commits.

[SNIP patch]

Nice!, now I get to keep the merge between the "svn" and the "merge" branch. The parents are swapped though:
before last dcommit:
  Parent: b31cef1d3c6655441854ea8649359f0fc27f3e87 (friend)
  Parent: ed95b698c2e3336d387fed3763b213b3b90ebf4e (add some stuff)
  Branch: svn
  Follows: 
  Precedes: 

    Merge branch 'merge' into svn

after dcommit:

Parent: ed95b698c2e3336d387fed3763b213b3b90ebf4e (add some stuff)
Parent: b31cef1d3c6655441854ea8649359f0fc27f3e87 (friend)
Branches: svn, remotes/trunk
Follows: 
Precedes: 

    Merge branch 'merge' into svn
    
    
    git-svn-id: file:////usr/local/src/tst-git-svn/mysvnrepo/trunk@3 1585b9b0-b13 ....


Will this also work for merging stuff from latest u-boot?

I am doing dev. on my own u-boot branch and from time to time I want
to merge in the latest from WD tree, then dcommit that merge. Later
I want repeat that cycle.

I have a SVN repo with my changes in it and I have grafted the beginning of
that tree into a clone of WDs tree.

 Jocke

Eric Wong wrote:

Yes, "mainline" meaning the history that would be committed to SVN if
history were linear.
  

I think the first parent is always the right one to follow. The only 
time you won't hit a git-svn revision is if the user is trying to commit 
a branch that is not originally derived from a git-svn branch, and IMO 
that's something that git-svn is perfectly justified in refusing to do. 
Also, the default "git log" output will follow the first parent; users 
who run that are going to have a natural expectation that the subsequent 
commits will be merged into the most recent git-svn revision as shown by 
that tool.

This only works if a merge is the first commit to be committed
in a chain of commits.
  

As for the more complex case of a chain of commits with a merge in the 
middle, IMO walking along the chain of first parents until you hit a 
git-svn revision, then proceeding forward in time from there rewriting 
parents as you've described, is always going to be the right thing to 
do. Or at least, all the use cases I can think of seem to be correctly 
covered by that approach. Can someone come up with counterexamples?

This is great -- I'm looking forward to ditching my hackish merge script!

At the risk of getting ahead of myself, here's one more thought: in the 
case where a merge's parents are all git-svn revisions -- that is, where 
the user is using git to merge svn branches -- I wonder if it makes 
sense to optionally record that merge somehow in the commit comment on 
the svn side. I think that could be made relatively human-readable so as 
not to be too obnoxious for people browsing the svn history. That way 
someone pulling down a fresh git-svn clone of the svn repo could get a 
nice clean history with the merges represented properly in the git 
revision history.

That's justifiable in another way too: the autogenerated comments on git 
merge commits won't really make much sense over on the svn side, where 
merges are thought of in terms of revision ranges. So replacing the 
git-specific merge message with an svn-specific one doesn't seem 
unreasonable to me. (Again, optionally.) And in cases where the user has 
supplied his own merge comment on the git side, annotating it with the 
additional git-svn metadata seems reasonable to me. We are already fine 
with the git-side comments having a line of git-svn metadata, after all.

Most of the svn-side merge comments in my company's repo look like 
either "svn merge -r12345:67890 mybranch" (where the developer wants to 
make the merge's inputs very explicit to avoid any confusion) or "Merge 
revisions 12345 through 67890 from mybranch", occasionally surrounded by 
some explanatory text. If git-svn replaced the canned git merge message 
with a canned message like one of those, people wouldn't be able to tell 
I'd used git-svn instead of svn to do the merge.

-Steve

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

 git-svn.perl                     |   72 +++++++++++++++++++++++++++----
 t/t9114-git-svn-dcommit-merge.sh |   89 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 152 insertions(+), 9 deletions(-)
 create mode 100755 t/t9114-git-svn-dcommit-merge.sh

diff --git a/git-svn.perl b/git-svn.perl
index 0ae8d70..4290676 100755
--- a/git-svn.perl
+++ b/git-svn.perl

@@ -372,16 +372,9 @@ sub cmd_dcommit {
 		die "Unable to determine upstream SVN information from ",
 		    "$head history\n";
 	}
-	my $c = $refs[-1];
 	my $last_rev;
-	foreach my $d (@refs) {
-		if (!verify_ref("$d~1")) {
-			fatal "Commit $d\n",
-			      "has no parent commit, and therefore ",
-			      "nothing to diff against.\n",
-			      "You should be working from a repository ",
-			      "originally created by git-svn\n";
-		}
+	my ($linear_refs, $parents) = linearize_history($gs, \@refs);
+	foreach my $d (@$linear_refs) {
 		unless (defined $last_rev) {
 			(undef, $last_rev, undef) = cmt_metadata("$d~1");
 			unless (defined $last_rev) {

@@ -403,6 +396,9 @@ sub cmd_dcommit {
 			                svn_path => '');
 			if (!SVN::Git::Editor->new(\%ed_opts)->apply_diff) {
 				print "No changes\n$d~1 == $d\n";
+			} elsif ($parents->{$d} && @{$parents->{$d}}) {
+				$gs->{inject_parents_dcommit}->{$last_rev} =
+				                               $parents->{$d};
 			}
 		}
 	}

@@ -821,6 +817,59 @@ sub working_head_info {
 	(undef, undef, undef, undef);
 }
 
+sub read_commit_parents {
+	my ($parents, $c) = @_;
+	my ($fh, $ctx) = command_output_pipe(qw/cat-file commit/, $c);
+	while (<$fh>) {
+		chomp;
+		last if '';
+		/^parent ($sha1)/ or next;
+		push @{$parents->{$c}}, $1;
+	}
+	close $fh; # break the pipe
+}
+
+sub linearize_history {
+	my ($gs, $refs) = @_;
+	my %parents;
+	foreach my $c (@$refs) {
+		read_commit_parents(\%parents, $c);
+	}
+
+	my @linear_refs;
+	my %skip = ();
+	my $last_svn_commit = $gs->last_commit;
+	foreach my $c (reverse @$refs) {
+		next if $c eq $last_svn_commit;
+		last if $skip{$c};
+
+		unshift @linear_refs, $c;
+		$skip{$c} = 1;
+
+		# we only want the first parent to diff against for linear
+		# history, we save the rest to inject when we finalize the
+		# svn commit
+		my $fp_a = verify_ref("$c~1");
+		my $fp_b = shift @{$parents{$c}} if $parents{$c};
+		if (!$fp_a || !$fp_b) {
+			die "Commit $c\n",
+			    "has no parent commit, and therefore ",
+			    "nothing to diff against.\n",
+			    "You should be working from a repository ",
+			    "originally created by git-svn\n";
+		}
+		if ($fp_a ne $fp_b) {
+			die "$c~1 = $fp_a, however parsing commit $c ",
+			    "revealed that:\n$c~1 = $fp_b\nBUG!\n";
+		}
+
+		foreach my $p (@{$parents{$c}}) {
+			$skip{$p} = 1;
+		}
+	}
+	(\@linear_refs, \%parents);
+}
+
 package Git::SVN;
 use strict;
 use warnings;

@@ -1541,6 +1590,11 @@ sub get_commit_parents {
 	if (my $cur = ::verify_ref($self->refname.'^0')) {
 		push @tmp, $cur;
 	}
+	if (my $ipd = $self->{inject_parents_dcommit}) {
+		if (my $commit = delete $ipd->{$log_entry->{revision}}) {
+			push @tmp, @$commit;
+		}
+	}
 	push @tmp, $_ foreach (@{$log_entry->{parents}}, @tmp);
 	while (my $p = shift @tmp) {
 		next if $seen{$p};

diff --git a/t/t9114-git-svn-dcommit-merge.sh b/t/t9114-git-svn-dcommit-merge.sh
new file mode 100755
index 0000000..d6ca955
--- /dev/null
+++ b/t/t9114-git-svn-dcommit-merge.sh

@@ -0,0 +1,89 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Eric Wong
+# Based on a script by Joakim Tjernlund <joakim.tjernlund@transmode.se>
+
+test_description='git-svn dcommit handles merges'
+
+. ./lib-git-svn.sh
+
+big_text_block () {
+cat << EOF
+#
+# (C) Copyright 2000 - 2005
+# Wolfgang Denk, DENX Software Engineering, wd@denx.de.
+#
+# See file CREDITS for list of people who contributed to this
+# project.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation; either version 2 of
+# the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+# MA 02111-1307 USA
+#
+EOF
+}
+
+test_expect_success 'setup svn repository' "
+	svn co $svnrepo mysvnwork &&
+	mkdir -p mysvnwork/trunk &&
+	cd mysvnwork &&
+		big_text_block >> trunk/README &&
+		svn add trunk &&
+		svn ci -m 'first commit' trunk &&
+		cd ..
+	"
+
+test_expect_success 'setup git mirror and merge' "
+	git svn init $svnrepo -t tags -T trunk -b branches &&
+	git svn fetch &&
+	git checkout --track -b svn remotes/trunk &&
+	git checkout -b merge &&
+	echo new file > new_file &&
+	git add new_file &&
+	git commit -a -m 'New file' &&
+	echo hello >> README &&
+	git commit -a -m 'hello' &&
+	echo add some stuff >> new_file &&
+	git commit -a -m 'add some stuff' &&
+	git checkout svn &&
+	mv -f README tmp &&
+	echo friend > README &&
+	cat tmp >> README &&
+	git commit -a -m 'friend' &&
+	git pull . merge
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify pre-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_expect_success 'git svn dcommit merges' "
+	git svn dcommit
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify post-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn\` = \
+	     x\`git rev-parse --verify refs/remotes/trunk \` &&
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_done

-- 
Eric Wong

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?


Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
can I attach the svn tree to git u-boot tree without using a graft?

I want to be able to recreate my own git repo by cloning the orginal u-boot
repo and the svn repo.

 Jocke

On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:

quoted

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?


Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
can I attach the svn tree to git u-boot tree without using a graft?

I want to be able to recreate my own git repo by cloning the orginal u-boot
repo and the svn repo.

 Jocke

Tried using --no-metadata(git svn clone --no-metadata) in my little test
script I sent earlier and got
  "Unable to determine upstream SVN information from HEAD history"
when dcommiting, -i trunk didn't help either.

It is not entierly clear to me what --no-metadata means to me.
Does git-svn still rewrite commits?
I can't rebuild rev_db file, if lost, but I guess I could still
do a new git-svn clone and restore my repo? I guess I lose something
if I do that but what?
Also don't really understand why git-svn log doesn't work, can't it get
that info from the svn repo?

 Jocke

Eric Wong wrote:

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.
  

I poked at this some tonight with an eye toward the use case of using 
git to merge svn branches. I ran into one inconvenience and one bug. 
I'll try playing with the "use lots of git branches to develop on one 
svn branch" use case some too, but for now, here are the notes I took, 
along with the commands if anyone wants to reproduce what I did. 
Hopefully this won't be too annoying to read. The bug is near the bottom.

svn repo with a trunk and a branch, each with changes (no conflicts at 
first, keep it simple to start)

$ svnadmin create svnrepo
$ svn co file://`pwd`/svnrepo svnclient
$ cd svnclient
$ mkdir trunk tags branches
$ echo test file number 1 > trunk/testfile1
$ echo test file number 2 > trunk/testfile2
$ svn add *
$ svn commit -m "initial commit"
$ echo trunk change 1 >> trunk/testfile1
$ svn commit -m "trunk change 1"
$ echo trunk change 2 >> trunk/testfile2
$ svn commit -m "trunk change 2"
$ svn cp trunk branches/mybranch
$ svn commit -m "make a branch"
$ echo trunk change 3 >> trunk/testfile1
$ svn commit -m "post-branch change in trunk"
$ echo branch change 1 >> branches/mybranch/testfile2
$ svn commit -m "change in branch"

git-svn clone of this dinky repo

$ cd ..
$ git-svn clone --trunk=trunk --branches=branches --tags=tags 
file://`pwd`/svnrepo gitclone
$ cd gitclone

Try to merge trunk change into branch using git

$ git reset --hard mybranch
$ git merge trunk

Conflicts! what's going on?

$ gitk --all

Aha, looks like git-svn guessed wrong about where I made the branch; it 
thinks the branch comes from the initial rev. Easy enough to hack 
around, but might be nice to be able to do this using git-svn's history 
rewriting rather than a grafts file.

$ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > .git/info/grafts
$ git reset --hard mybranch
$ git merge trunk

No conflicts now. Let's see what git-svn thinks it should do

$ git-svn dcommit -n

Looks like the right diff

$ git-svn dcommit

Refresh gitk display. Looks good, the new revision is a merge with the 
right parents. Let's check it out in svn land

$ cd ../svnclient
$ svn up
$ cat branches/mybranch/testfile1

Yep, the trunk change is there, nice! Now for a couple more revs with a 
conflict.

$ echo post-merge trunk change >> trunk/testfile1
$ svn commit -m "trunk change after merge"
$ echo post-merge conflicting change >> trunk/testfile2
$ svn commit -m "trunk change with conflict"
$ cd ../gitclone
$ git-svn fetch
$ git merge -m "change with conflict" trunk

Conflict, as expected

$ vi testfile2
$ git add testfile2
$ git commit
$ git-svn dcommit
Transaction is out of date: Out of date: '/trunk/testfile1' in 
transaction '9-1' at /Users/koreth/git/git-svn line 398

Hmm, this merge was in mybranch, not in trunk

$ git log --first-parent

Yes, the most recent commit with a git-svn-id line has a mybranch URL. 
So why is it complaining about a trunk file being out of date?

My experimentation pretty much ended there (I tried a few things to 
clear the error up, but none of them helped.)

This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
MacPorts. I used the git-svn from the "pu" branch since it had this 
patch and all the recent fixes.

Let me know if you need more details. Hope this is helpful.

-Steve

Joakim Tjernlund [off-list ref] wrote:

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:

quoted

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?

Use a do_switch()-enabled SVN to avoid do_update().  do_update will
redownload everything.  I have patched 1.4.3 debian packages with source
and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
have fixed the bindings, but 1.4.3 claimed the same thing, too...
Confirmation of it working in SVN 1.4.4 would be nice.

Lastly, when adding the above u-boot svn repo into a fresh u-boot
clone from WD, can I attach the svn tree to git u-boot tree without
using a graft?

Not with the current version.  The 1.5.0 (or previous, I forget) allowed
forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
nobody was using it, and it would be difficult to get working since
fetch can now works on multiple trees and the same revision numbers can
appear in multiple trees.

I want to be able to recreate my own git repo by cloning the orginal
u-boot repo and the svn repo.

-- 
Eric Wong

Joakim Tjernlund [off-list ref] wrote:

On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:

quoted

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:

quoted

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?


Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
can I attach the svn tree to git u-boot tree without using a graft?

I want to be able to recreate my own git repo by cloning the orginal u-boot
repo and the svn repo.

 Jocke

Tried using --no-metadata(git svn clone --no-metadata) in my little test
script I sent earlier and got
  "Unable to determine upstream SVN information from HEAD history"
when dcommiting, -i trunk didn't help either.

It is not entierly clear to me what --no-metadata means to me.
Does git-svn still rewrite commits?

--no-metadata is really only useful for people doing one-shot imports
and abandoning SVN.  It leaves out the git-svn-id: lines at the bottom
of commit messages, but still sets the committer/author names/email/date
to what is in the SVN repository.

I can't rebuild rev_db file, if lost, but I guess I could still
do a new git-svn clone and restore my repo? I guess I lose something
if I do that but what?

If you lose your rev_db file with no-metadata, you'll have to redo the
git-svn clone

Also don't really understand why git-svn log doesn't work, can't it get
that info from the svn repo?

Getting git-svn log working with --no-metadata would require radically
different code.  dcommit would be very different, too.  So yes, they
don't work because I'm lazy.

-- 
Eric Wong

-----Original Message-----
From: Eric Wong [mailto:normalperson@yhbt.net] 
Sent: den 20 juni 2007 08:56
To: Joakim Tjernlund
Cc: Junio C Hamano; Steven Grimm; git@vger.kernel.org
Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
merge information

Joakim Tjernlund [off-list ref] wrote:

quoted

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:

quoted

dcommit will still rewrite the HEAD commit and the 

history of the first

quoted

quoted

parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are 

rewritten to SVN

quoted

quoted

with dcommit.  Commits written to SVN will still not have 

any merge

quoted

quoted

information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <redacted>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced 

performance for

quoted

quoted

   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be 

appreciated.  Be very

quoted

quoted

   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the 

diffs are sane.

quoted

quoted

  Thanks

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  

file:///usr/local/src/tst-git-svn/svn-uboot-repo

quoted

  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: 

file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => 
file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-ubo
ot-1.1.4, 2

quoted

  Found branch parent: (tags/snap-uboot-1.1.4) 

81eef14963597cc99ba375f52e6d0b3bc09e25f8

quoted

  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?

Use a do_switch()-enabled SVN to avoid do_update().  do_update will
redownload everything.  I have patched 1.4.3 debian packages 
with source
and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
have fixed the bindings, but 1.4.3 claimed the same thing, too...
Confirmation of it working in SVN 1.4.4 would be nice.

Confirmed as requested, I installed 1.4.4(Gentoo) an ran the
same test case. Now I see "Following parent with do_switch"
instead and it is almost instant. It felt though that
git-svn was somewhat slower importing large diffs.

 Jocke

quoted

Lastly, when adding the above u-boot svn repo into a fresh u-boot
clone from WD, can I attach the svn tree to git u-boot tree without
using a graft?

Not with the current version.  The 1.5.0 (or previous, I 
forget) allowed
forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
nobody was using it, and it would be difficult to get working since
fetch can now works on multiple trees and the same revision 
numbers can
appear in multiple trees.

If you reconsider this, please let me know.

 Jocke

-----Original Message-----
From: Steven Grimm [mailto:koreth@midwinter.com] 
Sent: den 14 juni 2007 08:31
To: Eric Wong
Cc: Junio C Hamano; Joakim Tjernlund; git@vger.kernel.org
Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
merge information

Eric Wong wrote:

quoted

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs 

are sane.

quoted

I poked at this some tonight with an eye toward the use case of using 
git to merge svn branches. I ran into one inconvenience and one bug. 
I'll try playing with the "use lots of git branches to develop on one 
svn branch" use case some too, but for now, here are the 
notes I took, 
along with the commands if anyone wants to reproduce what I did. 
Hopefully this won't be too annoying to read. The bug is near 
the bottom.

svn repo with a trunk and a branch, each with changes (no 
conflicts at 
first, keep it simple to start)

$ svnadmin create svnrepo
$ svn co file://`pwd`/svnrepo svnclient
$ cd svnclient
$ mkdir trunk tags branches
$ echo test file number 1 > trunk/testfile1
$ echo test file number 2 > trunk/testfile2
$ svn add *
$ svn commit -m "initial commit"
$ echo trunk change 1 >> trunk/testfile1
$ svn commit -m "trunk change 1"
$ echo trunk change 2 >> trunk/testfile2
$ svn commit -m "trunk change 2"
$ svn cp trunk branches/mybranch
$ svn commit -m "make a branch"
$ echo trunk change 3 >> trunk/testfile1
$ svn commit -m "post-branch change in trunk"
$ echo branch change 1 >> branches/mybranch/testfile2
$ svn commit -m "change in branch"

git-svn clone of this dinky repo

$ cd ..
$ git-svn clone --trunk=trunk --branches=branches --tags=tags 
file://`pwd`/svnrepo gitclone
$ cd gitclone

Try to merge trunk change into branch using git

$ git reset --hard mybranch
$ git merge trunk

Conflicts! what's going on?

$ gitk --all

Aha, looks like git-svn guessed wrong about where I made the 
branch; it 
thinks the branch comes from the initial rev. Easy enough to hack 
around, but might be nice to be able to do this using 
git-svn's history 
rewriting rather than a grafts file.

Yes, that would be nice indeed.

$ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > 
.git/info/grafts
$ git reset --hard mybranch
$ git merge trunk

No conflicts now. Let's see what git-svn thinks it should do

$ git-svn dcommit -n

Looks like the right diff

$ git-svn dcommit

Refresh gitk display. Looks good, the new revision is a merge 
with the 
right parents. Let's check it out in svn land

$ cd ../svnclient
$ svn up
$ cat branches/mybranch/testfile1

Yep, the trunk change is there, nice! Now for a couple more 
revs with a 
conflict.

$ echo post-merge trunk change >> trunk/testfile1
$ svn commit -m "trunk change after merge"
$ echo post-merge conflicting change >> trunk/testfile2
$ svn commit -m "trunk change with conflict"
$ cd ../gitclone
$ git-svn fetch
$ git merge -m "change with conflict" trunk

Conflict, as expected

$ vi testfile2
$ git add testfile2
$ git commit
$ git-svn dcommit
Transaction is out of date: Out of date: '/trunk/testfile1' in 
transaction '9-1' at /Users/koreth/git/git-svn line 398

Maybe this can help?
http://svn.haxx.se/subusers/archive-2005-02/0096.shtml
http://subclipse.tigris.org/faq.html#out-of-date

Hmm, this merge was in mybranch, not in trunk

$ git log --first-parent

Yes, the most recent commit with a git-svn-id line has a 
mybranch URL. 
So why is it complaining about a trunk file being out of date?

My experimentation pretty much ended there (I tried a few things to 
clear the error up, but none of them helped.)

This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
MacPorts. I used the git-svn from the "pu" branch since it had this 
patch and all the recent fixes.

Let me know if you need more details. Hope this is helpful.

-Steve

quoted

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?

Use a do_switch()-enabled SVN to avoid do_update().  do_update will
redownload everything.  I have patched 1.4.3 debian packages with source
and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
have fixed the bindings, but 1.4.3 claimed the same thing, too...
Confirmation of it working in SVN 1.4.4 would be nice.

I upgraded to svn 1.4.4 and it seemed to work OK, but then i noticed
some problems, 
 git clone -t tags -T trunk -b branches svn+ssh://devsrv/svn/TM-uboot
failed(error msg was something with "Malformed data ...")

I decided to go back to 1.4.3 with you patch, switch-editor-perl.diff
applied. Now the error was gone, but the clone operation did use
do_update insted of do_switch. Did I miss someting?

 Jocke