[PATCH 0/4] some git-svnimport improvements

DORMANTno replies

10 messages, 3 authors, 2016-06-15 · open the first message on its own page

[PATCH 0/4] some git-svnimport improvements

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

This patch series teaches git-svnimport about the svn:executable and
svn:ignore Subversion properties (try to guess from their names what
they do!), and lets the user specify a file with username -> Full Name
<email@address> mappings.

I vaguely recall some discussion some time ago about the format of
this kind of username mapping file, but I ended up just picking the
simplest format I could think of since that was the boring part.

[PATCH 2/4] Convert executable flag

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

Convert the svn:executable property to file mode 755 when converting
an SVN repository to GIT.

Signed-off-by: Karl Hasselström <redacted>

---

 git-svnimport.perl |   20 +++++++++++++-------
 1 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/git-svnimport.perl b/git-svnimport.perl
index ee2940f..6603b96 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -112,16 +112,22 @@ sub file {
 		    DIR => File::Spec->tmpdir(), UNLINK => 1);
 
 	print "... $rev $path ...\n" if $opt_v;
-	my $pool = SVN::Pool->new();
-	eval { $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
-	$pool->clear;
+	my (undef, $properties);
+	eval { (undef, $properties)
+		   = $self->{'svn'}->get_file($path,$rev,$fh); };
 	if($@) {
 		return undef if $@ =~ /Attempted to get checksum/;
 		die $@;
 	}
+	my $mode;
+	if (exists $properties->{'svn:executable'}) {
+		$mode = '0755';
+	} else {
+		$mode = '0644';
+	}
 	close ($fh);
 
-	return $name;
+	return ($name, $mode);
 }
 
 package main;
@@ -296,7 +302,7 @@ sub get_file($$$) {
 	my $svnpath = revert_split_path($branch,$path);
 
 	# now get it
-	my $name;
+	my ($name,$mode);
 	if($opt_d) {
 		my($req,$res);
 
@@ -316,8 +322,9 @@ sub get_file($$$) {
 			return undef if $res->code == 301; # directory?
 			die $res->status_line." at $url\n";
 		}
+		$mode = '0644'; # can't obtain mode via direct http request?
 	} else {
-		$name = $svn->file("$svnpath",$rev);
+		($name,$mode) = $svn->file("$svnpath",$rev);
 		return undef unless defined $name;
 	}
 
@@ -331,7 +338,6 @@ sub get_file($$$) {
 	chomp $sha;
 	close $F;
 	unlink $name;
-	my $mode = "0644"; # SV does not seem to store any file modes
 	return [$mode, $sha, $path];
 }
 

[PATCH 1/4] Mention -r in usage summary

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

I added the -r option to git-svnimport some time ago, but forgot to
update the usage summary in the documentation.

Signed-off-by: Karl Hasselström <redacted>

---

 Documentation/git-svnimport.txt |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index 5c543d5..b5c7721 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -10,10 +10,10 @@ git-svnimport - Import a SVN repository 
 SYNOPSIS
 --------
 'git-svnimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ] [ -d | -D ]
-			[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
-			[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
-			[ -s start_chg ] [ -m ] [ -M regex ]
-			<SVN_repository_URL> [ <path> ]
+		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
+		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
+		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
+		<SVN_repository_URL> [ <path> ]
 
 
 DESCRIPTION

[PATCH 4/4] Read author names and emails from a file

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

Read a file with lines on the form

  username User's Full Name [off-list ref]

and use "User's Full Name [off-list ref]" as the GIT author and
committer for Subversion commits made by "username". If encountering a
commit made by a user not in the list, abort.

Signed-off-by: Karl Hasselström <redacted>

---

 Documentation/git-svnimport.txt |   13 ++++++++++++-
 git-svnimport.perl              |   23 ++++++++++++++++++++---
 2 files changed, 32 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index c95ff84..e0e3a5d 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -13,7 +13,8 @@ SYNOPSIS
 		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
-		[ -I <ignorefile_name> ] <SVN_repository_URL> [ <path> ]
+		[ -I <ignorefile_name> ] [ -A <author_file> ]
+		<SVN_repository_URL> [ <path> ]
 
 
 DESCRIPTION
@@ -71,6 +72,16 @@ When importing incrementally, you might 
 	syntaxes are similar enough that using the Subversion patterns
 	directly with "-I .gitignore" will almost always just work.)
 
+-A <author_file>::
+	Read a file with lines on the form
+
+	  username User's Full Name <email@addres.org>
+
+	and use "User's Full Name <email@addres.org>" as the GIT
+	author and committer for Subversion commits made by
+	"username". If encountering a commit made by a user not in the
+	list, abort.
+
 -m::
 	Attempt to detect merges based on the commit message. This option
 	will enable default regexes that try to capture the name source
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 0dd9fab..75ce8e0 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -30,7 +30,7 @@ $SIG{'PIPE'}="IGNORE";
 $ENV{'TZ'}="UTC";
 
 our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
-    $opt_b,$opt_r,$opt_I,$opt_s,$opt_l,$opt_d,$opt_D);
+    $opt_b,$opt_r,$opt_I,$opt_A,$opt_s,$opt_l,$opt_d,$opt_D);
 
 sub usage() {
 	print STDERR <<END;
@@ -38,12 +38,12 @@ Usage: ${\basename $0}     # fetch/updat
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
        [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
-       [-m] [-M regex] [SVN_URL]
+       [-m] [-M regex] [-A author_file] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("b:C:dDhiI:l:mM:o:rs:t:T:uv") or usage();
+getopts("A:b:C:dDhiI:l:mM:o:rs:t:T:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -68,6 +68,19 @@ if ($opt_M) {
 	push (@mergerx, qr/$opt_M/);
 }
 
+our %users = ();
+if ($opt_A) {
+	die "Cannot open $opt_A\n" unless -f $opt_A;
+	open(my $authors,$opt_A);
+	while(<$authors>) {
+		chomp;
+		next unless /^(\S+)\s+(.+?)\s+<(\S+)>$/;
+		(my $user,my $name,my $email) = ($1,$2,$3);
+		$users{$user} = [$name,$email];
+	}
+	close($authors);
+}
+
 select(STDERR); $|=1; select(STDOUT);
 
 
@@ -485,6 +498,10 @@ sub commit {
 
 	if (not defined $author) {
 		$author_name = $author_email = "unknown";
+	} elsif ($opt_A) {
+		die "User $author is not listed in $opt_A\n"
+		    unless exists $users{$author};
+		($author_name,$author_email) = @{$users{$author}};
 	} elsif ($author =~ /^(.*?)\s+<(.*)>$/) {
 		($author_name, $author_email) = ($1, $2);
 	} else {

[PATCH 3/4] Convert the svn:ignore property

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

Put the value of the svn:ignore property in a regular file when
converting a Subversion repository to GIT. The Subversion and GIT
ignore syntaxes are similar enough that it often just works to set the
filename to .gitignore and do nothing else.

Signed-off-by: Karl Hasselström <redacted>

---

 Documentation/git-svnimport.txt |    8 +++++
 git-svnimport.perl              |   60 +++++++++++++++++++++++++++++++++++++--
 2 files changed, 64 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index b5c7721..c95ff84 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -13,7 +13,7 @@ SYNOPSIS
 		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
-		<SVN_repository_URL> [ <path> ]
+		[ -I <ignorefile_name> ] <SVN_repository_URL> [ <path> ]
 
 
 DESCRIPTION
@@ -65,6 +65,12 @@ When importing incrementally, you might 
 	Prepend 'rX: ' to commit messages, where X is the imported
 	subversion revision.
 
+-I <ignorefile_name>::
+	Import the svn:ignore directory property to files with this
+	name in each directory. (The Subversion and GIT ignore
+	syntaxes are similar enough that using the Subversion patterns
+	directly with "-I .gitignore" will almost always just work.)
+
 -m::
 	Attempt to detect merges based on the commit message. This option
 	will enable default regexes that try to capture the name source
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 6603b96..0dd9fab 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -29,19 +29,21 @@ die "Need SVN:Core 1.2.1 or better" if $
 $SIG{'PIPE'}="IGNORE";
 $ENV{'TZ'}="UTC";
 
-our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,$opt_b,$opt_r,$opt_s,$opt_l,$opt_d,$opt_D);
+our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
+    $opt_b,$opt_r,$opt_I,$opt_s,$opt_l,$opt_d,$opt_D);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from SVN
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
-       [-d|-D] [-i] [-u] [-r] [-s start_chg] [-m] [-M regex] [SVN_URL]
+       [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
+       [-m] [-M regex] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("b:C:dDhil:mM:o:rs:t:T:uv") or usage();
+getopts("b:C:dDhiI:l:mM:o:rs:t:T:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -130,6 +132,24 @@ sub file {
 	return ($name, $mode);
 }
 
+sub ignore {
+	my($self,$path,$rev) = @_;
+
+	print "... $rev $path ...\n" if $opt_v;
+	my (undef,undef,$properties)
+	    = $self->{'svn'}->get_dir($path,$rev,undef);
+	if (exists $properties->{'svn:ignore'}) {
+		my ($fh, $name) = tempfile('gitsvn.XXXXXX',
+					   DIR => File::Spec->tmpdir(),
+					   UNLINK => 1);
+		print $fh $properties->{'svn:ignore'};
+		close($fh);
+		return $name;
+	} else {
+		return undef;
+	}
+}
+
 package main;
 use URI;
 
@@ -341,6 +361,34 @@ sub get_file($$$) {
 	return [$mode, $sha, $path];
 }
 
+sub get_ignore($$$$$) {
+	my($new,$old,$rev,$branch,$path) = @_;
+
+	return unless $opt_I;
+	my $svnpath = revert_split_path($branch,$path);
+	my $name = $svn->ignore("$svnpath",$rev);
+	if ($path eq '/') {
+		$path = $opt_I;
+	} else {
+		$path = File::Spec->catfile($path,$opt_I);
+	}
+	if (defined $name) {
+		my $pid = open(my $F, '-|');
+		die $! unless defined $pid;
+		if (!$pid) {
+			exec("git-hash-object", "-w", $name)
+			    or die "Cannot create object: $!\n";
+		}
+		my $sha = <$F>;
+		chomp $sha;
+		close $F;
+		unlink $name;
+		push(@$new,['0644',$sha,$path]);
+	} else {
+		push(@$old,$path);
+	}
+}
+
 sub split_path($$) {
 	my($rev,$path) = @_;
 	my $branch;
@@ -546,6 +594,9 @@ sub commit {
 						my $opath = $action->[3];
 						print STDERR "$revision: $branch: could not fetch '$opath'\n";
 					}
+				} elsif ($node_kind eq $SVN::Node::dir) {
+					get_ignore(\@new, \@old, $revision,
+						   $branch,$path);
 				}
 			} elsif ($action->[0] eq "D") {
 				push(@old,$path);
@@ -554,6 +605,9 @@ sub commit {
 				if ($node_kind eq $SVN::Node::file) {
 					my $f = get_file($revision,$branch,$path);
 					push(@new,$f) if $f;
+				} elsif ($node_kind eq $SVN::Node::dir) {
+					get_ignore(\@new, \@old, $revision,
+						   $branch,$path);
 				}
 			} else {
 				die "$revision: unknown action '".$action->[0]."' for $path\n";

Re: [PATCH 4/4] Read author names and emails from a file

From: Andreas Ericsson <hidden>
Date: 2016-06-15 22:42:20

Karl Hasselström wrote:
Read a file with lines on the form

  username User's Full Name [off-list ref]

and use "User's Full Name [off-list ref]" as the GIT author and
committer for Subversion commits made by "username". If encountering a
commit made by a user not in the list, abort.
This is a good thing, but wouldn't it be better to use the same format 
as that of cvsimport's -A flag? Also, I imagine one would like to save 
those files within the .git directory for incremental importing, also 
like cvsimport does.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

[PATCH 2/2] Save username -> Full Name <email@addr.es> map file

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

When the user specifies a username -> Full Name [off-list ref] map
file with the -A option, save a copy of that file as
$git_dir/svn-authors. When running git-svnimport with an existing GIT
directory, use $git_dir/svn-authors (if it exists) unless a file was
explicitly specified with -A.

Signed-off-by: Karl Hasselström <redacted>

---

 Documentation/git-svnimport.txt |    5 +++++
 git-svnimport.perl              |   25 ++++++++++++++++++++-----
 2 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index 912a808..a158813 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -82,6 +82,11 @@ When importing incrementally, you might 
 	"username". If encountering a commit made by a user not in the
 	list, abort.
 
+	For convenience, this data is saved to $GIT_DIR/svn-authors
+	each time the -A option is provided, and read from that same
+	file each time git-svnimport is run with an existing GIT
+	repository without -A.
+
 -m::
 	Attempt to detect merges based on the commit message. This option
 	will enable default regexes that try to capture the name source
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 86837ed..639aa41 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -13,6 +13,7 @@
 use strict;
 use warnings;
 use Getopt::Std;
+use File::Copy;
 use File::Spec;
 use File::Temp qw(tempfile);
 use File::Path qw(mkpath);
@@ -68,10 +69,16 @@ if ($opt_M) {
 	push (@mergerx, qr/$opt_M/);
 }
 
+# Absolutize filename now, since we will have chdir'ed by the time we
+# get around to opening it.
+$opt_A = File::Spec->rel2abs($opt_A) if $opt_A;
+
 our %users = ();
-if ($opt_A) {
-	die "Cannot open $opt_A\n" unless -f $opt_A;
-	open(my $authors,$opt_A);
+our $users_file = undef;
+sub read_users($) {
+	$users_file = File::Spec->rel2abs(@_);
+	die "Cannot open $users_file\n" unless -f $users_file;
+	open(my $authors,$users_file);
 	while(<$authors>) {
 		chomp;
 		next unless /^(\S+?)\s*=\s*(.+?)\s*<(.+)>\s*$/;
@@ -302,6 +309,14 @@ EOM
 -d $git_dir
 	or die "Could not create git subdir ($git_dir).\n";
 
+my $default_authors = "$git_dir/svn-authors";
+if ($opt_A) {
+	read_users($opt_A);
+	copy($opt_A,$default_authors) or die "Copy failed: $!";
+} else {
+	read_users($default_authors) if -f $default_authors;
+}
+
 open BRANCHES,">>", "$git_dir/svn2git";
 
 sub node_kind($$$) {
@@ -498,8 +513,8 @@ sub commit {
 
 	if (not defined $author) {
 		$author_name = $author_email = "unknown";
-	} elsif ($opt_A) {
-		die "User $author is not listed in $opt_A\n"
+	} elsif (defined $users_file) {
+		die "User $author is not listed in $users_file\n"
 		    unless exists $users{$author};
 		($author_name,$author_email) = @{$users{$author}};
 	} elsif ($author =~ /^(.*?)\s+<(.*)>$/) {

[PATCH 1/2] Let git-svnimport's author file use same syntax as git-cvsimport's

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

git-cvsimport uses a username => Full Name [off-list ref] mapping
file with this syntax:

  kha=Karl Hasselström [off-list ref]

Since there is no reason to use another format for git-svnimport, use
the same format.

Signed-off-by: Karl Hasselström <redacted>

---

 Documentation/git-svnimport.txt |    4 ++--
 git-svnimport.perl              |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e0e3a5d..912a808 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -75,9 +75,9 @@ When importing incrementally, you might 
 -A <author_file>::
 	Read a file with lines on the form
 
-	  username User's Full Name <email@addres.org>
+	  username = User's Full Name <email@addr.es>
 
-	and use "User's Full Name <email@addres.org>" as the GIT
+	and use "User's Full Name <email@addr.es>" as the GIT
 	author and committer for Subversion commits made by
 	"username". If encountering a commit made by a user not in the
 	list, abort.
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 75ce8e0..86837ed 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -74,7 +74,7 @@ if ($opt_A) {
 	open(my $authors,$opt_A);
 	while(<$authors>) {
 		chomp;
-		next unless /^(\S+)\s+(.+?)\s+<(\S+)>$/;
+		next unless /^(\S+?)\s*=\s*(.+?)\s*<(.+)>\s*$/;
 		(my $user,my $name,my $email) = ($1,$2,$3);
 		$users{$user} = [$name,$email];
 	}

Re: [PATCH 1/2] Let git-svnimport's author file use same syntax as git-cvsimport's

From: Jon Loeliger <hidden>
Date: 2016-06-15 22:42:20

On Mon, 2006-02-27 at 17:08, Karl Hasselström wrote:
quoted hunk
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e0e3a5d..912a808 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -75,9 +75,9 @@ When importing incrementally, you might 
 -A <author_file>::
 	Read a file with lines on the form
 
-	  username User's Full Name <email@addres.org>
+	  username = User's Full Name <email@addr.es>
 
-	and use "User's Full Name <email@addres.org>" as the GIT
+	and use "User's Full Name <email@addr.es>" as the GIT
 	author and committer for Subversion commits made by
 	"username". If encountering a commit made by a user not in the
 	list, abort.
Actually, I believe that "example.com" was reserved
specifically for instances such as this.

See:
    http://www.faqs.org/rfcs/rfc2606.html

jdl

Re: [PATCH 1/2] Let git-svnimport's author file use same syntax as git-cvsimport's

From: Karl Hasselström <hidden>
Date: 2016-06-15 22:42:20

On 2006-03-01 15:19:53 -0600, Jon Loeliger wrote:
On Mon, 2006-02-27 at 17:08, Karl Hasselström wrote:
quoted
 	Read a file with lines on the form

-	  username User's Full Name [off-list ref]
+	  username = User's Full Name [off-list ref]

-	and use "User's Full Name [off-list ref]" as the GIT
+	and use "User's Full Name [off-list ref]" as the GIT
 	author and committer for Subversion commits made by
 	"username". If encountering a commit made by a user not in the
 	list, abort.
Actually, I believe that "example.com" was reserved specifically for
instances such as this.
Yes, I know. But email.address@example.com is much longer then
email@addr.es, and not half as funny. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help