[PATCH] Let git-svnimport clean up SVK commit messages.

Subsystems: documentation, the rest

DORMANTno replies

8 messages, 4 authors, 2016-06-15 · open the first message on its own page

[PATCH] Let git-svnimport clean up SVK commit messages.

From: Dave O'Neill <hidden>
Date: 2016-06-15 22:43:17

SVK likes to begin all commit messages with a line of the format:
   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
which makes the import desperately ugly in git.  This adds a -k option to move
this extra SVK commit line to the end of the commit message, rather than
keeping it at the beginning.
---
 Documentation/git-svnimport.txt |    6 +++++-
 git-svnimport.perl              |   18 +++++++++++++++---
 2 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e97d15e..23a54b0 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 --------
 [verse]
 'git-svnimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ] [ -d | -D ]
-		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
+		[ -C <GIT_repository> ] [ -i ] [ -k ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
 		[ -I <ignorefile_name> ] [ -A <author_file> ]
@@ -68,6 +68,10 @@ When importing incrementally, you might need to edit the .git/svn2git file.
 	Prepend 'rX: ' to commit messages, where X is the imported
 	subversion revision.
 
+-k::
+	Remove SVK-style revision/commiter info from the top of commit
+	messages and move it to the bottom.
+
 -I <ignorefile_name>::
 	Import the svn:ignore directory property to files with this
 	name in each directory. (The Subversion and GIT ignore
diff --git a/git-svnimport.perl b/git-svnimport.perl
index f459762..7076f41 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -32,20 +32,20 @@ $ENV{'TZ'}="UTC";
 
 our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
     $opt_b,$opt_r,$opt_I,$opt_A,$opt_s,$opt_l,$opt_d,$opt_D,$opt_S,$opt_F,
-    $opt_P,$opt_R);
+    $opt_P,$opt_R,$opt_k);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from SVN
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev] [-R repack_each_revs]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
-       [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
+       [-d|-D] [-i] [-k] [-u] [-r] [-I ignorefilename] [-s start_chg]
        [-m] [-M regex] [-A author_file] [-S] [-F] [-P project_name] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("A:b:C:dDFhiI:l:mM:o:rs:t:T:SP:R:uv") or usage();
+getopts("A:b:C:dDFhiI:kl:mM:o:rs:t:T:SP:R:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -835,7 +835,19 @@ sub commit {
 		$pw->writer();
 		$pr->reader();
 
+
+		if ( $opt_k ) {
+			# Remove SVK cruft in commit message
+			my ($svk_info) = $message =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+			if( $svk_info ) {
+				# and stick it on the end so we don't lose it
+				$message .= "\n$svk_info"
+			}
+		}
+
+
 		$message =~ s/[\s\n]+\z//;
+
 		$message = "r$revision: $message" if $opt_r;
 
 		print $pw "$message\n"
-- 
1.5.2.2.239.g89630

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Steven Grimm <hidden>
Date: 2016-06-15 22:43:17

Dave O'Neill wrote:
SVK likes to begin all commit messages with a line of the format:
   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
which makes the import desperately ugly in git.  This adds a -k option to move
this extra SVK commit line to the end of the commit message, rather than
keeping it at the beginning.
  
Any chance of applying this to git-svn instead? There has been talk of 
deprecating git-svnimport since git-svn now does everything 
git-svnimport does, and more. (If you believe that's not the case, 
please describe what you're doing with git-svnimport that you can't do 
with git-svn.)

-Steve

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Sam Vilain <hidden>
Date: 2016-06-15 22:43:17

Dave O'Neill wrote:
SVK likes to begin all commit messages with a line of the format:
   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
which makes the import desperately ugly in git.  This adds a -k option to move
this extra SVK commit line to the end of the commit message, rather than
keeping it at the beginning.
This is a good idea, of course if somebody didn't specify the magic -I
switch to their 'svk sm' incantation then there will be multiple changes
listed in a single revision

some examples

http://dev.catalystframework.org/svnweb/Catalyst/revision/?rev=6477
svn log -r 7190:7190 http://svn.pugscode.org/pugs

There was also a pretty nasty bug in SVK which pushed huge commitlogs
with no changes

see for example

http://utsl.gen.nz/gitweb/?p=pugs;a=commit;h=817b73f
(or:
svn log -r 14734:14734 http://svn.pugscode.org/pugs
svn diff -r 14733:14734 http://svn.pugscode.org/pugs
)

That occurred often enough that it might even be worth detecting and
dealing with specially.  ie, if multiple SVK changesets are seen in a
commit with no changes, mark it as likely bogus.

Yeah, I'm not sure what to say about all this other than "lolsvn".

Sam.

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Dave O'Neill <hidden>
Date: 2016-06-15 22:43:17

On Wed, Jun 20, 2007 at 08:19:33PM -0700, Steven Grimm wrote:
Any chance of applying this to git-svn instead? There has been talk of 
deprecating git-svnimport since git-svn now does everything 
git-svnimport does, and more. (If you believe that's not the case, 
please describe what you're doing with git-svnimport that you can't do 
with git-svn.)
Sure, I can probably apply it to git-svn as well, but based on the
testing I've done, git-svnimport still works better for what I'm doing.
I'm trying to do a straight conversion from SVN to git, so we can do
away with our Subversion repositories, so I won't need the bidirectional
support of git-svn.  

If git-svn had a --one-way option that could 
  - know that I'm converting, and import my SVN tags and branches to
    local tags and heads rather than remotes
  - turn off the git-svn-id: tags in the commit
  - avoid preserving a .git/svn/ directory (a git-svn conversion was 50%
    larger than one from git-svnimport because of this)
  - convert svn:ignore attrs to .gitignore (like the -I option of
    git-svnignore)

then I wouldn't need git-svnimport.  Alternatively, if there's some way
to postprocess my clone to do all of the above, that would probably be
good enough.

Cheers,
Dave

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Dave O'Neill <hidden>
Date: 2016-06-15 22:43:17

On Thu, Jun 21, 2007 at 06:01:01PM +1200, Sam Vilain wrote:
This is a good idea, of course if somebody didn't specify the magic -I
switch to their 'svk sm' incantation then there will be multiple changes
listed in a single revision
Yeah, I don't think there's much hope of fully cleaning up those
messages.  My goal was just to make 'git log --pretty=oneline' show
something meaningful from the commit message, rather than:

edfbf48adabb68f9b3973f5950570c355f3ee811  r6211@oxygen:  dmo | 2007-04-17 17:28:23 -0400
d5b3268e815ce9e88dd617a5b62df80da263f396  r6210@oxygen:  dmo | 2007-04-17 15:54:04 -0400

Cheers,
Dave

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Jeffrey C. Ollie <hidden>
Date: 2016-06-15 22:43:17

On Thu, 2007-06-21 at 10:14 -0400, Dave O'Neill wrote:
On Wed, Jun 20, 2007 at 08:19:33PM -0700, Steven Grimm wrote:
quoted
Any chance of applying this to git-svn instead? There has been talk of 
deprecating git-svnimport since git-svn now does everything 
git-svnimport does, and more. (If you believe that's not the case, 
please describe what you're doing with git-svnimport that you can't do 
with git-svn.)
Sure, I can probably apply it to git-svn as well, but based on the
testing I've done, git-svnimport still works better for what I'm doing.
I'm trying to do a straight conversion from SVN to git, so we can do
away with our Subversion repositories, so I won't need the bidirectional
support of git-svn.  

If git-svn had a --one-way option that could 
  - know that I'm converting, and import my SVN tags and branches to
    local tags and heads rather than remotes
This quickie Python script should convert the "tag branches" to regular
Git tags:

        import os
        refs = os.popen('git-for-each-ref --format="%(objectname)%00%(refname)%00" refs/remotes/tags')
        for line in refs:
            [sha, ref] = line.split('\0')[:2]
            os.system('git tag %s %s' % (ref[18:], sha))

  - turn off the git-svn-id: tags in the commit
git svn fetch --no-metadata
  - avoid preserving a .git/svn/ directory (a git-svn conversion was 50%
    larger than one from git-svnimport because of this)
Once you are done importing, you can delete that directory.
  - convert svn:ignore attrs to .gitignore (like the -I option of
    git-svnignore)
git svn show-ignore
then I wouldn't need git-svnimport.  Alternatively, if there's some way
to postprocess my clone to do all of the above, that would probably be
good enough.

[PATCH] Let git-svnimport and git-svn clean up SVK commit messages

From: Dave O'Neill <hidden>
Date: 2016-06-15 22:43:17

SVN repositories that have been committed to with SVK will have commits that
contain SVK metadata on the first line of the message:

   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ

After importing, git will then pick up this first as the commit title, which
isn't terribly informative in 'git log --pretty=oneline' or gitk.

This patch adds a -k option to git-svnimport and a --fix-svk-message option
(and fixSvkMessage config option) to git-svn.  Enabling this option allows
git-svn to try and move that metadata to the end of the commit message,
possibly giving us a more meaningful first line.
---
 Documentation/git-svn.txt       |   13 +++++++++++++
 Documentation/git-svnimport.txt |    6 +++++-
 git-svn.perl                    |   23 ++++++++++++++++++++---
 git-svnimport.perl              |   15 ++++++++++++---
 4 files changed, 50 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index c0d7d95..5cd89f9 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -54,6 +54,8 @@ COMMANDS
 	Set the 'useSvmProps' option in the [svn-remote] config.
 --use-svnsync-props;;
 	Set the 'useSvnsyncProps' option in the [svn-remote] config.
+--fix-svk-message;;
+	Set the 'fixSvkMessage' option in the [svn-remote] config.
 --rewrite-root=<URL>;;
 	Set the 'rewriteRoot' option in the [svn-remote] config.
 --username=<USER>;;
@@ -385,6 +387,14 @@ svn-remote.<name>.rewriteRoot::
 	the repository with a public http:// or svn:// URL in the
 	metadata so users of it will see the public URL.
 
+svn-remote.<name>.fixSvkMessage::
+	SVN repositories that have been committed to with SVK will have
+	commits that contain SVK metadata on the first line of the
+	message, which git will then pick up as the commit title.  Enabling
+	this option allows git-svn to try and move that metadata to the end
+	of the commit message, possibly giving us a more meaningful first
+	line.
+
 Since the noMetadata, rewriteRoot, useSvnsyncProps and useSvmProps
 options all affect the metadata generated and used by git-svn; they
 *must* be set in the configuration file before any history is imported
@@ -393,6 +403,9 @@ and these settings should never be changed once they are set.
 Additionally, only one of these four options can be used per-svn-remote
 section because they affect the 'git-svn-id:' metadata line.
 
+The fixSvkMessage option may be set in combination with any of these, and
+can be set after import if desired.
+
 --
 
 BASIC EXAMPLES
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e97d15e..23a54b0 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 --------
 [verse]
 'git-svnimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ] [ -d | -D ]
-		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
+		[ -C <GIT_repository> ] [ -i ] [ -k ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
 		[ -I <ignorefile_name> ] [ -A <author_file> ]
@@ -68,6 +68,10 @@ When importing incrementally, you might need to edit the .git/svn2git file.
 	Prepend 'rX: ' to commit messages, where X is the imported
 	subversion revision.
 
+-k::
+	Remove SVK-style revision/commiter info from the top of commit
+	messages and move it to the bottom.
+
 -I <ignorefile_name>::
 	Import the svn:ignore directory property to files with this
 	name in each directory. (The Subversion and GIT ignore
diff --git a/git-svn.perl b/git-svn.perl
index 50128d7..1451f0d 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -83,6 +83,7 @@ my %init_opts = ( 'template=s' => \$_template, 'shared:s' => \$_shared,
                   'trunk|T=s' => \$_trunk, 'tags|t=s' => \$_tags,
                   'branches|b=s' => \$_branches, 'prefix=s' => \$_prefix,
                   'minimize-url|m' => \$Git::SVN::_minimize_url,
+		  'fix-svk-message' => \$Git::SVN::_fix_svk_message,
 		  'no-metadata' => sub { $icv{noMetadata} = 1 },
 		  'use-svm-props' => sub { $icv{useSvmProps} = 1 },
 		  'use-svnsync-props' => sub { $icv{useSvnsyncProps} = 1 },
@@ -280,6 +281,11 @@ sub do_git_init_db {
 		command_noisy('config', "$pfx.$i", $icv{$i});
 		$set = $i;
 	}
+
+	if( defined $Git::SVN::_fix_svk_message ) {
+		command_noisy('config', "$pfx.fixSvkMessage", $Git::SVN::_fix_svk_message);
+	}
+
 }
 
 sub init_subdir {
@@ -826,7 +832,7 @@ sub working_head_info {
 package Git::SVN;
 use strict;
 use warnings;
-use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent
+use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent $_fix_svk_message
             $_repack $_repack_flags $_use_svm_props $_head
             $_use_svnsync_props $no_reuse_existing $_minimize_url/;
 use Carp qw/croak/;
@@ -849,7 +855,7 @@ BEGIN {
 	# per [svn-remote "..."] section.  Command-line options will *NOT*
 	# override options set in an [svn-remote "..."] section
 	no strict 'refs';
-	for my $option (qw/follow_parent no_metadata use_svm_props
+	for my $option (qw/follow_parent no_metadata use_svm_props fix_svk_message
 			   use_svnsync_props/) {
 		my $key = $option;
 		$key =~ tr/_//d;
@@ -1613,7 +1619,18 @@ sub do_git_commit {
 	}
 	defined(my $pid = open3(my $msg_fh, my $out_fh, '>&STDERR', @exec))
 	                                                           or croak $!;
-	print $msg_fh $log_entry->{log} or croak $!;
+	my $log_msg = $log_entry->{log};
+
+	if( $self->fix_svk_message ) {
+		# Remove SVK cruft in commit message
+		$log_msg =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+		if( $1 ) {
+			# and stick it on the end so we don't lose it
+			$log_msg .= "\n$1"
+		}
+	}
+
+	print $msg_fh $log_msg or croak $!;
 	unless ($self->no_metadata) {
 		print $msg_fh "\ngit-svn-id: $log_entry->{metadata}\n"
 		              or croak $!;
diff --git a/git-svnimport.perl b/git-svnimport.perl
index f459762..89afdd8 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -32,20 +32,20 @@ $ENV{'TZ'}="UTC";
 
 our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
     $opt_b,$opt_r,$opt_I,$opt_A,$opt_s,$opt_l,$opt_d,$opt_D,$opt_S,$opt_F,
-    $opt_P,$opt_R);
+    $opt_P,$opt_R,$opt_k);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from SVN
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev] [-R repack_each_revs]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
-       [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
+       [-d|-D] [-i] [-k] [-u] [-r] [-I ignorefilename] [-s start_chg]
        [-m] [-M regex] [-A author_file] [-S] [-F] [-P project_name] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("A:b:C:dDFhiI:l:mM:o:rs:t:T:SP:R:uv") or usage();
+getopts("A:b:C:dDFhiI:kl:mM:o:rs:t:T:SP:R:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -835,6 +835,15 @@ sub commit {
 		$pw->writer();
 		$pr->reader();
 
+		if ( $opt_k ) {
+			# Remove SVK cruft in commit message
+			$message =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+			if( $1 ) {
+				# and stick it on the end so we don't lose it
+				$message .= "\n$1"
+			}
+		}
+
 		$message =~ s/[\s\n]+\z//;
 		$message = "r$revision: $message" if $opt_r;
 
-- 
1.5.2.2.239.g89630

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

From: Dave O'Neill <hidden>
Date: 2016-06-15 22:43:17

On Thu, Jun 21, 2007 at 10:00:00AM -0500, Jeffrey C. Ollie wrote:
This quickie Python script should convert the "tag branches" to regular
Git tags:

        import os
        refs = os.popen('git-for-each-ref --format="%(objectname)%00%(refname)%00" refs/remotes/tags')
        for line in refs:
            [sha, ref] = line.split('\0')[:2]
            os.system('git tag %s %s' % (ref[18:], sha))
Thanks!  I'll give that a try.  It would be nice if this sort of cleanup
were part of git-svn in some way, though.  I doubt I'll be the last
person wanting an easy conversion from SVN.

Cheers,
Dave
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help