Hello,
I have started a while back working on support for svn:externals
support for git-svn, but since I'm not that satisfied with the current
status of the patch, I haven't modified git-svn itself and just left
the sh script I made as a PoC as it was.
There's still work to be done to it, but I the current version is
functional enough to be probably found useful by more people than
myself.
Current status follows:
Current functionality:
- fetches all the externals of an already svn-fetched repo
- support for svn:externals refresh
- if the location of the external has changed, the current working
copy will be placed aside and a new directory will be created
instead
- if the remote URI is the same (maybe a verison bump, there will
be a 'git svn rebase'
- remove support (useful for testing purposes or clean restarts)
- avoid zombie externals at all costs - in some repos empty
svn:externals might exist; svn ignores such externals, so git should
do the same
TODO:
- take into account the revision of an external, if it exists
- do not do deep svn cloning, to avoid legthy operations, just pull HEAD
(this actually needs changes in git-svn itself)
- use/create shallow copies to git svn repos (one revision should be enough
for most externals)
- use submodules for externals
Any comments are welcome.
--
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein
subversion does not process in any way empty svn:externals properties
and doesn't list them in any way; git svn externals shouldn't do that
either
Signed-off-by: Eddy Petrișor <redacted>
---
t/t9101-git-svn-props.sh | 24 +++++++++++++++++++++++-
1 files changed, 23 insertions(+), 1 deletions(-)
@@ -0,0 +1,76 @@+#!/bin/sh+#+# Copyright (c) 2008 Eddy PetriÈor+++test_description='git svn properties pulls and updates externals'+../lib-git-svn.sh++test_expect_success'load repository with base directory''+svnadminload-q"$rawsvnrepo"<../t9130/clean.dump+'++test_expect_success'checkout the simple repo''+mkdir-pgitrepo&&+(+cdgitrepo&&+gitsvninit"$svnrepo/trunk"&&+gitsvnfetch&&+cd..+)+'++test_expect_success'add an external in the repo''+svncheckout"$svnrepo/trunk"work.svn&&+(+cdwork.svn&&+svnpropsetsvn:externals"external $(echo$svnrepo|sed"s#\ #%20#")/external".&&+svncommit-m"added external directory to repo"&&+svnupdate&&+cd..&&+rm-frwork.svn+)+'++test_expect_success'git svn fetch the property changes and fetch externals''+cdgitrepo&&+gitsvnrebase&&+gitsvnshow-externals|grep-q"^/external"&&+gitsvn-externals&&+cd..+'++test_expect_success'create an external pinned to a version''+svncheckout"$svnrepo/trunk"work.svn&&+(+cdwork.svn&&+svnpropsetsvn:externals"external -r2 $(echo$svnrepo|sed"s#\ #%20#")/external".&&+svncommit-m"pin the external to -r2"&&+svnupdate&&+cd..&&+rm-frwork.svn+)&&+svncheckout"$svnrepo/external"ext.svn&&+(+cdext.svn&&+echo"0">externalfile&&+svncommit-m"mark the bad floating revision for the external"&&+cd..&&+rm-frext.svn+)+'++test_expect_failure'refresh revisioned externals in git''+cdgitrepo&&+gitconfig--list>>/tmp/test&&+gitsvnrebase&&+gitsvn-externals&&+grep1external/externalfile&&+cd..+'++#TODO: add test_expect_success for a URI change for the external++++test_done
From: Eddy Petrisor <redacted>
Current functionality:
- fetches all the externals of an already svn-fetched repo
- support for svn:externals refresh
- if the location of the external has changed, the current working
copy will be placed aside and a new directory will be created
instead
- if the remote URI is the same (maybe a verison bump, there will
be a 'git svn rebase'
- remove support (useful for testing purposes or clean restarts)
- avoid zombie externals at all costs - in some repos empty
svn:externals might exist; svn ignores such externals, so git should
do the same
TODO:
- take into account the revision of an external, if it exists
- do not do deep svn cloning, to avoid legthy operations, just pull HEAD
- add shallow copies to git svn repos (one revision should be enough
for most externals)
- use submodules for externals
---
.gitignore | 1 +
Makefile | 1 +
git-svn-externals.sh | 158 ++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 160 insertions(+), 0 deletions(-)
create mode 100755 git-svn-externals.sh
@@ -0,0 +1,158 @@+#!/bin/sh++set-e++GIT_SVN_LAST_DIR="$(pwd)"+trap"cd '$GIT_SVN_LAST_DIR'"INTEXITKILL+++USAGE='[help|init|refresh|show]'++LONG_USAGE='gitsvnexternalshelp+printthislonghelpmessage.+gitsvnexternalsinit[<pathspec>...]+pullallexternalsaccordingtocurrentrepostate.+gitsvnexternalsrefresh[<pathspec>...]+removescurrentexternalsandrepullsthem.+'+++SUBDIRECTORY_OK=Sometimes+OPTIONS_SPEC=+.git-sh-setup+cd_to_toplevel+# this should never fail+require_work_tree+++GIT_SVN_TOP_DIR="$(pwd)"++svn_revision_to_numrevision()+#$1 - repository URI+#$2 - the revision to be transformed+{+localREVSPEC+["$2"]&&REVSPEC="-r $2"||REVSPEC=''+["$1"]||die"Internal error: no repository given in svn_revision_to_numrevision"+LANG=Csvninfo"$1""$REVSPEC"|grep'Revision'|cut-f2-d':'|sed's#^\ \ *##g'+}++prune_inital_slashes()+{+echo"$1"|sed's#^//*##g'+}++git_svn_externals_remove()+{+localLASTDIR="$(pwd)"+cd_to_toplevel&&TOPLEVELDIR="$(pwd)"+gitsvnshow-externals|grep-vE'^(#.*|)$'|whilereadEXT_DIRREVEXT_SRC;do+if[-z"$REV"];then+echo"Skipping illegal external '$EXT_DIR' which has no value">&2+continue+fi+EXT_DIR="$(prune_inital_slashes"$EXT_DIR")"++if[-z"$EXT_SRC"];then+EXT_SRC="$REV"+REV="-rHEAD"+fi+REV=${REV#-r}++echo"Removing local copy for external '$EXT_DIR' ( $EXT_SRC@$REV )"+rm-fr"$EXT_DIR"+done++cd"$LASTDIR"+}++git_svn_init_dir_repo()+# assumes is ran from the parent dir of the repo+# $1 EXT_DIR+# $2 EXT_SRC+# $3 REV+{+localLASTDIR="$(pwd)"++mkdir-p"$1"+cd"$1"+gitsvninit"$2"+gitsvnfetch++cd"$LASTDIR"+}++git_svn_externals_refresh_external()+#$1 directory where the external is locally+#$2 remote url+#$3 revision at which the external is pinned at+{+localLASTDIR="$GIT_SVN_TOP_DIR"++localEXT_SRC="$2"+localEXT_DIR="$1"++ifcd"$EXT_DIR"2>/dev/null;then+localOLDDIREXISTS=yes+localOLD_EXT_SRC=$(gitconfig--getsvn-remote.svn.url)+localOLD_EXT_REV=$(gitconfig--getsvn-remote.svn.revision||svn_revision_to_numrevision$OLD_EXT_SRCHEAD)+cd-+else+localOLDDIREXISTS=no+localOLD_EXT_SRC="${EXT_SRC}"+localOLD_EXT_REV=$(svn_revision_to_numrevision$EXT_SRCHEAD)+fi++if["$OLD_EXT_SRC"!="$EXT_SRC"];then+echo"External URI definition changed. Moving away the old repo and pulling a new one."+cd"$LASTDIR"+localSUBDIR="$(basename"$EXT_DIR")"+localDIRNAME="$(dirname"$EXT_DIR")"+NEWDIR="$(mktemp-p"$DIRNAME""$SUBDIR.obsolete.XXXXXX")"&&rm-f"$NEWDIR"+mv"$EXT_DIR""$NEWDIR"+echo" Old copy moved to '$NEWDIR'."+git_svn_init_dir_repo"$EXT_DIR""$EXT_SRC""$REV"+elif["$OLDDIREXISTS"="no"];then+git_svn_init_dir_repo"$EXT_DIR""$EXT_SRC""$REV"+else+cd"$EXT_DIR"+gitsvnrebase+fi++cd"$LASTDIR"+return0+}++git_svn_externals_refresh()+{+gitsvnshow-externals|grep-vE'^(#.*|)$'|whilereadEXT_DIRREVEXT_SRC;do+if[-z"$REV"];then+echo"Skipping illegal external '$EXT_DIR' which has no value">&2+continue+fi+EXT_DIR="$(prune_inital_slashes"$EXT_DIR")"++if[-z"$EXT_SRC"];then+EXT_SRC="$REV"+REV="-rHEAD"+fi+REV=${REV#-r}++echo"Refreshing local copy for external '$EXT_DIR' ( $EXT_SRC@$REV )"+cd"$GIT_SVN_TOP_DIR"+git_svn_externals_refresh_external"$EXT_DIR""$EXT_SRC""$REV"+done++cd"$GIT_SVN_TOP_DIR"+}+++case$1in+--remove)+git_svn_externals_remove+;;+-r|--refresh|*)+git_svn_externals_refresh+;;+esac+
Please CC me, I am not listed.
2008/8/29 Eddy Petrișor [off-list ref]:
Hello,
I have started a while back working on support for svn:externals
support for git-svn, but since I'm not that satisfied with the current
status of the patch, I haven't modified git-svn itself and just left
the sh script I made as a PoC as it was.
There's still work to be done to it, but I the current version is
functional enough to be probably found useful by more people than
myself.
Current status follows:
Current functionality:
- fetches all the externals of an already svn-fetched repo
- support for svn:externals refresh
- if the location of the external has changed, the current working
copy will be placed aside and a new directory will be created
instead
- if the remote URI is the same (maybe a verison bump, there will
be a 'git svn rebase'
- remove support (useful for testing purposes or clean restarts)
- avoid zombie externals at all costs - in some repos empty
svn:externals might exist; svn ignores such externals, so git should
do the same
TODO:
- take into account the revision of an external, if it exists
- do not do deep svn cloning, to avoid legthy operations, just pull HEAD
(this actually needs changes in git-svn itself)
- use/create shallow copies to git svn repos (one revision should be enough
for most externals)
- use submodules for externals
Any comments are welcome.
--
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein
--
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein
From: Eric Wong <hidden> Date: 2016-06-15 22:45:15
Eddy Petrișor [off-list ref] wrote:
Hello,
Hi Eddy,
I have started a while back working on support for svn:externals
support for git-svn, but since I'm not that satisfied with the current
status of the patch, I haven't modified git-svn itself and just left
the sh script I made as a PoC as it was.
There's still work to be done to it, but I the current version is
functional enough to be probably found useful by more people than
myself.
Current status follows:
Current functionality:
- fetches all the externals of an already svn-fetched repo
- support for svn:externals refresh
- if the location of the external has changed, the current working
copy will be placed aside and a new directory will be created
instead
- if the remote URI is the same (maybe a verison bump, there will
be a 'git svn rebase'
- remove support (useful for testing purposes or clean restarts)
- avoid zombie externals at all costs - in some repos empty
svn:externals might exist; svn ignores such externals, so git should
do the same
TODO:
- take into account the revision of an external, if it exists
- do not do deep svn cloning, to avoid legthy operations, just pull HEAD
(this actually needs changes in git-svn itself)
git svn clone -r<latest_revision_number> URL should work if you extract
the revision number easily. Specifying "-rHEAD" will only work if the
branch of the external you're tracking was the last modified revision in
the repository, so it's not very useful. "svn log" seems to have the
same semantics as git-svn as far as -rHEAD being useful or not...
- use/create shallow copies to git svn repos (one revision should be enough
for most externals)
- use submodules for externals
I'm not sure if mapping submodules to externals is a good idea
because externals don't require exact revisions and submodules do.
There's also an issue I was just made aware of two days ago with
submodules and git-svn that I haven't had time to work on.
Another user also privately reported a bug to me about git-svn having
trouble dcommitting when using submodules. I've attached the test case
here in case you have any thoughts on how to handle this (I think the
easiest would be to ignore submodules on dcommit entirely).
Any comments are welcome.
Also some small portability issues: "grep -q" is definitely unportable
in my experience. There are probably some more that I am missing my eye
at this time of night...
--
Eric Wong
From: Eric Wong <hidden> Date: 2016-06-15 22:45:16
Eric Wong [off-list ref] wrote:
Eddy Petrișor [off-list ref] wrote:
quoted
- use submodules for externals
I'm not sure if mapping submodules to externals is a good idea
because externals don't require exact revisions and submodules do.
There's also an issue I was just made aware of two days ago with
submodules and git-svn that I haven't had time to work on.
Definitely a complex problem. A new submodule from git mapping to an
new svn:external probably doesn't work very well, either. There's also
UI/policy issues with creating a new directory/branch in SVN to house
the external. It will also will not work for (the few) projects that
are using the root of the SVN repository directly.
Another user also privately reported a bug to me about git-svn having
trouble dcommitting when using submodules. I've attached the test case
here in case you have any thoughts on how to handle this (I think the
easiest would be to ignore submodules on dcommit entirely).
Well, I'm stuck on this problem. Since that user that reported this
emailed me privately, I'm bcc-ing him on this issue, too.
I've tried the following, and it gets the commit into SVN without the
submodules using dcommit.
--- a/git-svn.perl+++ b/git-svn.perl
@@ -3364,7 +3364,7 @@ sub new {subgenerate_diff{my($tree_a,$tree_b)=@_;-my@diff_tree=qw(diff-tree -z -r);+my@diff_tree=qw(diff-tree --ignore-submodules -z -r);if($_cp_similarity){push@diff_tree,"-C$_cp_similarity";}else{
However, all is not well. The rebase/reset step in dcommit gets
triggered with a rebase because the submodule is dropped from the tree
when we refetch the commit from SVN.
The simplest would be to say we don't support mixing submodules with
git-svn. I'm fine with this statement.
The clever way would be introducing a custom revprop for git-svn
to use to declare submodule dependencies. Then git-svn can
recreate the information needed for a revprop on fetch.
That would mean regular SVN users would not get access to submodules.
Cutting SVN users off from being able to effectively use an SVN repo
doesn't make sense to me. The project might as well be using git
entirely and ditch SVN. Of course, some PHB somewhere will want to keep
the project on SVN even when all the developers are using git-svn...
Since gmane seems to have broken the attachment I previously sent,
here again is the patch inlined:
From bddc57078188d4798a9a74401aa0da8ca35a4db1 Mon Sep 17 00:00:00 2001
From: Eric Wong <redacted>
Date: Fri, 29 Aug 2008 02:09:50 -0700
Subject: [PATCH] git-svn: Add submodule ignore test
---
t/t9126-git-svn-submodule.sh | 44 ++++++++++++++++++++++++++++++++++++++++++
1 files changed, 44 insertions(+), 0 deletions(-)
create mode 100644 t/t9126-git-svn-submodule.sh
@@ -0,0 +1,44 @@+#!/bin/sh+# Copyright (c) 2008 Eric Wong++test_description='git-svn submodule'++../lib-git-svn.sh++test_expect_success'setup repo''svn co "$svnrepo" svnfoo'++test_expect_success'add files to svn repo''+touchsvnfoo/foo.txt+(cdsvnfoo&&svnadd*&&svncommit-m"test commit")+'++test_expect_success'git-svn clone the svn repo''+git-svnclone"$svnrepo"gitfoo+'++test_expect_success'create a git repo to include as a submodule''+mkdirgit_to_include+(+cdgit_to_include&&+gitinit&&+echo"This is another file">another.txt&&+gitaddanother.txt&&+gitcommit-a-m"Adding file to submodule"&&+gitlog+)+'++test_expect_success'add the submodule''+(+cdgitfoo&&+gitsubmoduleadd"$remove_trash"/git_to_include\+git_as_submodule&&+gitcommit-a-m"Adding submodule"+)+'++test_expect_success'try to rebase and commit''+(cdgitfoo&&gitsvnrebase&&gitsvndcommit)+'++test_done
(Please keep the CC. Thanks)
2008/8/29 Eric Wong [off-list ref]:
Eddy Petrișor [off-list ref] wrote:
quoted
Hello,
Hi Eddy,
Hello and sorry for the late reply.
(I was on a small vacation away from the computer in the last two weeks.)
quoted
I have started a while back working on support for svn:externals
support for git-svn, but since I'm not that satisfied with the current
status of the patch, I haven't modified git-svn itself and just left
the sh script I made as a PoC as it was.
There's still work to be done to it, but I the current version is
functional enough to be probably found useful by more people than
myself.
WRT the revision pinning, it seems to me that is enough to locate that
revision on the URI in question and checkout that revision. Still I am
unsure if it would be wise to (stash +) svn rebase + checkout the
pinned version (+ stash pop), since one would needlessly pull newer
stuff as the remote svn HEAD advances, but the pinned version might
simply stagnate.
I already have/wrote some code that follows the remote HEAD or a
specific for the necessary, but I am unsure if is still present in the
PoC script, is not that hard (in sh - a "svn info" on the URI, not on
the local copy would reveal the real revision of the HEAD).
quoted
Current status follows:
Current functionality:
- fetches all the externals of an already svn-fetched repo
- support for svn:externals refresh
- if the location of the external has changed, the current working
copy will be placed aside and a new directory will be created
instead
- if the remote URI is the same (maybe a verison bump, there will
be a 'git svn rebase'
- remove support (useful for testing purposes or clean restarts)
- avoid zombie externals at all costs - in some repos empty
svn:externals might exist; svn ignores such externals, so git should
do the same
TODO:
- take into account the revision of an external, if it exists
- do not do deep svn cloning, to avoid legthy operations, just pull HEAD
(this actually needs changes in git-svn itself)
git svn clone -r<latest_revision_number> URL should work if you extract
the revision number easily.
Why was I under the impression that this wasn't working? Or was I
expecting a shallow repo?
Specifying "-rHEAD" will only work if the
branch of the external you're tracking was the last modified revision in
the repository, so it's not very useful.
as I already said, "svn info URI" can return the real revision, no
need to ness with the pseudo-revision HEAD.
"svn log" seems to have the
same semantics as git-svn as far as -rHEAD being useful or not...
quoted
- use/create shallow copies to git svn repos (one revision should be enough
for most externals)
- use submodules for externals
I'm not sure if mapping submodules to externals is a good idea
because externals don't require exact revisions and submodules do.
I don't think I can follow you. Externals actually require exact
revisions or can be made to pretend as if they do in git-svn context
with continuous HEAD refresh.
There's also an issue I was just made aware of two days ago with
submodules and git-svn that I haven't had time to work on.
Another user also privately reported a bug to me about git-svn having
trouble dcommitting when using submodules. I've attached the test case
here in case you have any thoughts on how to handle this (I think the
easiest would be to ignore submodules on dcommit entirely).
Probably, and try later to tackle the problem.
quoted
Any comments are welcome.
Also some small portability issues: "grep -q" is definitely unportable
in my experience. There are probably some more that I am missing my eye
at this time of night...
Will fix it.
--
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein
(Please keep the CC. Thanks)
I already have/wrote some code that follows the remote HEAD or a
specific for the necessary, but I am unsure if is still present in the
That was supposed to be:
specific revision for such cases, but I am unsure if is still present in the
--
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein