git-svn too slow, contacts upstream svn repo

4 messages, 2 authors, 2016-06-15 · open the first message on its own page

git-svn too slow, contacts upstream svn repo

From: Quark <hidden>
Date: 2016-06-15 22:57:27

hello list,

I have been using git-svn in an corporate environment where svn repo has lot of branches, (lot means > 100). To avoid cloning all branches my config looks as below

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        branches = srcroot/branches/{branch_1, branch_2, branch_3}:refs/remotes/*

Now, when a new branch of my interest is added in svn repo, I had like it to be in my git-repo as well. In an ideal world, one shall simply add that branch name to list in curly braces above, but that does not work. I had love to be proved wrong here. Somebody on stackoverflow.com suggested (to else's question, not mine) to add one more "fetch" as show below.

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        fetch = srcroot/branch_4:refs/remotes/* # NEW BRANCH
        branches = srcroot/branches/{branch_1, branch_2, branch_3}:refs/remotes/*

haven't tried it, neither do I like it.

I do have a solution which WORKS, it is modified version of an example from \doc\git\html\git-svn.html page. Below I have pasted the example and modified it to reflect my specifics.

# assume an existing git-svn repo
        D:\sourcecode
# Clone locally - make sure the refs/remotes/ space matches the server
        mkdir project
        cd project
        git init
        git remote add origin file:///D:/sourcecode # file:// is delibrate, want true n/w behaviour
        git config --replace-all remote.origin.fetch '+refs/remotes/*:refs/remotes/*'
        git fetch
# Prevent fetch/pull from local git repo,
# we only want to use git svn for form here
       git config --remove-section remote.origin
# Create a local branch from one of the branches just fetched
        git checkout -b master FETCH_HEAD
# Initialize 'git svn' locally (be sure to use the same URL and -T/-b/-t options as were used on server)
        git svn init http://svn.example.com/project

# edit config to include newly added branch in curly braces
# Pull the latest changes from Subversion
        git svn fetch -r <recentish rev, say BEGIN>:HEAD

Above works exactly as I want, except the last step of "git-svn fetch" connects to svn repo for each revision from BEGIN to HEAD. I want fairly long history (for pretty blame). Which implies my BEGIN is not so recentish adterall. Hence, it takes awfully long to finish, 2 days typically.

When GIT_TRACE is 1, below is output received

trace: exec: 'git-svn' 'fetch'
trace: run_command: 'git-svn' 'fetch'
trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall'
trace: built-in: git 'config' '--bool' '--get' 'svn.parent'
trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache'
trace: built-in: git 'config' '--get' 'svn.revision'
trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout'
trace: built-in: git 'config' '--get' 'svn.authorsprog'
trace: built-in: git 'config' '--bool' '--get' 'svn.followparent'
trace: built-in: git 'config' '--get' 'svn.authorsfile'
trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps'
trace: built-in: git 'config' '--get' 'svn.username'
trace: built-in: git 'config' '--get' 'svn.repackflags'
trace: built-in: git 'config' '--bool' '--get' 'svn.localtime'
trace: built-in: git 'config' '--int' '--get' 'svn.repack'
trace: built-in: git 'config' '--get' 'svn.ignorepaths'
trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize'
trace: built-in: git 'config' '--bool' '--get' 'svn.quiet'
trace: built-in: git 'config' '--get' 'svn.ignorerefs'
trace: built-in: git 'config' '--get' 'svn.configdir'
trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom'
trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps'
trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata'
trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor'
trace: built-in: git 'rev-parse' '--symbolic' '--all'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '524908'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '9f1414be94ab007b62ace31bf4d210a069276127..refs/remotes/branch_1' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '5eb0a454bcf066a8199b851add9ec07cde80119d..refs/remotes/branch_2' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '383b68b8514010a71efe10821e5ccc3541903ceb..refs/remotes/branch_3' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '89fe1a1d2cfca0886003f043c408fb5afadfec93..refs/remotes/trunk' '--'

I keep an watch for "svn-remote.svn.branches-maxRev", this serves as my progress. This wait is too much for a simple branch. Though, I must point out, day feels fresh once it finishes :) Does any one know any for this?

If you paid attention this is Windows machine
git version 1.8.1.msysgit.1


thanks,
Quark

Re: git-svn too slow, contacts upstream svn repo

From: Quark <hidden>
Date: 2016-06-15 22:57:27


----- Original Message -----
From: Quark <redacted>
To: Git List <redacted>
Cc: 
Sent: Monday, 27 May 2013 8:55 PM
Subject: git-svn too slow, contacts upstream svn repo

hello list,

I have been using git-svn in an corporate environment where svn repo has lot of 
branches, (lot means > 100). To avoid cloning all branches my config looks as 
below

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        branches = srcroot/branches/{branch_1, branch_2, 
branch_3}:refs/remotes/*

Now, when a new branch of my interest is added in svn repo, I had like it to be 
in my git-repo as well. In an ideal world, one shall simply add that branch name 
to list in curly braces above, but that does not work. I had love to be proved 
wrong here. Somebody on stackoverflow.com suggested (to else's question, not 
mine) to add one more "fetch" as show below.

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        fetch = srcroot/branch_4:refs/remotes/* # NEW BRANCH
        branches = srcroot/branches/{branch_1, branch_2, 
branch_3}:refs/remotes/*

haven't tried it, neither do I like it.

I do have a solution which WORKS, it is modified version of an example from 
\doc\git\html\git-svn.html page. Below I have pasted the example 
and modified it to reflect my specifics.

# assume an existing git-svn repo
        D:\sourcecode
# Clone locally - make sure the refs/remotes/ space matches the server
        mkdir project
        cd project
        git init
        git remote add origin file:///D:/sourcecode # file:// is delibrate, want 
true n/w behaviour
        git config --replace-all remote.origin.fetch 
'+refs/remotes/*:refs/remotes/*'
        git fetch
# Prevent fetch/pull from local git repo,
# we only want to use git svn for form here
       git config --remove-section remote.origin
# Create a local branch from one of the branches just fetched
        git checkout -b master FETCH_HEAD
# Initialize 'git svn' locally (be sure to use the same URL and -T/-b/-t 
options as were used on server)
        git svn init http://svn.example.com/project

# edit config to include newly added branch in curly braces
# Pull the latest changes from Subversion
        git svn fetch -r <recentish rev, say BEGIN>:HEAD

Above works exactly as I want, except the last step of "git-svn fetch" 
connects to svn repo for each revision from BEGIN to HEAD. I want fairly long 
history (for pretty blame). Which implies my BEGIN is not so recentish adterall. 
Hence, it takes awfully long to finish, 2 days typically.

When GIT_TRACE is 1, below is output received

trace: exec: 'git-svn' 'fetch'
trace: run_command: 'git-svn' 'fetch'
trace: built-in: git 'config' '--bool' '--get' 
'svn.fetchall'
trace: built-in: git 'config' '--bool' '--get' 
'svn.parent'
trace: built-in: git 'config' '--bool' '--get' 
'svn.noauthcache'
trace: built-in: git 'config' '--get' 'svn.revision'
trace: built-in: git 'config' '--bool' '--get' 
'svn.nocheckout'
trace: built-in: git 'config' '--get' 'svn.authorsprog'
trace: built-in: git 'config' '--bool' '--get' 
'svn.followparent'
trace: built-in: git 'config' '--get' 'svn.authorsfile'
trace: built-in: git 'config' '--bool' '--get' 
'svn.useSvmProps'
trace: built-in: git 'config' '--get' 'svn.username'
trace: built-in: git 'config' '--get' 'svn.repackflags'
trace: built-in: git 'config' '--bool' '--get' 
'svn.localtime'
trace: built-in: git 'config' '--int' '--get' 
'svn.repack'
trace: built-in: git 'config' '--get' 'svn.ignorepaths'
trace: built-in: git 'config' '--int' '--get' 
'svn.logwindowsize'
trace: built-in: git 'config' '--bool' '--get' 
'svn.quiet'
trace: built-in: git 'config' '--get' 'svn.ignorerefs'
trace: built-in: git 'config' '--get' 'svn.configdir'
trace: built-in: git 'config' '--bool' '--get' 
'svn.addauthorfrom'
trace: built-in: git 'config' '--bool' '--get' 
'svn.useSvnsyncProps'
trace: built-in: git 'config' '--bool' '--get' 
'svn.noMetadata'
trace: built-in: git 'config' '--bool' '--get' 
'svn.uselogauthor'
trace: built-in: git 'rev-parse' '--symbolic' '--all'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' 
'524908'
trace: built-in: git 'rev-list' '--pretty=raw' 
'--reverse' 
'9f1414be94ab007b62ace31bf4d210a069276127..refs/remotes/branch_1' 
'--'
trace: built-in: git 'rev-list' '--pretty=raw' 
'--reverse' 
'5eb0a454bcf066a8199b851add9ec07cde80119d..refs/remotes/branch_2' 
'--'
trace: built-in: git 'rev-list' '--pretty=raw' 
'--reverse' 
'383b68b8514010a71efe10821e5ccc3541903ceb..refs/remotes/branch_3' 
'--'
trace: built-in: git 'rev-list' '--pretty=raw' 
'--reverse' 
'89fe1a1d2cfca0886003f043c408fb5afadfec93..refs/remotes/trunk' 
'--'

I keep an watch for "svn-remote.svn.branches-maxRev", this serves as 
my progress. This wait is too much for a simple branch. Though, I must point 
out, day feels fresh once it finishes :) Does any one know any for this?

If you paid attention this is Windows machine
git version 1.8.1.msysgit.1


thanks,
Quark

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
is this not right forum?

thanks,
Quark

Re: git-svn too slow, contacts upstream svn repo

From: Konstantin Khomoutov <hidden>
Date: 2016-06-15 22:57:28

On Tue, 28 May 2013 15:42:57 +0800 (SGT)
Quark [off-list ref] wrote:
quoted
I have been using git-svn in an corporate environment where svn
repo has lot of branches, (lot means > 100). To avoid cloning all
branches my config looks as below
[...]
is this not right forum?
As a matter of fact, this mailing list is the only correct place to ask
questions like yours.  But this is free software after all -- people
who could answer your question may be busy/absent at the moment or even
not involved in the project anymore (in the worst case).  So be
prepared to wait some time.  Also be prepared for your particular
trouble not being solved.

In the meantime, I think Thomas provided you with valuable suggestions
in reply to your mirror post on git-users, so you could possibly
explore them.

Re: git-svn too slow, contacts upstream svn repo

From: Quark <hidden>
Date: 2016-06-15 22:57:28




----- Original Message -----
From: Konstantin Khomoutov <redacted>
To: Quark <redacted>
Cc: Git List <redacted>
Sent: Tuesday, 28 May 2013 5:24 PM
Subject: Re: git-svn too slow, contacts upstream svn repo

On Tue, 28 May 2013 15:42:57 +0800 (SGT)
Quark [off-list ref] wrote:
quoted
 > I have been using git-svn in an corporate environment where svn
 > repo has lot of branches, (lot means > 100). To avoid cloning all
 > branches my config looks as below
[...]
quoted
 is this not right forum?
As a matter of fact, this mailing list is the only correct place to ask
questions like yours.  But this is free software after all -- people
who could answer your question may be busy/absent at the moment or even
not involved in the project anymore (in the worst case).  So be
prepared to wait some time.  Also be prepared for your particular
trouble not being solved.

In the meantime, I think Thomas provided you with valuable suggestions
in reply to your mirror post on git-users, so you could possibly
explore them.
I was impatient not because feeling-of-entitlement, but I thought it might get lost in pile of e-mails
Secondly was not sure if this is right forum, I might be making noise after all, thanks for your clarification.
Now will you please excuse, I have to check Thomas's suggestions? :)
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help