From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
Hi,
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to treat
category-a and category-b as branches, which isn't right at all. As a
result, git-svn seems to skip most (if not all) revisions that occur in
these directories and creates a lot of entries in unhandled.log.
I've also encountered an index corruption in one of the
.git/svn/<branch>/index files which I think it probably related.
I've had a quick look at the source myself, but perl isn't my strong
point. What I think it should do is something like recurse down the tree
from the directory given looking for folders that copy from trunk (or
some other branch that already exists). That would work perfectly well
for the default flat branch storage method as well as the one we use.
The only other problem is in branch naming, which could clash if you
only use the outer-most directory name, so I'd suggest something that
involves concatenating the folders in the path relative to 'branches' to
keep them unique (if git can handle slashes in branch names then all the
better).
Thanks for reading.
--
Russ
From: David Kastrup <hidden> Date: 2016-06-15 22:43:33
Russ Brown [off-list ref] writes:
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
--
David Kastrup
From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
David Kastrup wrote:
Russ Brown [off-list ref] writes:
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
I specified the 'branches' directory, but that's because earlier in the
life of the repo we did just do the flat branch layout, but decided to
make it more structured once that got unwieldy.
Is it possible to specify more than one folder for the branches option?
--
Russ
From: David Kastrup <hidden> Date: 2016-06-15 22:43:33
Russ Brown [off-list ref] writes:
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
I specified the 'branches' directory, but that's because earlier in
the life of the repo we did just do the flat branch layout, but
decided to make it more structured once that got unwieldy.
Cough, cough. _What_ did you specify in your .git/config file
regarding the svn structure? Please quote the section.
Is it possible to specify more than one folder for the branches
option?
It is possible to adapt the config section to the actual layout. If
not otherwise, by starting with
git svn init
with a clean slate, editing the config file, and only then actually
fetching stuff.
However, git-svn will not magically start guessing that you changed
your structure around. You have to edit the configuration
appropriately.
--
David Kastrup
From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
David Kastrup wrote:
Russ Brown [off-list ref] writes:
quoted
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
I specified the 'branches' directory, but that's because earlier in
the life of the repo we did just do the flat branch layout, but
decided to make it more structured once that got unwieldy.
Cough, cough. _What_ did you specify in your .git/config file
regarding the svn structure? Please quote the section.
Erm, sorry.
[svn-remote "svn"]
url = svn://svn.<name>.com
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
(URL changed in case it annoys my employers)
I didn't write this by hand: it was generated by git-svn init.
quoted
Is it possible to specify more than one folder for the branches
option?
It is possible to adapt the config section to the actual layout. If
not otherwise, by starting with
git svn init
with a clean slate, editing the config file, and only then actually
fetching stuff.
However, git-svn will not magically start guessing that you changed
your structure around. You have to edit the configuration
appropriately.
That's why I suggested that a method involving detecting branches based
on whether the directory is a copy of trunk or another branch might
'magically' work in all scenarios. I've used a similar branch 'scanning'
technique before for a different reason. But I realise there may be
technical reasons as to why that might not be possible.
--
Russ
From: Peter Baumann <hidden> Date: 2016-06-15 22:43:33
On Tue, Sep 04, 2007 at 10:21:22AM -0500, Russ Brown wrote:
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
I specified the 'branches' directory, but that's because earlier in
the life of the repo we did just do the flat branch layout, but
decided to make it more structured once that got unwieldy.
Cough, cough. _What_ did you specify in your .git/config file
regarding the svn structure? Please quote the section.
Erm, sorry.
[svn-remote "svn"]
url = svn://svn.<name>.com
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
(URL changed in case it annoys my employers)
I didn't write this by hand: it was generated by git-svn init.
Try something like this:
[svn-remote "svn"]
# trunk
fetch = trunk:refs/remotes/trunk
# branches
fetch = branches/category-a/branch_a:refs/remotes/svn/branch_a
fetch = branches/category-a/branch_b:refs/remotes/svn/branch_b
fetch = branches/category-b/branch_c:refs/remotes/svn/branch_c
# tags
tags = tags/*:refs/remotes/tags/*
(Not sure if wildcards will work here, but I'm sure you could experiment and try
it out :-)
-Peter
From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
Peter Baumann wrote:
On Tue, Sep 04, 2007 at 10:21:22AM -0500, Russ Brown wrote:
quoted
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
David Kastrup wrote:
quoted
Russ Brown [off-list ref] writes:
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to
treat category-a and category-b as branches, which isn't right at
all. As a result, git-svn seems to skip most (if not all) revisions
that occur in these directories and creates a lot of entries in
unhandled.log.
So what did you specify in your .git/config file regarding the svn
structure?
I specified the 'branches' directory, but that's because earlier in
the life of the repo we did just do the flat branch layout, but
decided to make it more structured once that got unwieldy.
Cough, cough. _What_ did you specify in your .git/config file
regarding the svn structure? Please quote the section.
Erm, sorry.
[svn-remote "svn"]
url = svn://svn.<name>.com
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
(URL changed in case it annoys my employers)
I didn't write this by hand: it was generated by git-svn init.
Try something like this:
[svn-remote "svn"]
# trunk
fetch = trunk:refs/remotes/trunk
# branches
fetch = branches/category-a/branch_a:refs/remotes/svn/branch_a
fetch = branches/category-a/branch_b:refs/remotes/svn/branch_b
fetch = branches/category-b/branch_c:refs/remotes/svn/branch_c
# tags
tags = tags/*:refs/remotes/tags/*
(Not sure if wildcards will work here, but I'm sure you could experiment and try
it out :-)
Thanks a lot! I'll try it out.
I wonder if I'm going to have to fetch in stages here. i.e. configuring
the branches for the original layout, fetching up the point where the
layout changed a bit and reconfigure, fetch some more, tweak again etc.
From: Eric Wong <hidden> Date: 2016-06-15 22:43:33
Russ Brown [off-list ref] wrote:
Hi,
Hi Russ,
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to treat
category-a and category-b as branches, which isn't right at all. As a
result, git-svn seems to skip most (if not all) revisions that occur in
these directories and creates a lot of entries in unhandled.log.
This is a known problem with the way git-svn handles branches and tags.
Nested branch (and tags) structures aren't supported with globbing and
so you can't have more than one branches/tags specification in your
config.
I've also encountered an index corruption in one of the
.git/svn/<branch>/index files which I think it probably related.
Not sure about that. git-svn should detect and attempt to fix index
corruptions (which happened for me since I interrupt git-svn quite
often when working on it).
I've had a quick look at the source myself, but perl isn't my strong
point. What I think it should do is something like recurse down the tree
from the directory given looking for folders that copy from trunk (or
some other branch that already exists). That would work perfectly well
for the default flat branch storage method as well as the one we use.
The globbing functionality of branches in git-svn is some of the ugliest
code I've ever written in my life. Somehow I got it working for the
simple general cases but I've been afraid to touch it for a while...
The only other problem is in branch naming, which could clash if you
only use the outer-most directory name, so I'd suggest something that
involves concatenating the folders in the path relative to 'branches' to
keep them unique (if git can handle slashes in branch names then all the
better).
As Peter suggested, disable globbing for branches and use explicit
fetch refspecs for now...
--
Eric Wong
From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
Eric Wong wrote:
Russ Brown [off-list ref] wrote:
quoted
Hi,
Hi Russ,
quoted
I'm having some trouble with using git-svn to fetch a repository, and I
think it's because the repository doesn't store branches as a flat list
directly under the 'branches' directory.
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
etc. category-a and category-b are simple directories created using svn
mkdir. The branches are created using svn cp.
It helps us to organise the branches better, but the rationale is
besides the point. The problem is that git-svn seems to want to treat
category-a and category-b as branches, which isn't right at all. As a
result, git-svn seems to skip most (if not all) revisions that occur in
these directories and creates a lot of entries in unhandled.log.
This is a known problem with the way git-svn handles branches and tags.
Nested branch (and tags) structures aren't supported with globbing and
so you can't have more than one branches/tags specification in your
config.
quoted
I've also encountered an index corruption in one of the
.git/svn/<branch>/index files which I think it probably related.
Not sure about that. git-svn should detect and attempt to fix index
corruptions (which happened for me since I interrupt git-svn quite
often when working on it).
I've since trashes that repo and it's not happened again, so it must
have been cosmic particles or something. :)
quoted
I've had a quick look at the source myself, but perl isn't my strong
point. What I think it should do is something like recurse down the tree
from the directory given looking for folders that copy from trunk (or
some other branch that already exists). That would work perfectly well
for the default flat branch storage method as well as the one we use.
The globbing functionality of branches in git-svn is some of the ugliest
code I've ever written in my life. Somehow I got it working for the
simple general cases but I've been afraid to touch it for a while...
Yeah, I know what you mean. :) A really nasty thing to get working and
once you do you don't want to touch it again. :)
quoted
The only other problem is in branch naming, which could clash if you
only use the outer-most directory name, so I'd suggest something that
involves concatenating the folders in the path relative to 'branches' to
keep them unique (if git can handle slashes in branch names then all the
better).
As Peter suggested, disable globbing for branches and use explicit
fetch refspecs for now...
I've actually knocked up a rough script which generates a list of
refspec lines for you given a repo URL, trunk reference and branches
directory. It uses svn log -v --xml and pipes it through a couple of
XSLT templates, and basically looks for all copies that copy from trunk
(recursively: so it includes branches of branches too). I can post it to
the list if you'd find it useful or interesting.
It's generating output that looks sensible to me, but the results aren't
quite what I'd expected. I'll paste a sample in here in case there's
anything obvious someone might spot that I've missed
# This line was generated by git-svn init, and I kept it
fetch = all/trunk:refs/remotes/trunk
# These lines generated by my tool, dirnames replaced for security reasons:
fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
git branch -a doesn't list any of those branches after fetch completes.
Looking back at the output from fetch, all revisions applied were to trunk.
Anything wrong with those fetch lines?
Thanks for your time.
--
Russ
From: Eric Wong <hidden> Date: 2016-06-15 22:43:33
Russ Brown [off-list ref] wrote:
Eric Wong wrote:
quoted
Russ Brown [off-list ref] wrote:
quoted
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
quoted
quoted
The only other problem is in branch naming, which could clash if you
only use the outer-most directory name, so I'd suggest something that
involves concatenating the folders in the path relative to 'branches' to
keep them unique (if git can handle slashes in branch names then all the
better).
As Peter suggested, disable globbing for branches and use explicit
fetch refspecs for now...
I've actually knocked up a rough script which generates a list of
refspec lines for you given a repo URL, trunk reference and branches
directory. It uses svn log -v --xml and pipes it through a couple of
XSLT templates, and basically looks for all copies that copy from trunk
(recursively: so it includes branches of branches too). I can post it to
the list if you'd find it useful or interesting.
It's generating output that looks sensible to me, but the results aren't
quite what I'd expected. I'll paste a sample in here in case there's
anything obvious someone might spot that I've missed
# This line was generated by git-svn init, and I kept it
fetch = all/trunk:refs/remotes/trunk
# These lines generated by my tool, dirnames replaced for security reasons:
fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
git branch -a doesn't list any of those branches after fetch completes.
Looking back at the output from fetch, all revisions applied were to trunk.
Anything wrong with those fetch lines?
From your tree diagram, it seemed that trunk/ and branches/ were at
the same depth in your SVN repository. However, in your generated
fetch lines they all started with "branches/" in front, yet your
trunk fetch line had "all/" in front of trunk, so maybe prefixing
the generated ones with "all/" helps?
From: Russ Brown <hidden> Date: 2016-06-15 22:43:33
Eric Wong wrote:
Russ Brown [off-list ref] wrote:
quoted
Eric Wong wrote:
quoted
Russ Brown [off-list ref] wrote:
quoted
Basically, we have a structure like this:
|
+-trunk
+-tags
+-branches
+ category-a
+ branch-a
+ branch-b
+ category-b
+ branch-c
+ branch-d
quoted
quoted
quoted
The only other problem is in branch naming, which could clash if you
only use the outer-most directory name, so I'd suggest something that
involves concatenating the folders in the path relative to 'branches' to
keep them unique (if git can handle slashes in branch names then all the
better).
As Peter suggested, disable globbing for branches and use explicit
fetch refspecs for now...
I've actually knocked up a rough script which generates a list of
refspec lines for you given a repo URL, trunk reference and branches
directory. It uses svn log -v --xml and pipes it through a couple of
XSLT templates, and basically looks for all copies that copy from trunk
(recursively: so it includes branches of branches too). I can post it to
the list if you'd find it useful or interesting.
It's generating output that looks sensible to me, but the results aren't
quite what I'd expected. I'll paste a sample in here in case there's
anything obvious someone might spot that I've missed
# This line was generated by git-svn init, and I kept it
fetch = all/trunk:refs/remotes/trunk
# These lines generated by my tool, dirnames replaced for security reasons:
fetch = branches/folder/projecta:refs/remotes/svn/folder/projecta
fetch = branches/folder/projectb:refs/remotes/svn/folder/projectb
fetch = branches/folder/projectc:refs/remotes/svn/folder/projectc
fetch = branches/folder/projectd:refs/remotes/svn/folder/projectd
fetch = branches/folder/projecte:refs/remotes/svn/folder/projecte
fetch = branches/folder/projectf:refs/remotes/svn/folder/projectf
fetch = branches/folder/projectg:refs/remotes/svn/folder/projectg
git branch -a doesn't list any of those branches after fetch completes.
Looking back at the output from fetch, all revisions applied were to trunk.
Anything wrong with those fetch lines?
From your tree diagram, it seemed that trunk/ and branches/ were at
the same depth in your SVN repository. However, in your generated
fetch lines they all started with "branches/" in front, yet your
trunk fetch line had "all/" in front of trunk, so maybe prefixing
the generated ones with "all/" helps?
Ah! I think I know what I've done. I generated the fetch list from the
svn repository itself, but ran svn-init against an svk mirror (using
--use-svm-props). Nicely spotted. :)
Thanks again!