From: Junio C Hamano <hidden> Date: 2016-06-15 22:57:06
Eric Wong [off-list ref] writes:
Ilya Basin [off-list ref] wrote:
quoted
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
Sounds good, but it's not I who decided to use the config file.
Ugh, I didn't review Ray's original commit closely enough to notice
this :x
Perhaps we should migrate users to use YAML storage for this, instead
(we already use YAML for Git::SVN::Memoize::YAML).
But does it solve the impedance mismatch between "per tree"
information and "per project" information? Unless you key the
information not just with path but also with revision or tree object
name, use of YAML vs config would not make a difference in the
semantics, I am afraid.
I am reading the placeholder-added flag as: "This .gitignore file
does not exist in the Subversion original; it is there only so that
we can keep the otherwise empty diretory in the checkout, and it
should not be pushed back to the Subversion side". Am I mistaken?
That however is not a property of the directory containing it (or
the path to that .gitignore file) that is valid throughout the
history of the project. It is a property of a specific tree object
(or you could say it is a property of the revision). When at some
point in the history the upstream project adds .gitignore there
because many people use git-svn to contribute to their project, it
stops to be "should not be pushed back".
So it seems to me that the information this "placeholder added"
thing wants to express belongs to the tree object (and .gitignore
file itself is a natural place to have that information).
Fwiw, I've never been a fan of placeholders only accepted it since it's
off-by-default but it worked well enough for Ray.
My personal philosophy has always been: git svn users should leave
no trace or indication they're using a non-standard SVN client.
From: Eric Wong <hidden> Date: 2016-06-15 22:57:06
Junio C Hamano [off-list ref] wrote:
Eric Wong [off-list ref] writes:
quoted
Ilya Basin [off-list ref] wrote:
quoted
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
Sounds good, but it's not I who decided to use the config file.
Ugh, I didn't review Ray's original commit closely enough to notice
this :x
Perhaps we should migrate users to use YAML storage for this, instead
(we already use YAML for Git::SVN::Memoize::YAML).
But does it solve the impedance mismatch between "per tree"
information and "per project" information? Unless you key the
information not just with path but also with revision or tree object
name, use of YAML vs config would not make a difference in the
semantics, I am afraid.
No it doesn't solve the impedance mismatch, but the YAML project would
be more flexible than the git config file.
I am reading the placeholder-added flag as: "This .gitignore file
does not exist in the Subversion original; it is there only so that
we can keep the otherwise empty diretory in the checkout, and it
should not be pushed back to the Subversion side". Am I mistaken?
You're right, I had forgotten this feature completely :x
That however is not a property of the directory containing it (or
the path to that .gitignore file) that is valid throughout the
history of the project. It is a property of a specific tree object
(or you could say it is a property of the revision). When at some
point in the history the upstream project adds .gitignore there
because many people use git-svn to contribute to their project, it
stops to be "should not be pushed back".
So it seems to me that the information this "placeholder added"
thing wants to express belongs to the tree object (and .gitignore
file itself is a natural place to have that information).
Perhaps that was the better way to go...
How would (the presumably few) existing users of this feature be
affected?
Currently with the config file, there are problems with interop between
git-svn users that do git <-> git repo sharing, an updated version with
the "placeholder added" .gitignore would allow git <-> git repo sharing,
but only between users of newer git versions. Perhaps that's fine and
better than the current situation.
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
But the config file is not an option too: I have 400 tags, each has
200 empty folders.
Instead I decided to store the paths in a text file (see
https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).
I'm not planning to push this change to you.
The last error I encountered is:
r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)
Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.
I think It's unrelated to empty dirs.
On Wed, May 1, 2013 at 10:49 PM, Eric Wong [off-list ref] wrote:
Junio C Hamano [off-list ref] wrote:
quoted
Eric Wong [off-list ref] writes:
quoted
That however is not a property of the directory containing it (or
the path to that .gitignore file) that is valid throughout the
history of the project. It is a property of a specific tree object
(or you could say it is a property of the revision). When at some
point in the history the upstream project adds .gitignore there
because many people use git-svn to contribute to their project, it
stops to be "should not be pushed back".
So it seems to me that the information this "placeholder added"
thing wants to express belongs to the tree object (and .gitignore
file itself is a natural place to have that information).
Perhaps that was the better way to go...
How would (the presumably few) existing users of this feature be
affected?
Currently with the config file, there are problems with interop between
git-svn users that do git <-> git repo sharing, an updated version with
the "placeholder added" .gitignore would allow git <-> git repo sharing,
but only between users of newer git versions. Perhaps that's fine and
better than the current situation.
The original patch was geared towards increasing the fidelity of a
one-time svn->git migration (ie. where svn won't be used anymore). I
recall investigating a method to enforce this by disallowing future
git-svn fetches, but I can't remember if I was successful. Given this
perspective, I'm not sure that existing users need to be supported.
Then, as Junio mentions, future versions of git that store placeholder
info in the tree/file object could open the possibility of proper
git<->git sharing and resync with the original svn repo.
- Ray
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
But the config file is not an option too: I have 400 tags, each has
200 empty folders.
Instead I decided to store the paths in a text file (see
https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).
I'm not planning to push this change to you.
The last error I encountered is:
r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)
Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.
I think It's unrelated to empty dirs.
Can you get an lsof on the git-svn process right before this?
What's your open files limit?
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
But the config file is not an option too: I have 400 tags, each has
200 empty folders.
Instead I decided to store the paths in a text file (see
https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).
I'm not planning to push this change to you.
The last error I encountered is:
r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)
Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.
I think It's unrelated to empty dirs.
EW> Can you get an lsof on the git-svn process right before this?
/.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc
/.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN
/.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz
etc.
EW> What's your open files limit?
1024
--
JCH> comment line "# added by git-svn only to keep the directory" and
JCH> consider a directory that has nothing but .gitignore that consists
JCH> of only that exact comment line an "added placeholder" directory to
JCH> work it around.
But the config file is not an option too: I have 400 tags, each has
200 empty folders.
Instead I decided to store the paths in a text file (see
https://github.com/basinilya/git/commit/a961aedd81cb8676a52cfe71ccb6eba0f9e64b90 ).
I'm not planning to push this change to you.
The last error I encountered is:
r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)
Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.
I think It's unrelated to empty dirs.
EW>> Can you get an lsof on the git-svn process right before this?
IB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc
IB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN
IB> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz
IB> etc.
EW>> What's your open files limit?
IB> 1024
Why no call to close() from temp_release() in Git.pm?
--
The last error I encountered is:
r7009 = 39805bb078983e34f2fc8d2c8c02d695d00d11c0 (refs/remotes/DMC4_Basic)
Too many open files: Can't open file '/home/il/builds/sicap/gitsvn/prd_dmc4.svn/db/revs/0/786': Too many open files at /.snapshots/persist/builds/git/git-git/perl/blib/lib/Git/SVN/Ra.pm line 282.
I think It's unrelated to empty dirs.
EW>>> Can you get an lsof on the git-svn process right before this?
IB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/A4O_OTQxWc
IB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/LfpcENJduN
IB>> /.snapshots/persist/builds/sicap/gitsvn/aaa/.git/Dkk7pN4Mpz
IB>> etc.
EW>>> What's your open files limit?
IB>> 1024
IB> Why no call to close() from temp_release() in Git.pm?
Found, fixed. It was related to empty dirs.
--
Hi Eric. I'm out of spare time and I still unable to import my repo.
The code of SVN.pm is too complex. Please help me.
Here's the list of my issues:
* I think git-svn doesn't handle the case, when a tag is deleted.
I expected it to rename the ref from "tags/tagname" to
"tags/tagname@rev", but that doesn't happen.
If a tag is replaced, there's no way to tell what was the previous
state of that tag: git-svn just rewrites the ref.
On the contrary, the temporary refs (with "@rev" suffix), used for
re-import subdir tags are kept after successful reimport, although
they have no usage.
* As I said already, I have 25k revisions and 200 tags created from
subdirs in trunk. This increases the import time from 2h to 12h.
I would bear it, if it had to be done once, but fetching a new
revision may cause re-import of all 25k revisions too.
You should implement some mechanism to find the parent branches of
subdir tags. Maybe the unused refs I mentioned in the previous issue
are good candidates for that, but I would name them somehow
different to distinguish with deleted/replaced tags/branches.
* There are mistake commits in the svn history, similar to this:
------------------------------------------------------------------------
r21255 | xxx_xxxxxx_xxxxxxxxx | 2012-03-02 18:46:30 +0300 (Fri, 02 Mar 2012) | 1 line
Changed paths:
A /tags/dmagentenabler-4.1.31/DMAgent (from /tags:20998)
Delivery 4.1.31
------------------------------------------------------------------------
git-svn tries to creates a tag, containing dirs with other tags.
Technically, behaves correctly, but it hangs, because of the size of
the commit.
To solve it, I had to edit the svn dump file:
Node-path: tags/dmagentenabler-4.1.31/DMAgent
Node-kind: dir
Node-action: add
-Node-copyfrom-rev: 20998
-Node-copyfrom-path: tags
+Prop-content-length: 10
+Content-length: 10
+
+PROPS-END
Revision-number: 21256
It creates an empty dir, instead of copying. Since the author
noticed the mistake, he immediately deleted the dir in the next
revision, so it works.
From: Eric Wong <hidden> Date: 2016-06-15 22:57:10
Ilya Basin [off-list ref] wrote:
Hi Eric. I'm out of spare time and I still unable to import my repo.
The code of SVN.pm is too complex. Please help me.
Sorry, most what I do nowadays for git-svn is ACK/NACK changes.
git-svn has made itself obsolete for most contributors, myself included;
so it's hard for us to devote significant amounts of time on it since
we no longer see SVN repos in our day-to-day work.
Given the differences between branching/tagging in SVN and git, I
suspect some history may always be too complex/convoluted to
automatically import. Perhaps an interactive mode can be introduced
to follow history...
Anyways, thank you for documenting these issues and suggesting fixes.
Hopefully somebody with sufficient motivation can continue your work
down the line.
IB> * I think git-svn doesn't handle the case, when a tag is deleted.
IB> I expected it to rename the ref from "tags/tagname" to
IB> "tags/tagname@rev", but that doesn't happen.
IB> If a tag is replaced, there's no way to tell what was the previous
IB> state of that tag: git-svn just rewrites the ref.
OK, I figured out that git-svn creates a merge commit having one of
its parents the previous state of the tag and another parent the state
of the new copy src folder.