Re: git-svnimport failed and now git-repack hates me

Linus Torvalds [off-list ref] wrote:

On Wed, 3 Jan 2007, Chris Lee wrote:

quoted

First issue I ran into: On a machine with 4GB of RAM, when I tried to
do a full import, git-svnimport died after 309906 revisions, saying
that it couldn't fork.

Managing memory with the Perl SVN libraries has been very painful in my
experience.

Part of it is Perl, which (as far as I know) never frees allocated
memory back to the OS (although Perl can reuse the allocated memory for
other things).  I'm CC-ing the resident Perl guru on this...

I'm also fairly certain that most higher-level languages have this
problem.

I suspect you'd have been better off just re-starting, and using something 
like

	while :
	do
		git svnimport -l 1000 <...>
		.. figure out some way to decide if it's all done ..
		git repack -d
	done

However, I don't know what the proper magic is for svnimport to do that 
sane "do it in chunks and tell when you're all done". Or even better - to 
just make it repack properly and not keep everything in memory.

<shameless self-promotion>
	git-svn already does this chunking internally

	Just set the repack interval to something smaller than 1000;
	(--repack=100) if you experience timeouts.
</shameless self-promotion>

-- 
Eric Wong