Re: Figured out how to get Mozilla into git

On Fri, 9 Jun 2006, Linus Torvalds wrote:

On Sat, 10 Jun 2006, Martin Langhoff wrote:

quoted

Now I don't know how much memory or time this took, but it clearly
completed ok. And, it's now a single pack, weighting a grand total of
617MB

Ok, that's more than reasonable. That should be fairly easily mapped on a 
32-bit architecture without any huge problems, even with some VM 
fragmentation going on. It might be borderline (and you definitely want a 
3:1 VM user:kernel split), but considering that the original CVS archive 
was apparently 3GB, having a single 617M pack-file is still pretty damn 
good.  That's like 20% of the original, with all the obvious distribution 
advantages.

I played a bit with git-repack on that repo.  the git-pack-objects 
memory usage grew to around 760MB (git-rev-list was less than that).  So 
LRU of partial pack mappings might bring that down significantly.

Then I used git-repack -a -f --window=20 --depth=20 which produced a 
nice 468MB pack file along with the invariant 45MB index file for a 
grand total of 535MB for the whole repo (the .git/refs/ directory alone 
still occupies 17MB on disk).

So it is probably worth having deeper delta chains for large historic 
repositories as the deep revisions are unlikely to be referenced that 
often while the saving is quite significant.


Nicolas