Re: pack operation is thrashing my server

On Sun, 7 Sep 2008, Jon Smirl wrote:

On 9/7/08, Linus Torvalds [off-list ref] wrote:

quoted

 On Sat, 6 Sep 2008, Jon Smirl wrote:
 >

quoted

When I was playing with those giant Mozilla packs speed of zlib wasn't

 > a big problem. Number one problem was the repack process exceeding 3GB
 > which forced me to get 64b hardware and 8GB of memory. If you start
 > swapping in a repack, kill it, it will probably take a month to
 > finish.


.. and you'd make things much much WORSE?

My observations on the Mozilla packs indicated that the problems were
elsewhere in git, not in the decompression algorithms. Why does a
single 2000 delta chain take 15% of the entire pack time? Something
isn't right when long chains are processed which triggers far more
decompressions than needed.

Please have a look at commit eac12e2d4d7f.  This fix improved things for 
my gcc repack tests.


Nicolas