Re: Something is broken in repack

On 12/9/07, Junio C Hamano [off-list ref] wrote:

Junio C Hamano [off-list ref] writes:

quoted

Nicolas Pitre [off-list ref] writes:

quoted

On Fri, 7 Dec 2007, Jon Smirl wrote:

quoted

Starting with a 2GB pack of the same data my process size only grew to
3GB with 2GB of mmaps.

Which is quite reasonable, even if the same issue might still be there.

So the problem seems to be related to the pack access code and not the
repack code.  And it must have something to do with the number of deltas
being replayed.  And because the repack is attempting delta compression
roughly from newest to oldest, and because old objects are typically in
a deeper delta chain, then this might explain the logarithmic slowdown.

So something must be wrong with the delta cache in sha1_file.c somehow.

I was reaching the same conclusion but haven't managed to spot anything
blatantly wrong in that area.  Will need to dig more.

Does this problem have correlation with the use of threads?  Do you see
the same bloat with or without THREADED_DELTA_SEARCH defined?

I just started a non-threaded one. It will be four or five hours
before it finishes.

-- 
Jon Smirl
jonsmirl@gmail.com