Re: pack operation is thrashing my server

On 2008.08.14 16:14:26 -0700, Linus Torvalds wrote:

On Thu, 14 Aug 2008, Nicolas Pitre wrote:

quoted

Possible.  However, the fact that both the "Compressing objects" and the 
"Writing objects" phases during a repack (without -f) together are 
_faster_ than the "Counting objects" phase is a sign that something is 
more significant than cache misses here, especially when tree 
information is a small portion of the total pack data size.

Hmm. I think I may have clue.

The size of the delta cache seems to be a sensitive parameter for this 
thing. Not so much for the git archive, but working on the kernel tree, 
raising it to 1024 seems to give a 20% performance improvement. That, in 
turn, implies that we may be unpacking things over and over again because 
of bad locality wrt delta generation. 

Since you mention the delta cache, uau (no idea about his real name) on
#git was talking about some delta cache optimizations lately, although
he was dealing with "git log -S", maybe it affects rev-list in a similar
way. Unfortunately, I can't seem to find any code for that, just a
description of what he did and some numbers on the results in the IRC
logs.

http://colabti.org/irclogger/irclogger_log/git?date=2008-08-04,Mon#l65

Maybe that helps in some way.

Björn