Re: Git is not scalable with too many refs/*

On Mon, 26 Sep 2011 12:07:52 -0600, Martin Fick wrote:
-- snip --

Ahh, I think I have some more clues.  So while this change
does not speed things up for me normally, I found a case
where it does!  I  set my .git/config to have

  [core]
        compression = 0

and ran git-gc on my repo.  Now, with a modern git with this
optimization in it (1.7.6, 1.7.7.rc0...), 'git branch' is
almost instantaneous (.05s)!  But, if I revert c774aa it
takes > ~15s.

I don't understand this.  I don't see why core.compression should have 
anything to do with refs ...

So, it appears that this optimization is foiled by
compression?  In the case when this optimization helps, it
save about 15s, when it hurts (with compression), it seems
to cost > 3mins.  I am not sure this optimization is worth
it?  Would there be someway for it to adjust to the repo
conditions?

Well, in the case I tried it was 1.2s vs 40s.  It would seem that you 
have managed to find some corner case.  It doesn't seem right to punish 
everyone who has large numbers of refs by making their commands take 
orders of magnitude longer to save one person 3m.  Much better to find, 
understand and fix the actual cause.

I really can't see what effect core.compression can have on loading the 
ref_list.  Certainly the sort doesn't load anything from the object 
database.  It would be really good to profile and find out what is 
taking all the time - I am assuming that the CPU is at 100% for the 3+ 
minutes?

Random thought.  What happens to the with compression case if you leave 
the commit in, but add a sleep(15) to the end of sort_refs_list?

-- 
Julian