Re: q: git-fetch a tad slow?

Ingo Molnar [off-list ref] wrote:

* Shawn O. Pearce [off-list ref] wrote:

quoted

Ingo Molnar [off-list ref] wrote:

quoted

Setup/background: distributed kernel testing cluster, [...]

Problem: i noticed that git-fetch is a tad slow:

  titan:~/tip> time git-fetch
  real    0m2.372s

note that titan is a very beefy box, almost 3 GHz Core2Duo:

That isn't going to matter if you have a quadratic algorithm and a
large dataset.  Especially when the inner loops are doing multiple
system calls per item in a long list of items.  :-|   Linux is fast,
but it isn't magic pixie dust.  It cannot fix broken applications.
 

[...] So if we have a quadratic overhead on number of 
branches, that's going to be quite a PITA.

Right.

quoted

I wonder if git-pack-refs + fetching only a single branch will get you 
closer to the tip-fetch time.

should i pack on both repos? I dont explicitly pack anything, but on the 
server it goes into regular gc runs. (which will pack most stuff, 
right?)

git-gc automatically runs `git pack-refs --all --prune` like I
recommended, unless you disabled it with config gc.packrefs = false.
So its probably already packed.

What does `find .git/refs -type f | wc -l` give for the repository
on the central server?  If its more than a handful (~20) I would
suggest running git-gc before testing again.

But I'm really suspecting that this is just our quadratic matching
algorithm running up against a large number of branches, causing
it to suck.

jgit at least uses an O(N) algorithm here, but since it is written
in Java its of course slow compared to C Git.  Takes a while to
get that JVM running.

I'll try to find some time to reproduce the issue and look at the
bottleneck here.  I'm two days into a new job so my git time has
been really quite short this week.  :-|

-- 
Shawn.