Re: Git performance on OS X

On 20 apr 2008, at 01:26, Linus Torvalds wrote:

It may be that the problem with OS X is a sucky pathname cache  
mechanism.

The trivial patch cut down the number of stat() calls by a fair  
amount,
but the calls that got removed were all of the "do two 'lstat()'  
calls on
the exact same pathname consecutively" type.

Maybe OS X has some very limited pathname caching that catches that,  
or
even if not, it just ends up being very nice in the D$, so it's not  
a big
deal. And then the real suckiness happens only with bigger workloads.

Yes, I just tested this.

	for (int i = 0; i < 50000; i++) {
		sprintf(s, "/Users/pieter/test/perf/%i", i);
		int ret = lstat(s, a);
	}

This loop needs about 3 seconds to  run. Replacing the i with 10 in  
the sprintf reduces it to 0.24seconds.

quoted

Command                                            Mean     Std
git status .                                       13.970  1.298
/Users/pieter/projects/External/git/git-status .   13.759  0.321

This is the WebKit archive, right?

For me, doing a "time git status ." on the WebKit thing I just  
cloned from
git://git.webkit.org/WebKit.git is much faster: 1.264s (and it goes  
down
by maybe 5-10% with my lstat-avoidance patch).

Is there any system-level profiler for OS X to get a clue where that  
cost
is, in case it's not the lstat() at all?

Yes, that was the webkit repo (the test above was in a dir with 50k  
files).

Alas, I tried to create a nice profiling for the "git status .". In  
the Instruments application I can create a sampler, but I see no way  
to export it. The option to export the script as a dtrace script is  
greyed out in the menu.

 From the sampler, it appears that the lstat calls still account for  
most of the time. I have uploaded a screenshot to http://ss.frim.nl/==759.png 
. It actually shows quite nicely when the lstats are being done --  
it's when the CPU is idle. Next to the lstats, the read_tree_recursive  
is also called often.

- Pieter