Re: Git performance on OS X

On Apr 19, 2008, at 5:06 PM, Linus Torvalds wrote:

On Sat, 19 Apr 2008, Roman Shaposhnik wrote:

quoted

quoted

Is there any system-level profiler for OS X to get a clue where  
that cost
is, in case it's not the lstat() at all?

If it happens on Leopard, DTrace would be a perfect way to query  
the system:

Well, I'd really like to see a traditional _time_ profile, not system
call counts.

Good point. I just thought your original question was simply about  
confirming
a hunch of an enormous # of lstat syscalls. Now, if by _time_ profile
you mean how much time gets spent in each syscall than the following
should help (time is in nanoseconds):

$ dtrace -n 'syscall::*:entry /pid==$target/ { self->ts=timestamp; }  
syscall::*:return /pid==$target/ { @[probefunc]=sum(timestamp-self- 
 >ts); }' -c "echo Hello World"
dtrace: description 'syscall::*:entry ' matched 468 probes
Hello World
dtrace: pid 1400 has exited

   getpid                                                         1392
   sysi86                                                         2799
   getrlimit                                                      2918
   setcontext                                                     4273
   fstat64                                                        6220
   mmap                                                          15419
   munmap                                                        22593
   write                                                         27860
   ioctl                                                         30750

The script can be modified slightly to also profile all of the libc.

On the other hand, if by real system profile you mean a full fledged
sampling of the application itself than I can suggest running Git
under Shark, and not under Instruments. Although I'm extremly
curious about *why* would you need a full fledged *application*
level profile of Git as opposed to a profile of how Git interacts
with an OS.

The system call profile is trivial - it's generally going to be pretty
similar under OS X and Linux (modulo library differences, but git  
doesn't
really use any really complex libraries that would do system calls).

The problem we've had in the past is that Linux is simply an order of
magnitude faster (sometimes more) at some operations than OS X is, so
issues that show up on OS X don't even show up on Linux.

This was the case for doing lots of small "mmap()/munmap()" calls, for
example, where we literally had a load where OS X was two orders of
magnitude slower. We switched over to reading the files with "pread()"
instead of mmap(), and that fixed that particular issue.

Totally understood!

Thanks,
Roman.