[RFH] git-log vs git-rev-list performance

Hi all,

   perhaps this will turn out to be a bit academic, anyway...

I'm almost ready to release qgit-2.1, a lot of improvements went in
but still I cannot beat the speed of stable qgit-1.5 series.

I really profiled this puppy and I think qgit-2.1 is _internally_ much
faster then qgit-1.5, but at the end of the day qgit-1.5 is about 13%
faster then the new 2.1 at loading repositories.


The loading speed is the sum of two factors:

- speed of underlying git-rev-list / git-log command

- qgit overhead in parsing and storing the git plumbing outpt


I think qgit-2.0 code is much more efficient then the old one because
the overhead went down from 17% of qgit-1.5 to current 6% of qgit-2.1,
it means that opening and loading a repository with qgit-2.1 is only
6% slower then running the underlying git command from the command
line.


So the problem seems to be in the underlying command that for qgit-1.5
is git-rev-list while for new qgit-2.1 is git-log


To have some numbers I have tested on the Linux repository with the
actual git commands used by the two versions of qgit:

[marco@localhost linux-2.6]$ git --version
git version 1.5.4-rc1.GIT


[marco@localhost linux-2.6]$ time git log --topo-order --no-color
--parents --boundary -z --log-size
--pretty=format:"%m%HX%PX%n%an<%ae>%n%at%n%s%n%b" HEAD > /dev/null
3.60user 0.09system 0:03.70elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+27156minor)pagefaults 0swaps


[marco@localhost linux-2.6]$ time git rev-list --topo-order --no-color
--parents --boundary -z --header HEAD > /dev/null
2.89user 0.08system 0:02.98elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+27156minor)pagefaults 0swaps


[ BTW, yes I have a new laptop! ;-) ]


Note that the output is smaller for git-log because --pretty=format
asks for less info then the --header option used by git-rev-list. To
be precise, output is  57.113.153 bytes for git-rev-list against
41.755.328 bytes for git-log.



So the bottom line is that git-log is 24% slower then git-rev-list
although size of its output is 36% smaller!


Could someone be so kind to explain me why these differences? I'm not
so confident with git-log /git-rev-list internals.


Thanks
Marco