Re: Poor performance of git describe in big repos

On 31 May 2013 17:17, Jeff King [off-list ref] wrote:

On Fri, May 31, 2013 at 12:27:11PM +0200, Thomas Rast wrote:

quoted

Thomas Rast [off-list ref] writes:

quoted

However, if that turns out to be the culprit, it's not fixable
currently[1].  Having commits with insanely long messages is just, well,
insane.

[1]  unless we do a major rework of the loading infrastructure, so that
we can teach it to load only the beginning of a commit as long as we are
only interested in parents and such

Actually, Peff, doesn't your commit parent/tree pointer caching give us
this for free?

It does. You can test it from the "jk/metapacks" branch at
git://github.com/peff/git. After building, you'd need to do:

  $ git gc
  $ git metapack --all --commits

in the target repository. You can check that it's working because "git
rev-list --all --count" should be an order of magnitude faster. You may
need to add "save_commit_buffer = 0" in any commands you are checking,
though, as the optimization can only kick in if parse_commit does not
want to save the buffer as a side effect.

Is this a command line argument? The tools don't seem to think so.

Anyway it seems to make a marginal difference to my case:

09:08 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771

real    0m14.105s
user    0m12.409s
sys     0m1.660s
09:11 ajb@sloy/x86_64 [work.git] >git gc
Counting objects: 399436, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (110874/110874), done.
Writing objects: 100% (399436/399436), done.
Total 399436 (delta 281538), reused 398357 (delta 280493)
Checking connectivity: 399436, done.
09:12 ajb@sloy/x86_64 [work.git] >git metapack --all --commits
09:13 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771

real    0m12.781s
user    0m11.669s
sys     0m1.080s
09:32 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771

real    0m12.768s
user    0m11.817s
sys     0m0.908s
09:33 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771

real    0m12.642s
user    0m11.705s
sys     0m0.904s

I also looked into trying to just read the beginning part of a commit[1],
but it turned out not to be all that much of an improvement.

-Peff

[1] http://article.gmane.org/gmane.comp.version-control.git/212301

-- 
Alex, homepage: http://www.bennee.com/~alex/