From: Thomas Rast <hidden> Date: 2016-06-15 22:57:31
Thomas Rast [off-list ref] writes:
However, if that turns out to be the culprit, it's not fixable
currently[1]. Having commits with insanely long messages is just, well,
insane.
[1] unless we do a major rework of the loading infrastructure, so that
we can teach it to load only the beginning of a commit as long as we are
only interested in parents and such
Actually, Peff, doesn't your commit parent/tree pointer caching give us
this for free?
--
Thomas Rast
trast@{inf,student}.ethz.ch
From: Jeff King <hidden> Date: 2016-06-15 22:57:31
On Fri, May 31, 2013 at 12:27:11PM +0200, Thomas Rast wrote:
Thomas Rast [off-list ref] writes:
quoted
However, if that turns out to be the culprit, it's not fixable
currently[1]. Having commits with insanely long messages is just, well,
insane.
[1] unless we do a major rework of the loading infrastructure, so that
we can teach it to load only the beginning of a commit as long as we are
only interested in parents and such
Actually, Peff, doesn't your commit parent/tree pointer caching give us
this for free?
It does. You can test it from the "jk/metapacks" branch at
git://github.com/peff/git. After building, you'd need to do:
$ git gc
$ git metapack --all --commits
in the target repository. You can check that it's working because "git
rev-list --all --count" should be an order of magnitude faster. You may
need to add "save_commit_buffer = 0" in any commands you are checking,
though, as the optimization can only kick in if parse_commit does not
want to save the buffer as a side effect.
I also looked into trying to just read the beginning part of a commit[1],
but it turned out not to be all that much of an improvement.
-Peff
[1] http://article.gmane.org/gmane.comp.version-control.git/212301
From: Alex Bennée <hidden> Date: 2016-06-15 22:57:32
On 31 May 2013 17:17, Jeff King [off-list ref] wrote:
On Fri, May 31, 2013 at 12:27:11PM +0200, Thomas Rast wrote:
quoted
Thomas Rast [off-list ref] writes:
quoted
However, if that turns out to be the culprit, it's not fixable
currently[1]. Having commits with insanely long messages is just, well,
insane.
[1] unless we do a major rework of the loading infrastructure, so that
we can teach it to load only the beginning of a commit as long as we are
only interested in parents and such
Actually, Peff, doesn't your commit parent/tree pointer caching give us
this for free?
It does. You can test it from the "jk/metapacks" branch at
git://github.com/peff/git. After building, you'd need to do:
$ git gc
$ git metapack --all --commits
in the target repository. You can check that it's working because "git
rev-list --all --count" should be an order of magnitude faster. You may
need to add "save_commit_buffer = 0" in any commands you are checking,
though, as the optimization can only kick in if parse_commit does not
want to save the buffer as a side effect.
Is this a command line argument? The tools don't seem to think so.
Anyway it seems to make a marginal difference to my case:
09:08 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771
real 0m14.105s
user 0m12.409s
sys 0m1.660s
09:11 ajb@sloy/x86_64 [work.git] >git gc
Counting objects: 399436, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (110874/110874), done.
Writing objects: 100% (399436/399436), done.
Total 399436 (delta 281538), reused 398357 (delta 280493)
Checking connectivity: 399436, done.
09:12 ajb@sloy/x86_64 [work.git] >git metapack --all --commits
09:13 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771
real 0m12.781s
user 0m11.669s
sys 0m1.080s
09:32 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771
real 0m12.768s
user 0m11.817s
sys 0m0.908s
09:33 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags
ajb-build-test-5225-2-gdc0b771
real 0m12.642s
user 0m11.705s
sys 0m0.904s
From: Jeff King <hidden> Date: 2016-06-15 22:57:32
On Mon, Jun 03, 2013 at 09:39:21AM +0100, Alex Bennée wrote:
quoted
in the target repository. You can check that it's working because "git
rev-list --all --count" should be an order of magnitude faster. You may
need to add "save_commit_buffer = 0" in any commands you are checking,
though, as the optimization can only kick in if parse_commit does not
want to save the buffer as a side effect.
Is this a command line argument? The tools don't seem to think so.
If you mean the "save_commit_buffer = 0", no; I mean you would have to
insert it somewhere in builtin/$CMD.c, and then recompile. However,
git-describe already has it, so it should work.
Anyway it seems to make a marginal difference to my case:
I get much better results:
$ cd linux-2.6
$ time git --no-pager describe --long --tags HEAD~800
v3.5-6956-gaa0b3b2
real 0m0.261s
user 0m0.248s
sys 0m0.012s
$ git metapack --commits --all
$ time git --no-pager describe --long --tags HEAD~800
v3.5-6956-gaa0b3b2
real 0m0.057s
user 0m0.032s
sys 0m0.024s
which implies that your time is being spent elsewhere. That topic
wouldn't avoid inflating tag objects from disk. Do you have really big
tag objects (or unannotated tags pointing to blobs)? What does:
git for-each-ref --format='%(object)' refs/tags |
git cat-file --batch-check |
sort -k 3nr |
head
say?
-Peff