Re: Poor performance of git describe in big repos

On 31 May 2013 09:46, Thomas Rast [off-list ref] wrote:

Alex Bennée [off-list ref] writes:

quoted

I think you are right. I was brave (well I assumed the tags would come
back from the upstream repo) and ran:

git for-each-ref | grep "refs/tags" | grep "commit" | cut -d '/' -f 3
| xargs git tag -d

So that deleted all unannotated tags pointing at commits, and then it
was fast.  Curious.

quoted

However I have some big commits it seems:

09:37 ajb@sloy/x86_64 [work.git] >(git for-each-ref | grep ' commit' |
cut -d\  -f1 | xargs -n1 git cat-file commit) | wc -c
1147231984

How many unique entries are there in that list, i.e., what does

  git for-each-ref | grep ' commit' | cut -d\  -f1 | sort -u | wc -l

09:49 ajb@sloy/x86_64 [work.git] >git for-each-ref | grep ' commit' |
cut -d\  -f1 | sort -u | wc -l
1508

say?  Perhaps you can also find the biggest commit, e.g. like so:

  git for-each-ref | grep ' commit' | cut -d\  -f1 |
  while read sha; do git cat-file commit $sha | wc -c; done |
  sort -n

Yeah there is a range from a few hundred bytes to a large number of 3M
commits. I guess I need to identify which commits they are and remove
the tags or convert them to annotated reference tags.

However, if that turns out to be the culprit, it's not fixable
currently[1].  Having commits with insanely long messages is just, well,
insane.

[1]  unless we do a major rework of the loading infrastructure, so that
we can teach it to load only the beginning of a commit as long as we are
only interested in parents and such

I'll do a bit of scripting to dig into the nature of these
uber-commits and try and work out how they cam about. I suspect they
are simply start of branch states in our broken and disparate history.

I'll get back to you once I've dug a little deeper.

--
Thomas Rast
trast@{inf,student}.ethz.ch

-- 
Alex, homepage: http://www.bennee.com/~alex/