Re: git-index-pack really does suck..

On Tue, 3 Apr 2007, Chris Lee wrote:

There's another issue here.

I'm running git-index-pack as part of a workflow like so:

$ git-verify-pack -v .git/objects/pack/*.idx > /tmp/all-objects
$ grep 'blob' /tmp/all-objects > /tmp/blob-objects
$ cat /tmp/blob-objects | awk '{print $1;}' | git-pack-objects
--delta-base-offset --all-progress --stdout > blob.pack
$ git-index-pack -v blob.pack

Instead of using --stdout with git-pack-object, you should provide it 
with a suitable base name for the resulting pack and the index will be 
created automatically along side the pack for you.  No need to use 
index-pack for that.

Now, when I run 'git-index-pack' on blob.pack in the current
directory, memory usage is pretty horrific (even with the applied
patch to not leak all everything). Shawn tells me that index-pack
should only be decompressing the object twice - once from the repo and
once from blob.pack - iff I call git-index-pack with --stdin, which I
am not.

If I move the blob.pack into /tmp, and run git-index-pack on it there,
it completes much faster and the memory usage never exceeds 200MB.
(Inside the repo, it takes up over 3G of RES according to top.)

The 3G should definitely be fixed with the added free().

The CPU usage is explained by the fact that you're running index-pack on 
objects that are all already found in your repo so the collision check 
is triggered.  This is more or like the same issue as if you tried to 
run unpack-objects on the same pack where none of your objects will 
actually be unpacked.


Nicolas