Re: git-index-pack really does suck..

On Tue, 3 Apr 2007, Linus Torvalds wrote:

On Tue, 3 Apr 2007, Dana How wrote:

quoted

Larger and larger pack files make me nervous.
They are expensive to manipulate,
and >2GB requires a file format change.

It sometimes also requires a new filesystem. There are a lot of
filesystems that can handle more than 4GB total, but not necessarily in a
single file.

The only really useful such filesystem is probably FAT, which is still
quite useful for things like USB memory sticks. But that is probably
already worth supporting.

So I think we want to support 64-bit (or at least something like 40+ bit)
pack-files, but yes, I think that even if/when we support it, we still
want to support the "multiple smaller pack-files" schenario exactly
because for some uses it's much *better* to have ten 2GB packfiles rather
than one 20GB pack-file.

however, for historical archives you may end up with wanting to do a 2GB 
packfile of 'recent' stuff, and a 14GB packfile of 'ancient' stuff (with the 
large one built with all space-saving options turned up all the way, no matter 
how much time it takes to build the pack)

David Lang