Re: kernel.org and GIT tree rebuilding

On Tue, 28 Jun 2005, Nicolas Pitre wrote:

Here's one improvement to the pack format, breaking it early so it won't 
affect anyone at this point: compressed object header.  Instead of the 
fixed 5 byte header, this patch convert it to a variable size granted 
most object are small enough to save on the storage of the significant 
size bytes which will be zero and packing the non-zero byte position 
with the object type.

Ok, this is against an older version and doesn't have that "read_sha1" 
thing, but yes, something like this would work.

I'd prefer the encoding to be a bit different, though: make the size be
encoded in seven bits per byte, with the high bit meaning "more to come". 
We can use four bits from the "type" byte for the initial value, making 
lengths 0-15 be free.

	unsigned long size;
	unsigned char c;

	c = *pack++;
	type = 
	size = c & 15;
	type = (c >> 4) & 7;
	while (c & 0x80) {
		c = *pack++;
		size = (size << 7) + (pack & 0x7f);
	}

or something. That's even denser.

However, I also end up wanting to add a "global header" to the pack-file, 
that contains at least the number of objects packed. We may not know how 
big the pack-file will be, but we'll at least know how many objects it 
has before we start writing it.

		Linus