Re: pack v4 trees with a canonical base

Nicolas Pitre [off-list ref] writes:

very much appreciated to> On Tue, 10 Sep 2013, Junio C Hamano wrote:

quoted

Duy Nguyen [off-list ref] writes:

quoted

On Tue, Sep 10, 2013 at 2:25 AM, Nicolas Pitre [off-list ref] wrote:

quoted

An eventual optimization to index-pack when completing a pack would be
to attempt the encoding of appended tree objects into the packv4 format
using the existing dictionary table in the pack, and fall back to the
canonical format if that table doesn't have all the necessary elements.

Yeah, it's on the improvement todo list. The way pack-objects creates
dictionaries right now, the tree dict should contain all elements the
base trees need so fall back is only necessary when trees are have
extra zeros in mode field.

Careful.

There may be trees in the wild that record 100775 or 100777 in the
mode field for executable blobs, which also need to be special
cased.

All the file mode bits are always preserved.  So this is not really a 
special case as far as the pack v4 encoding is concerned.

Ahh. OK.  It can theoretically be argued that you could further
squeeze 13 bits out per tree entry because you would need only 5
possible values (100644, 100755 120000, 40000, and 160000, all
octal) for the modes, but we will never know what other modes we
would want to use in the future, so not being over-tight and using
16-bit for this purpose is probably a good trade-off (squeezing 8
bits out per tree entry would make the shape of ident table entry
and tree path entry different and may hurt reusing the code to parse
these tables).

On Tue, 10 Sep 2013, Junio C Hamano wrote:

Nicolas Pitre [off-list ref] writes:

quoted

On Tue, 10 Sep 2013, Junio C Hamano wrote:

quoted

There may be trees in the wild that record 100775 or 100777 in the
mode field for executable blobs, which also need to be special
cased.

All the file mode bits are always preserved.  So this is not really a 
special case as far as the pack v4 encoding is concerned.

Ahh. OK.  It can theoretically be argued that you could further
squeeze 13 bits out per tree entry because you would need only 5
possible values (100644, 100755 120000, 40000, and 160000, all
octal) for the modes, but we will never know what other modes we
would want to use in the future, so not being over-tight and using
16-bit for this purpose is probably a good trade-off

Absolutely.  I tried not to lose any of the currently available 
extension possibilities in the canonical object format.

(squeezing 8 bits out per tree entry would make the shape of ident 
table entry and tree path entry different and may hurt reusing the 
code to parse these tables).

One could argue that 16 bits is much more than sufficient to encode a 
time zone offset too.  but again this didn't seem worth painting 
ourselves in a corner if ever some creative time zones are used.

Those table are also compressed.  So any repetition of the same mode bit 
pattern or sparseness in the tz bits is likely to be compressed well.


Nicolas