Re: [PATCH 09/19] documentation: add documentation for the bitmap format

On Fri, Oct 25, 2013 at 3:21 AM, Jeff King [off-list ref] wrote:

[+cc spearce; sorry, I really should have cc'd the whole series to you
     in the first place]

On Fri, Oct 25, 2013 at 08:16:18AM +0700, Nguyen Thai Ngoc Duy wrote:

quoted

quoted

+       - A header appears at the beginning:
+
+               4-byte signature: {'B', 'I', 'T', 'M'}
+
+               2-byte version number (network byte order)
+                       The current implementation only supports version 1
+                       of the bitmap index (the same one as JGit).

I suppose this means if I want to extend pack bitmaps to be used on
shallow clones, I need to step version to 2 before adding the shallow
points in .bitmap file as there's no chance of modifying v1 anymore,
correct?

It depends on how you want to change it. And what Shawn says. :)

Correct, I think we need to bump to version 2 for any changes.

This is an attempt to document the JGit format. My feeling is that there
should be a version bump if an existing implementation would barf on it,
and I assume Shawn would agree.  But if you want to simply add extra
data that would be ignored by an existing implementation, it would be OK
to add the data and mark it with a flag:

quoted

quoted

+               2-byte flags (network byte order)
+
+                       The following flags are supported:

[...]

That's how we added the name-hash cache in the final patch.

That being said, JGit is _not_ happy with that, and complains when any
flags besides OPT_FULL are used.  Whether that is because I am
misinterpreting the intent of the flags field, or because JGit is being
overly strict is up for debate.

Flags was created by Colby, I don't think he intended to use it for
determining if an optional section exists in the file or not. There is
insufficient information using only 1 bit to skip over an optional
section. The dircache file uses a length field with each extension
name to allow a reader to skip over sections it does not recognize
(and whose name is lowercase).

I think Colby and I talked about having additional optional sections
in this file, but Colby didn't want to overcomplicate the format early
on. So v1 is probably not very extensible and we may have to go to v2
to safely create an extension with the name hash cache used in this
series.

Given that the JGit v1 bitmap format has been shipping since JGit 3.0
and in Gerrit Code Review 2.6, its in use in the wild. So we aren't
going to go back and redefine v1.