Re: [PATCH 0/2] midx: prevent bitmap corruption when permuting pack order

On Fri, Dec 10, 2021 at 05:31:27PM -0500, Taylor Blau wrote:

I had originally imagined that storing the preferred pack's identity
alone would be enough to solve this bug. But that isn't quite so,
because we break ties among duplicate objects first by prefered-ness,
then by their pack's mtime. So that could change too, and it would cause
us to break in the same way.

At the bare minimum you need an ordering of all of the packs in the
MIDX (like I had originally imagined here). At most, we could do
something like what is unintentionally written here, which would allow
us to get rid of MIDX .rev files entirely. I think doing the former is
simpler, and I am not sure if there are practical advantages to the
latter.

Thinking on it more, I don't think this "at minimum you would need..."
is quite right either. It would suffice to know the identity of the
preferred pack, and the mtimes of all of the other packs, since that
alone is enough to reconstruct the object order.

That is pretty appealing, too, because knowing the order of packs would
require some major surgery (the order of packs isn't really something
the MIDX code thinks about, it's inferred from the way it sorts
objects).

So here I'd imagine we'd have two chunks:

  - `PREF`: a 4-byte network order integer identifier of the preferred
    pack
  - `MTME`: an array of 4-byte network order integers containing the
    mtimes of each pack in lexical order of the pack checksums.

Unless I'm missing something, I'm pretty sure that these two would be
sufficient to uniquely identify the object order (and, importantly, to
cause the MIDX's checksum to change when the object order does).

And these are vastly easier to write down than the ordering of the packs
themselves, while also being a much more incremental step forward. So I
think I'm more comfortable with this direction than anything discussed
so far.

Thanks,
Taylor