Re: [PATCH v4 00/29] tree-walk: mostly replace "mode" with "enum object_type"

On Sun, Mar 21 2021, Junio C Hamano wrote:

Ævar Arnfjörð Bjarmason [off-list ref] writes:

quoted

Yes, that would be a serious regression. I agree that all these
functions/callbacks etc. should have a way to get at the mode bits.

I'm adding "enum object_type", not removing the "mode" parameter in
read_tree_fn_t. This function (which is in "seen" as 03316f20347
(sparse-index: implement ensure_full_index(), 2021-03-16)) works just
fine in combination with this series.

The other APIs modified here all retain the ability to give you the mode
bit, they (the tree-walk.h changes) just optionally give you the option
of getting just the type (or just the path), and as it turns out most
users of the API can be converted over to that.

I have a vague feeling that such an approach may be still repeating
the same mistake.

If the original premise is that "unsigned mode" bit can be abused to
feed impossible values like 0100653 to the system and the code
should catch it, ...

quoted

The current codebase will allow you to stick arbitrary mode bits in
trees, ...

... then I would understand it if the approach is to introduce a
distinct type that enumerates all possible mode bit values (and
nothing else), and have the compiler validate the callchain, so that
nobody can pass bogus mode bits (and when reading mode bits fields
in existing objects, the one that converts from the series of octal
bytes to that distsinct type would notice and complain).  And we've
already seen from this particular breakages that the "distinct type"
appropriate to be used for that purpose is not "enum object_type".
It is not expressive enough to enumerate all possible mode bit
values (besides, an enum is interchangeable with an int, so there
isn't much protection we would be getting from the compiler---we
could use a small struct of a new type, and have one static const
instance for each possible mode bit combination, I guess, but the
point here is that insisting on using "enum object_type" seems to be
the source of the problem).

Yes, this is a good suggestion, but one which'll be much easier to do
after this series. Since the majority of callers don't care about the
raw mode bits or anything except if the entry is a blob/tree/commit.

I am afraid that it is even worse to pass both object type and
"unsigned mode" together.  It would still leave room for a bug to
pass nonsense mode bits, which we said we wanted to catch in our
original mission statement.  In addition, we now have a new room for
a bug, which is to pass an inconsistent pair of object type and mode
to the callchain.  Somebody would need to say "yuck, we got a mode
100644 but the type says TREE" now, in addition to validating if the
mode is sensible, which we should be doing somehow, no?

Indeed. This goes back to my "bear with me" comment upthread. I'm
planning to fix these cases too, and starting by moving most things to
"enum object_type" makes that a lot easier.

Right now we simply trust the mode bits, but that opens the door to the
same sort of bug I'm fixing in another series (that I need to re-roll)
where we have a "type commit" and "object ID" in a tag, but the ID is
really a tree or whatever.

We then get confused because we trusted the invalid metadata over
checking the type of the object ID we have in the object store.

So this particular series doesn't fix this bug at the end of it, the
"object_type" in tree-walk.c is just a function of deriving it from the
mode bits.

But once I've audited the cases where we're really just acting on the
type and don't have or use the mode (so we know we're not re-writing the
mode somewhere) then we would set the object_type to the actual type, as
in what we'd get from oid_object_info(ID).

Then we can e.g. scream murder about a mode/type mismatch in fsck, but
still be able to list/diff/inspect etc. that object in any codepath that
just wants to e.g. recursively iterate over the tree.

So, I am not sure how these changes are making anything better.

quoted

... I had a
summary of this in v1, but should probably have provided a recap[1].

Oh, absolutely.  When we iterate, we should be welcoming to those
who missed earlier iterations.