On Thu, Jul 09, 2026 at 04:43:58PM -0500, Justin Tobler wrote:

On 26/07/09 10:35AM, Patrick Steinhardt wrote:

quoted

The function `for_each_bitmapped_object()` can be used to iterate
through all objects covered by a bitmap. The benefit of this function is
that it allows the caller to efficiently handle some object filters. For
example, this can be used to filter out objects of a specific type with
some simple bitmap operations. But callers are currently required to
manually wire up the use of bitmaps though, and to do so they have to
reach into internals of a given object database source.

Introduce a new `struct odb_for_each_object_options::filter` field so
that the interface becomes generic. When set, then a backend may
optionally use the filter to skip some objects that it would have
otherwise yielded.

Note that the respective backends are free to ignore this field if they
cannot meaningfully optimize for a given filter, and consequently
callers need to verify whether they actually want the returned objects.
While annoying, we cannot easily lift this restriction anyway as the
object filter infrastructure supports some filters that cannot be
answered by the object database alone.

Huh, this feels rather awkward. So callers will always still have to
ensure correctness by filtering the result a second time? IIUC, the idea
is that the backend may be able to more efficiently process object
filtering so we would want it to attempt the first pass.

Is there a subset of object filters that we should expect any backend to
be able to answer? If so, maybe we should define a separate list of
object filter options specific to this interface? Any filtering not
supported would have to be deligated to the caller then.

It's a bit awkward, but it's also similar to how we handle this for
example in the reference backends with the exclude patterns. I don't
really think it makes sense to enforce that backends may only handle a
subset of object filters that we know the current backends support, as
that would artificially limit us.

For example, the "loose" backend already cannot efficiently handle many
of the filters that the "packed" backend can handle, like for example
filtering by type. So ultimately, the subset of filters that can be
handled efficiently by both backends is empty. And as the "files"
backend always combines both of these backends we wouldn't be able to
ever use the object filter at all there.

The same could be true for any future backend: we cannot assume how they
store their objects, so they might be able to efficiently handle filters
that the current backends cannot.

An alternative going forward could be to perform filtering of yielded
objects inside `odb_for_each_object()` itself so that it will filter out
any objects that the backends themselves couldn't filter efficiently.
But I'm not sure I want to go there as part of this series -- we only
have a single caller anyway that iterates with a filter, and that caller
already knows to manually filter references.

I'll add a bit of an explanation to the commit message.

Patrick