Re: [PATCH 07/13] object-filter: common declarations for object filtering

On 9/27/2017 8:05 PM, Jonathan Tan wrote:

On Wed, 27 Sep 2017 13:09:42 -0400
Jeff Hostetler [off-list ref] wrote:

quoted

On 9/26/2017 6:39 PM, Jonathan Tan wrote:

quoted

On Fri, 22 Sep 2017 20:30:11 +0000
Jeff Hostetler [off-list ref] wrote:

quoted

   Makefile        |   1 +
   object-filter.c | 269 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
   object-filter.h | 173 ++++++++++++++++++++++++++++++++++++
   3 files changed, 443 insertions(+)
   create mode 100644 object-filter.c
   create mode 100644 object-filter.h

I think these and list-objects-filter-* are multiple levels of
indirection too many. Would a single file with a few implementations of
filter_object_fn be sufficient?

I did that in my first draft and I found it confusing.

Each filter has 3 parts (some filter-specific data structures,
a filter callback routine, a driver to call the traverse code).
I found it easier to reason about each filter in isolation.
And it makes it easier to work on each independently and keep
their inclusion in separate commits.

I looked at object-filter.{c,h} a bit more. It seems that these files:
  1) define a struct that contains all the options that we would want
  2) supplies a way to populate this struct from code that uses parse-options
  3) supplies a way to populate this struct from code that calculates
     options by hand
  4) supplies a way to populate this struct from "protocol" ("<key>" or
     "<key> <value>" strings)

And the next commit takes the struct that object-filter.{c,h} produces
and actually performs the traversal.

I think this can be significantly simplified, though. Would this work:
  a) Define the object_filter_options struct, but make all fields
     correspond to a single parameter each. Define
     OBJECT_FILTER_OPTIONS_INIT to initialize everything to 0 except for
     large_byte_limit to ULONG_MAX (so that we can detect if something
     else is set to it).
  b) Define one single OPT_PARSE_FILTER macro containing all the
     parameters. We can use the non-callback macros here. That solves 2)
     above.
  c) Define a function that takes in (int *argc, char ***argv) that can
     "massage" it to remove all filter-related arguments, storing them in
     a object_filter_options struct. That solves 3) above. As discussed
     in the API documentation, this means that argument lists of the form
     "--unknown --known" (where "--unknown" takes an argument) are
     processed differently, but then again, rev-list never supported them
     anyway (it required "--unknown=<arg>").
  d) Define a function that converts "<key>" into "--<key>" and "<key>
     <value>" into "--<key>=<value>", and use the existing mechanism.
     That solves 4) above.

This removes the need to maintain the lists of one-per-argument
functions, including the parse_filter_* and opt_parse_filter_* functions
declared in the header file. If we were to add a feature, we wouldn't
need to change anything in the caller, and wouldn't need to hand-edit
object_filter_hand_parse_arg() and object_filter_hand_parse_protocol().

Maybe.  What I have here now is the result of adding these arguments to
rev-list and pack-objects (in the current patch series), and also to
fetch-pack, fetch, clone, upload-pack, index-pack, and the transport and
protocol code (in a follow-on patch series that I've omitted for the moment).
And there will probably be a few more, such as fsck, gc, and etc.  I hesitate
to refine the macros too much further until we've agreement on the overall
approach and terms.

Thanks
Jeff