Re: [PATCH v2 0/5] Git filter protocol

On 27 Jul 2016, at 21:08, Jakub Narębski [off-list ref] wrote:

W dniu 2016-07-27 o 02:06, larsxschneider@gmail.com pisze:

quoted

From: Lars Schneider <redacted>

Hi,

thanks a lot for the extensive reviews. I tried to address all mentioned
concerns and summarized them below. The most prominent changes since v1 are
the following:
* Git offers a number of filter capabilities that a filter can request
 (right now only "smudge" and "clean" - in the future maybe "cleanFromFile",
 "smudgeToFile", and/or "stream")
* pipe communication uses a packet format (pkt-line) based protocol

I wonder if it would make sense to support both whole-file pipe communication,
and packet format (pkt-line) pipe communication.

The problem with whole-file pipe communication (original proposal for
new filter protocol is that it needs file size upfront.  For some types
of filters it is not a problem:
- if a filtered file has the same size as original, like for rot13
  example in the test for the feature
- if you can calculate the resulting file size from original size,
  like for most if not all encryption formats (that includes GPG,
  uudecode, base64, quoted-printable, hex, etc.); same for decryption,
  and from converting between fixed-width encodings
- if resulting file size is saved somewhere that is easy to get, like
  for LFS solutions (I think).

For other filters it is serious problem.  For example indent, keyword
expansion, rezipping with zero compression (well, maybe not this one,
but at least the reverse of it), converting between encodings where
at least one is variable width (like UTF-8),...

IMHO writing whole-file persistent filters is easier than using pkt-line.
On the other hand using pkt-line allow for more detailed progress report.

I initially wanted to support only "while-file" pipe, too.

But Peff ($gmane/299902), Duy, and Eric, seemed to prefer the pkt-line
solution (gmane is down - otherwise I would have given you the links).

After I have looked at it I think the pkt-line solution is indeed nicer
for the following reasons:

(1) A stream optimized version (read/write in separate threads) of the filter
    protocol can be implemented in the future without changing the protocol
(2) pkt-line is a simple and easy to implement format
(3) Reuse of existing Git communication infrastructure
    -> code and documentation are less surprising to people that know Git
    -> you can use GIT_TRACE_PACKET to easily inspect the
       communication between Git and the filter process
(4) The overheads is neglect able (4 byte header vs 65516 byte content)

quoted

* a long running filter application is defined with "filter.<driver>.process"

I hope that won't confuse Git users into trying to use single-shot
filters with a new protocol...

Yes, that was my intention for this new config.

Thanks,
Lars