Re: Looking for help to understand external filter driver code

On 19 Jul 2016, at 23:33, Junio C Hamano [off-list ref] wrote:

Lars Schneider [off-list ref] writes:

quoted

Git writes --> 4 byte filename length
Git writes --> filename string

Why limit to 32GB?  Perhaps NUL termination is more appropriate
here?

OK, I will use NUL termination for the filename. 
You're also right about the limit - I will use 8 byte to encode the 
content length.

quoted

Git writes --> 4 byte content length
Git writes --> content string
Git reads <-- 4 byte filtered content length
Git reads <-- filtered content

Do you really need to force the sender to know the length in
advance?  Together with the sequential nature of the above exchange,
i.e. the filter is forbidden from producing even a single byte of
its output before reading everything Git feeds it, you are making it
impossible to use filters that perform streaming conversion.

That is correct. However, for my particular use case streaming
conversion wouldn't be useful anyways:
https://github.com/github/git-lfs/pull/1382

Of course, with the "sequential" thing, you do not have to worry
about deadlocking hence no need for poll/select, but I am not sure
that is a good thing.

Thanks for the confirmation. I consider to exchange a "filter protocol 
version" right after the filter process has started. That way someone 
could add a more evolved "filter driver protocol" later on that supports 
streaming and the external filter could pick whatever protocol is most
appropriate (and supported). Could that be an acceptable compromise
to get a serious review of the "sequential" thing?

Thanks,
Lars