Re: More on git over HTTP POST

"Shawn O. Pearce" [off-list ref] writes:

Show Refs
---------

Obtains the available refs from the remote repository.  The response
is a sequence of git "packet lines", one per ref, and a final flush
packet line to indicate the end of stream.

As the initial protocol exchange request, I suspect that you would regret
if you do not leave room for some "capability advertisement" in this
exchange.

With the git native protocol, we luckily found space to do so after the
ref payload (because pkt-line is "length + payload" format but the code
that reads payload happened to ignore anything after NUL).  You would want
to define how these are given by the server to the client over HTTP
channel.  For example, putting them on extra HTTP headers is probably Ok.

Junio C Hamano [off-list ref] wrote:

"Shawn O. Pearce" [off-list ref] writes:

quoted

Show Refs
---------

Obtains the available refs from the remote repository.  The response
is a sequence of git "packet lines", one per ref, and a final flush
packet line to indicate the end of stream.

As the initial protocol exchange request, I suspect that you would regret
if you do not leave room for some "capability advertisement" in this
exchange.

With the git native protocol, we luckily found space to do so after the
ref payload (because pkt-line is "length + payload" format but the code
that reads payload happened to ignore anything after NUL).  You would want
to define how these are given by the server to the client over HTTP
channel.  For example, putting them on extra HTTP headers is probably Ok.

Yea, I thought that the HTTP headers would be more than enough
space to add capability advertisements.  Most client libraries
will happily parse and store these for the application, and won't
make a fuss if the application doesn't read them.

Hence there's more than enough room in the protocol to extend it
in the future with additional capabilities.

We do have to be careful though.  Any cachable resource must only
rely upon the URI and the standard headers which compute into the
cache key for a request.  There aren't many, though I think the
Content-Type header may be among them.

-- 
Shawn.

Junio C Hamano wrote:

With the git native protocol, we luckily found space to do so after the
ref payload (because pkt-line is "length + payload" format but the code
that reads payload happened to ignore anything after NUL).  You would want
to define how these are given by the server to the client over HTTP
channel.  For example, putting them on extra HTTP headers is probably Ok.

I think that would be a mistake, just because it's one more thing for 
proxies to screw up on.  It's better to have negotiation information in 
the payload, before the "real" data.

Obviously one thing that needs to be included in each transaction is a 
transaction ID that will be reported back on the next transaction, since 
you can't rely on a persistent connection.

	-hpa

"H. Peter Anvin" [off-list ref] wrote:

Junio C Hamano wrote:

quoted

 For example, putting them [capabilities] on extra HTTP headers is probably Ok.

I think that would be a mistake, just because it's one more thing for  
proxies to screw up on.

I didn't realize we were in an era of proxies that are that
brain-damaged that they cannot relay the other headers.  The Amazon
S3 service relies heavily upon their own extended headers to make
their REST API work.  If proxies stripped that stuff out then the
client wouldn't work at all.

IOW I had thought we were past this dark age of the Internet.

It's better to have negotiation information in  
the payload, before the "real" data.

I guess I could do that.  At least for the really complex stuff.

Obviously one thing that needs to be included in each transaction is a  
transaction ID that will be reported back on the next transaction, since  
you can't rely on a persistent connection.

No.  That requires the server to maintain state.  We don't want to
do that if we can avoid it.  I would much rather have the clients
handle the state management as it simplifies the server side,
especially when you start talking about reverse proxies and/or
load-balancers running in front of the server farm.

-- 
Shawn.

On Sat, 2 Aug 2008, Shawn O. Pearce wrote:

"H. Peter Anvin" [off-list ref] wrote:

quoted

Junio C Hamano wrote:

quoted

 For example, putting them [capabilities] on extra HTTP headers is probably Ok.

I think that would be a mistake, just because it's one more thing for
proxies to screw up on.

I didn't realize we were in an era of proxies that are that
brain-damaged that they cannot relay the other headers.  The Amazon
S3 service relies heavily upon their own extended headers to make
their REST API work.  If proxies stripped that stuff out then the
client wouldn't work at all.

IOW I had thought we were past this dark age of the Internet.

actually, it's not just a matter of not getting 'past this dark age of the 
Internet', it's an issue that so many people are tunneling _everyting_ 
over http (including the bad guys tunneling malware) that proxies are 
getting more aggressive then they have ever been before in pulling apart 
the payload and analysing it before letting it get through to the far 
side.

David Lang

quoted

It's better to have negotiation information in
the payload, before the "real" data.

I guess I could do that.  At least for the really complex stuff.

quoted

Obviously one thing that needs to be included in each transaction is a
transaction ID that will be reported back on the next transaction, since
you can't rely on a persistent connection.

No.  That requires the server to maintain state.  We don't want to
do that if we can avoid it.  I would much rather have the clients
handle the state management as it simplifies the server side,
especially when you start talking about reverse proxies and/or
load-balancers running in front of the server farm.

Shawn O. Pearce wrote:

IOW I had thought we were past this dark age of the Internet.

If we were, there wouldn't be a need for this project at all.  The whole 
purpose of it is to deal with corporate proxies that try to prevent 
actual communication because of "security", and it's really hard to 
predict what utterly arbitrary heuristics they have applied.

	-hpa

david@lang.hm wrote:

actually, it's not just a matter of not getting 'past this dark age of 
the Internet', it's an issue that so many people are tunneling 
_everyting_ over http (including the bad guys tunneling malware) that 
proxies are getting more aggressive then they have ever been before in 
pulling apart the payload and analysing it before letting it get through 
to the far side.

... which is of course because of said proxies that this is happening, too.

There are too many idiots out there building "security software" and 
running IT departments, that's really the bottom line.

By the way, I want to say *thank you* to Shawn for tackling this 
project: this has been a major issue for kernel.org, and getting 
something like this deployed would be incredibly helpful.

	-hpa