Re: Is anyone working on a next-gen Git protocol?

On 10/07/2012 09:57 PM, Ævar Arnfjörð Bjarmason wrote:

On Wed, Oct 3, 2012 at 9:13 PM, Junio C Hamano [off-list ref] wrote:

quoted

Ævar Arnfjörð Bjarmason [off-list ref] writes:

quoted

I'm creating a system where a lot of remotes constantly fetch from a
central repository for deployment purposes, but I've noticed that even
with a remote.$name.fetch configuration to only get certain refs a
"git fetch" will still call git-upload pack which will provide a list
of all references.

It has been observed that the sender has to advertise megabytes of
refs because it has to speak first before knowing what the receiver
wants, even when the receiver is interested in getting updates from
only one of them, or worse yet, when the receiver is only trying to
peek the ref it is interested has been updated.

Has anyone started working on a next-gen Git protocol as a result of
this discussion? If not I thought I'd give it a shot if/when I have
time.

The current protocol is basically (S = Server, C = Client)

  S: Spew out first ref
  S: Advertisement of capabilities
  S: Dump of all our refs
  C/S: Declare wanted refs, negotiate with server
  S: Send pack to client, if needed

And I thought I'd basically turn it into:

  C: Connect to server, declare what protocol we understand
  C: Advertisement of capabilities
  S: Advertisement of capabilities
  C/S: Negotiate what we want
  C/S: Same as v1, without the advertisement of capabilities, and maybe
don't dump refs at all

Basically future-proofing it by having the client say what it supports
to begin with along with what it can handle (like in HTTP).

Then in the negotiation phase the client & server would go back &
forth about what they want & how they want it. I'd planned to
implement something like:

     C: want_refs refs/heads/*
     S: OK to that
     C: want_refs refs/tags/*
     S: OK to that

Or:

     C: want_refs refs/heads/master
     S: OK to that
     C: want_refs refs/tags/v*
     S: OK to that

You'll want that to be a single "wants" message to avoid incurring
insane amounts of roundtrip latency with lots of refs. github and
other hosted services are quite popular, but with my 120ms ping
rtt I'd be spending half a minute just telling the other side what
I want when I fetch from a repo with 250 refs.

It's a flagday and a half to change the protocol though, so I expect
it'll have to wait for 2.0, unless the current client-side part of
it is dumb and ignores existing refs when requesting its "wants", in
which case the server can just stop advertising existing refs and
most of the speedup is already done.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.