Re: [RFC PATCH] push: perform negotiation before sending packfile

Junio C Hamano [off-list ref] writes:

quoted

Any comments are welcome, especially if you have ideas about the overall
design or implementation, and/or if you've thought about similar things
before.

I think that it is a good way to prototype by adding a separate
session with the upload-pack to only learn about what is common
before talking to the receive-pack, presumably running on the same
repository and may share common view with the previous upload-pack.

And the code seems to implement the idea just fine.  I wonder how
much we can cut out the unnecessary resource usage in pathological
cases with this approach.

 - If the target ref got rewound, with the current send-pack
   approach, we may not find any common commit, or we may end up
   using a mediocre one---that would result in unnecessary packdata
   transfer, which we may be able to cut down significantly.

   I thought you guys have some server-side hacks to send a bit
   stale commit as a "fake" ref to help with this situation, though.
   So the comparison is really between how much each of these two
   approaches helps.

 - With this approach, we can cut down the initial ref advertisement
   from the receiving end to the minimum.  The protocol with
   negotiation could work correctly without receiving end
   advertising no refs, but I suspect that it would be very common
   in a publishing repository (not a shared everybody-pushes-into
   repository) that the tip of the target repository is known by the
   pusher, and it may help such a case to know where the ref being
   updated with this push originally points at.  But even in such a
   case, additional negotiation may help---the target branch may
   have not advanced, but may of the side branches the pusher have
   merged into the commit being pushed to the target branch may
   already be known by the receiver.

For a real implementation, I think we'd want to do the negotiation
inside the conversation between send-pack and receive-pack, so that
what is agreed to be common between two parties will not shift in
the middle (in the same spirit that upload-pack grabs all the
relevant refs first, advertises them, negotiates what is common and
creates a pack, all using the same worldview of where the tips of
refs are throughout the process, even if some refs change in the
meantime).

Thanks for a fun read.

For a real implementation, I think we'd want to do the negotiation
inside the conversation between send-pack and receive-pack, so that
what is agreed to be common between two parties will not shift in
the middle (in the same spirit that upload-pack grabs all the
relevant refs first, advertises them, negotiates what is common and
creates a pack, all using the same worldview of where the tips of
refs are throughout the process, even if some refs change in the
meantime).

Upload-pack does that for protocol v0 ssh:// and git:// but not
http(s)://, and does not do that for protocol v2, I believe.

If we were to do that, I don't think it would work for the transports
that have are stateless (e.g. HTTP). Also, this seems like it would
involve a significant reworking of how the server serves (receive-pack
would need to know to communicate just like upload-pack does temporarily
before proceeding with its usual behavior, and the client would need to
learn this new way - as opposed to the idea in this patch to do it
separately and reuse the already existing fetch and push mechanisms). I
think the greater complexity is not worth it for something that won't
work in HTTP, but I'm open to other opinions.