Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

Jeff King [off-list ref] writes:

... One
alternative would be to amend the bundle format so that rather than a
single file, you get a bundle header whose end says "...and my matching
packfile is 1234-abcd". And then the client knows that they can fetch
that separately from the same source.

I would imagine that we would introduce bundle v3 format for this.

It may want to say "my matching packfiles are these" to accomodate a
set of packs split at max-pack-size, but I am perfectly fine to say
you must create a single pack when you use a bundle with separate
header to keep things simpler.

It's an extra HTTP request, but it makes the code for client _and_
server way simpler. So the whole thing is basically then:

  0. During gc, server generates pack-1234abcd.pack. It writes matching
     tips into pack-1234abcd.info, which is essentially a bundle file
     whose final line says "pack-1234abcd.pack".

OK.

  1. Client contacts server via any git protocol. Server says
     "resumable=<url>". Let's says that <url> is
     https://example.com/repo/clones/1234abcd.bundle.

  2. Client goes to <url>. They see that they are fetching a bundle,
     and know not to do the usual smart-http or dumb-http protocols.
     They can fetch the bundle header resumably (though it's tiny, so it
     doesn't really matter).

Might be in megabytes range, though, with many refs.  It still is
tiny, though ;-).

  3. After finishing the bundle header, they see they need to grab the
     packfile. Based on the bundle header's URL and the filename
     contained within it, they know to get
     https://example.com/repo/clones/pack-1234abcd.pack". This is
     resumable, too.

OK.

  4. Client clones from bundled pack as normal; no root-finding magic
     required.

I like this part the most.

  5. Client runs incremental fetch against original repo from step 1.

And you'll notice, too, that all of the bundle-http magic kicks in
during step 2 because the client sees they're grabbing a bundle. Which
means that the <url> in step 1 doesn't _have_ to be a bundle. It can be
"go fetch from kernel.org, then come back to me".

Or it could be a packfile (and the client discovers roots), as you
mentioned in a separate message.  I personally do not think it buys
us much, as long as we do a bundle represented as a header and a
separate pack.