Re: resumable git-clone?

Johannes Schindelin [off-list ref] writes:

Now here is a thought: after an interrupted fetch, you could do a 
(possibly expensive) analysis what commits are dangling, but do not 
contain broken links in their _complete_ history.  Then mark them as 
(temporary) refs.

That's all too elaborate and sounds fairly expensive.

I think another practical solution is to reduce the need to
resume git-clone, and I foresee a nice side effect.

Let's talk a bit about the side effect first.  Often I see
people wondering...

 - I am forking off of somebody's big project;

 - I want to publish the result of _my_ part of the work;

 - I cannot afford the diskspace nor the bandwidth at the
   hosting service to offer the whole project, but my own work
   on top of the upstream is small enough;

So I've been thinking about teaching the server side, when
responding to a fetch request, to be able to say something like
"Sorry but this repository is for fetching by people who have at
least these commits from that other project I am a fork of."
We make that "at least these from that other project" part
machine readable, and git-fetch request git-clone makes can
redirect itself to fill the prerequisites before attempting to
go there again.

And one way of doing that "at least these commits" part could be
a pre-built, well known bundle.  The bulk of the older history
of a repository insanely large for your bandwidth requirements
could be made to a single bundle [*1*], and distributed from
other places like mirrored HTTP servers, or even from torrent
network.  The pre-built bundle does not have to be built too
often.  Probably after every other big release would do.

Once we have such a support, cloning from a regular full
repository ove an unreliable connection could fall back on
getting a bundle (a regular repository does not have to refuse
to serve downloaders that do not have the prerequisites, but
it could still suggest that as an alternate method to fall back
on.


[Footnote]

*1* Sheesh.  While I was writing this I noticed that the current
    'git bundle' is suitable only for incremental sneakernet but
    not for this particular use case.

    $ git bundle create 2612.bndl v2.6.12

    does not work, although

    $ git bundle create 2612-2613.bndl v2.6.12..v2.6.13

    does.  We need to fix this.

Hi,

On Wed, 8 Aug 2007, Junio C Hamano wrote:

So I've been thinking about teaching the server side, when responding to 
a fetch request, to be able to say something like "Sorry but this 
repository is for fetching by people who have at least these commits 
from that other project I am a fork of." We make that "at least these 
from that other project" part machine readable, and git-fetch request 
git-clone makes can redirect itself to fill the prerequisites before 
attempting to go there again.

Something like the alternates that http transport can use?

*1* Sheesh.  While I was writing this I noticed that the current
    'git bundle' is suitable only for incremental sneakernet but
    not for this particular use case.

    $ git bundle create 2612.bndl v2.6.12

    does not work, although

    $ git bundle create 2612-2613.bndl v2.6.12..v2.6.13

    does.  We need to fix this.

Yes, I noticed this a long time ago.  Time is lacking...

Ciao,
Dscho