Re: [PATCH v2] fetch-pack: don't try to fetch peel values with --all

On Wed, Jun 13, 2018 at 05:13:02PM -0400, Jeff King wrote:

On Tue, Jun 12, 2018 at 06:54:17PM +0000, Kirill Smelkov wrote:

quoted

quoted

If an extra connection isn't a problem, you might be better off with
"git ls-remote", and then picking through the results for refs of
interest, and then "git fetch-pack" to actually get the pack. That's how
git-fetch worked when it was a shell script (e.g., see c3a200120d, the
last shell version).

Yes, this is what I ended up doing:

https://lab.nexedi.com/kirr/git-backup/commit/899103bf

but for another reason - to avoid repeating for every fetched repository
slow (in case of my "big" destination backup repository) quickfetch()
checking in every spawned `git fetch`: git-backup can build index of
objects we already have ourselves only once at startup, and then in
fetch, after checking lsremote output, consult that index, and if we see
we already have everything for an advertised reference - just avoid
giving it to fetch-pack to process. It turns out for many pulled
repositories there is usually no references changed at all and this way
fetch-pack can be skipped completely:

https://lab.nexedi.com/kirr/git-backup/commit/3efed898

Thanks for sharing that, it's an interesting case. I'd hope that
git-fetch is smart enough not to even bother with quickfetch() if there
are no refs to update. But if we have even one change to fetch, then
yeah, in the general case it makes sense to me that you could do better
by amortizing the scan of local objects across many operations.

Thanks for feedback. For the reference in case of git-backup `git fetch`
or `git fetch-pack` would have to always do quickfetch scan or equivalent,
because in case of backup repo there is only one reference in it - its
master - and references of backed up repositories do not have anything
representing them in backup.git/refs/ .

Kirill