Re: Slow "git rev-list origin/master --not --all" or "git fetch" slow when downloading nothing

On Wed, Nov 5, 2008 at 6:37 PM, Linus Torvalds
[off-list ref] wrote:

On Wed, 5 Nov 2008, Santi Béjar wrote:

quoted

  In cold cache "git rev-list origin/master --not --all" is slow
reading many files:

Sorry, s/files/data/

Hmm. It sounds like you possibly don't have packed refs.

They are packed up to v2.6.27.

Have you done "git gc" on that thing lately? What does "strace" say?

               Linus

It is a recently cloned linux-2.6 repo, with 10 or so packs and 70
loose objects. It spends a good fraction in:

brk(0x8318000)                          = 0x8318000

and

mmap2(NULL, 33554432, PROT_READ, MAP_PRIVATE, 4, 0x2000) = 0xb2227000

(strace attatched).

I should have given more info:

It is an old computer (Pentium 4 2.5 GHz)
and the repo is on an external USB drive.

On Wed, Nov 5, 2008 at 6:32 PM, Junio C Hamano [off-list ref] wrote:

"Santi Béjar" [off-list ref] writes:

quoted

  In cold cache "git rev-list origin/master --not --all" is slow
reading many files:

cold cache:
$ /usr/bin/time git rev-list origin/master --not --all
0.03user 0.02system 0:04.57elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
77848inputs+0outputs (410major+1798minor)pagefaults 0swaps

hot cache:
$ /usr/bin/time git rev-list origin/master --not --all
0.01user 0.00system 0:00.06elapsed 31%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2207minor)pagefaults 0swaps

I think that, in this particular case (when the arguments are the tips
of some of the branches), this should not read that many files.

What kind of "many files" are you making git read?  Do you have too many
unpacked refs?  Too many loose objects?

See above.

quoted

... When nothing has changed in the remote repository (so
refs/<remote>/* has all the remote refs) the "git fetch" could be almost
instantaneous (even in coldcache),...

You at least need to read:

 - what "--all" refs point at; to find this out, you need to read all
  unpacked refs files, and one packed-refs file;

 - commit objects that these refs point at; to cull refs that do not point
  at committish and dereference tag objects that point at commit, you
  need to read these objects (either loose objects or in packs);

 - commit objects on the ancestry graph starting from the commit pointed
  at by origin/master and the commits from "--all" refs, until your
  traversal from origin/master hit one of the ancestors of "--all" refs.

Yes, in the general case it is, but in this case we can bypass the
checking of the --all refs after checking if all the given refs are
equal to some of the --all refs.

For me the main thing is the slow "git fetch" when downloading
nothing. I think it was/is faster without the quickfetch path (at
least in the coldcache case).
Maybe we can just check if the new fetched refs are already the tips
of the corresponding remote branch.

Santi