Re: [External Mail]Re: why git is so slow for a tiny git push?

On Tue, Oct 12 2021, Jeff King wrote:

On Tue, Oct 12, 2021 at 08:04:44AM +0000, 程洋 wrote:

quoted

I have bitmap indeed because my master server also serves as download server.
However I'm using git 2.17.0, and I didn't set repack.writeBitmaps

On that version and without the config, then perhaps you (or somebody)
passed "-b" to git-repack.

quoted

But why bitmaps can cause push to be slow? Do you mean that if
writeBitmaps is true, every push will regenerate bitmap file? If
that's what you mean, what I see is the only bitmap file in my repo
didn't change across time (the modify time is one month ago, long
before I run the experiment)

No, it is not regenerating the on-disk bitmaps. But when deciding the
set of objects to send, pack-objects will generate an internal bitmap
which is the set difference of objects reachable from the pushed refs,
minus objects reachable from the refs the other the other side told us
they had.

It uses the on-disk bitmaps as much as possible, but there may be
commits not covered by bitmaps (either because they were pushed since
the last repack which built bitmaps, or simply because it's too
expensive to put a bitmap on every commit, so we sprinkle them
throughout the commit history). In those cases we have to traverse parts
of the object graph by walking commits and opening up trees. This can be
expensive, and is where your time is going.

Reachability bitmaps _usually_ make things faster, but they have some
cases where they make things worse (especially if you have a ton of
refs, or haven't repacked recently).

If bitmaps are causing a problem for your push, they are likely to be
causing problems for fetches, too. But if you want to keep them to serve
fetches, but not use them for push, you should be able to do:

  git -c pack.usebitmaps=false push

For the last on-list discussion to (probably the same) problem, which in
turn references an even earlier one:
https://lore.kernel.org/git/878s6nfq54.fsf@evledraar.gmail.com/ (local)

I don't remember if my own report from mid-2019 said so or contradicts
this (and I didn't re-read the thread), but FWIW I *vaguely* recall that
the case I ran into *might* have had to do with a user running into this
on a shared "staging" server.

I.e. one where users logged in as their own user, cd'd to a shared git
repo they got a lock on, and ran fetch/push/deploy commands. One user
had a pack.useBitmaps=true or equivalent in their config (or had
manually run such a repack), so there were some very old stale bitmaps
around.

This was also a setup with a gc.bigPackThreshold configured, which now
that I think about it might have made it much worse, i.e. we'd keep that
*.bitmap on the "big pack" around forever.

But more generally with these side-indexes it seems to me that the code
involved might not be considering these sorts of edge cases, i.e. my
understanding from you above is that if we have bitmaps anywhere we'll
try to in-memory use them for all the objects in play? Or that otherwise
having "partial" bitmaps leads to pathological behavior.

tl;dr: We can't assume that a config of "I like to write side-index
A/B/C/... when I repack" means that the repo is in that state *now*, but
it seems that we do.