Hi git list,
I am trying to diagnose a strange problem in a VM running as a 'git
over ssh server', with one repo which periodically grows very quickly.
The complete dataset packs to a single pack+index of ~650MB. Growth is
slow, these are ASCII text reports that use a template -- highly
compressible. Reports come from a few dozen machines that log in every
hour.
However, something is happening that explodes the efficient pack into
an ungodly mess.
Do client pushes over git+ssh ever trigger a repack on the server? If
so, these repacking processes are racing with each other and taking
650MB to 7GB at which point we hit ENOSPC, sometimes pom killer joins
the party, etc.
pack dir looks like this, ordered by timestamp:
http://fpaste.org/55730/04636313/
cheers,
m
--
martin.langhoff@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
On Thu, Nov 21, 2013 at 10:21 AM, Martin Langhoff
[off-list ref] wrote:
Do client pushes over git+ssh ever trigger a repack on the server?
man git-config
[snip]
receive.autogc
By default, git-receive-pack will run "git-gc --auto" after
receiving data from git-push and updating refs. You can stop it by
setting this variable to false.
Oooooops!
Ok, couple problems here:
- if it's receiving from many pushers, it races with itself; needs
some lock or back-off mechanism
- alternatively, an splay mechanism. We have a "hard" threshold...
given many "pushers" acting in parallel, they'll all hit the threshold
at the same time. There is no need for this, we could randomize the
threshold by 20%; that would radically reduce the racy-ness
- auto repack in this scenario has a reasonable likelihood if being
visited by the OOM killer -- therefore it needs to fail more
gracefully, for example with tmpfile cleanup. Perhaps by having the
tmpfiles places in a tmpdir named with the pid of the child would make
this easier...
Naturally, I'll move quickly to disable this evil-spawn-automagic
setting and setup a cronjob. But I think it is possible to have
defaults that work more reliably and with lower risk of explosion.
thoughts?
m
--
martin.langhoff@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff