Re: Synchronous replication on push

Date: Mon, 4 Nov 2024 09:40:06 -0500
From: Konstantin Ryabitsev <redacted>

Alternatively, you can take a look at grokmirror, which is what kernel.org
uses:
https://pypi.org/project/grokmirror/

It's pull-based instead of push-based, for several reasons:

1. We replicate to multiple worldwide frontends, and we expect that some of
   them may be unreachable at the time when we attempt a push
2. This allows us to propagate repository deletes
3. This allows us to propagate details like descriptions and authors

Grokmirror also has a listener daemon that can trigger a pull, so it's
possible to have near-instantaneous replication by notifying the remote node
that a repository has been updated and should be pulled.

Thanks, that looks useful, but it's not quite what I'm looking for.

Part of the goal is essentially the same (qualitative) types of
service guarantee that Github advertises:[*] once the user's `git
push' command has succeeded with nonzero exit status, the objects and
ref updates have been written to multiple backing stores so it would
take a failure of a quorum of those backing stores to lose the data.

In particular, a backend may reject an update, and when this happens
(in the multi-backend case, by enough backends that no quorum is
reached), the user who ran `git push' needs to know that it failed so
they don't, e.g., delete their branch, run git gc, and go on their
merry way having silently lost data.


[*] https://github.blog/engineering/infrastructure/stretching-spokes/