Re: Making split commit graphs pick up new options (namely --changed-paths)

On Fri, Jun 11, 2021 at 01:47:28PM -0400, Derrick Stolee wrote:

On 6/10/2021 8:50 PM, Taylor Blau wrote:

quoted

On Fri, Jun 11, 2021 at 01:56:31AM +0200, Ævar Arnfjörð Bjarmason wrote:

quoted

So yeah, maybe we can just unlink() them right away, or another way to
handle the race is that load_commit_graph_chain() could just try again
from the beginning in such a case, and presumably picking up the fresh
just-rewritten chain.

I'd probably be in favor of the latter.

I want to point out that on Windows we cannot successfully unlink()
a layer that is currently being read by another Git process. That
will not affect server scenarios (to the best of my knowledge) but
is important to many end users.

Right, but isn't this already a problem today? Since the expiration
window is zero we are already effectively trying to unlink all merged
layers immediately:

  - Marking merged commit-graph layers as expired via
    mark_commit_graphs() by setting their mtime to "now", and then

  - Immediately removing all layers which have mtime older than an
    instant later in expire_commit_graphs().

(I almost suggested that a race already exists between multiple writers
that merge multiple layers of the commit-graph, but that race doesn't
exist because the commit-graph chain is written before other layers are
marked and expired.)

In any case, it seems like the return value from unlink() is
deliberately ignored in case another process is holding an expired layer
open when we try to unlink it. So we'll eventually clean up all layers
that don't belong to the commit-graph-chain, but at the granularity of
new writes.

(FWIW, I had to re-read 8d84097f96 (commit-graph: expire commit-graph
files, 2019-06-18) which mentions that a configuration variable would be
introduced to change the expiration window, but we don't have any such
configuration option. It also doesn't make any mention of handling this
problem on Windows, which made me think that the unlink() calls weren't
checking their return values by accident when in fact it was probably on
purpose.)

Thanks,
Taylor