Re: [PATCH v3 6/8] maintenance: add incremental-repack task

On 9/22/2020 7:26 PM, Jonathan Tan wrote:

quoted

+incremental-repack::
+	The `incremental-repack` job repacks the object directory
+	using the `multi-pack-index` feature. In order to prevent race
+	conditions with concurrent Git commands, it follows a two-step
+	process.

[snip]

quoted

First, it deletes any pack-files included in the
+	`multi-pack-index` where none of the objects in the
+	`multi-pack-index` reference those pack-files; this only happens
+	if all objects in the pack-file are also stored in a newer
+	pack-file. Second, it selects a group of pack-files whose "expected
+	size" is below the batch size until the group has total expected
+	size at least the batch size; see the `--batch-size` option for
+	the `repack` subcommand in linkgit:git-multi-pack-index[1]. The
+	default batch-size is zero, which is a special case that attempts
+	to repack all pack-files into a single pack-file.

This lacks the detail of what happens to the selected group of packfiles
(in the second step) - in particular, that a new packfile is created and
the MIDX is rewritten so that all references to the selected group are
updated to refer to the new packfile instead, thus making it possible to
delete the selected group of packfiles in a subsequent first step. All
this is explained in the documentation of git-multi-pack-index (expire
and repack), though, so it might be better to refer to that. E.g.

Here is my attempt to incorporate your recommendations into this doc:

incremental-repack::
	The `incremental-repack` job repacks the object directory
	using the `multi-pack-index` feature. In order to prevent race
	conditions with concurrent Git commands, it follows a two-step
	process. First, it calls `git multi-pack-index expire` to delete
	pack-files unreferenced by the `multi-pack-index` file. Second, it
	calls `git multi-pack-index repack` to select several small
	pack-files and repack them into a bigger one, and then update the
	`multi-pack-index` entries that refer to the small pack-files to
	refer to the new pack-file. This prepares those small pack-files
	for deletion upon the next run of `git multi-pack-index expire`.
	The selection of the small pack-files is such that the expected
	size of the big pack-file is at least the batch size; see the
	`--batch-size` option for the `repack` subcommand in
	linkgit:git-multi-pack-index[1]. The default batch-size is zero,
	which is a special case that attempts to repack all pack-files
	into a single pack-file.

Do we need get_midx_filename() to be global?

No, that does not appear to be important (to this patch). It _was_
necessary when doing the "verify, then delete if problematic" mode.
Thanks for catching it now that it is not necessary.

Thanks,
-Stolee