While Git allows users to select different reference backends, unlike
with objects, there is no flexibility in selecting the reference
directory. Currently, the reference format is obtained from the config
of the repository and the reference directory is set to the $GIT_DIR.

This patch series adds a new ENV variable 'GIT_REF_URI' which takes the
reference backend and path in a URI form:

    <reference_backend>://<URI-for-resource>

For e.g. 'reftable:///foo' or 'files://$GIT_DIR/ref_migration.0xBsa0'.

One use case for this is migration between different backends. On the
server side, migrating from the files backend to the newly introduced
reftable backend can be achieved by running 'git refs migrate'. However,
for large repositories with millions of references, this migration can
take from seconds to minutes.

For some background, at GitLab, the criteria for our migration was to
reduce the downtime of the migrate ideally to zero. So running 'git refs
migrate --ref-format=reftable' by itself wouldn't work, since it scales
with the number of references and we have repos with millions of
references, so we need to migrate without loosing any information. We
came up with the following plan:

  1. Run git-pack-refs(1) and note timestamp of the generated packed-refs
     file.
  2. Run git refs migrate –dry-run.
  3. If there are no ongoing reference requests (read/write)
     a. Lock the repository by blocking incoming requests (done on a
        layer above git, in Gitaly [1]).
     b. If the timestamp of the packed-refs file has changed, unlock
        the repo and repeat from step 1.
     c. Apply all the loose refs to the dry-run reftable folder (this
        requires support in Git to write refs to arbitrary folder).
     d. Move the reftable dry-run folder into the GIT_DIR.
     e. Swap the repo config
     f. Unlock repo access

Using such a route, scales much better since we only have to worry about
blocking the repository by O(ref written between #1 and #3a) and not
O(refs in repo). But for doing so, we need to be able to write to a
arbitrary reference backend + path. This is to add the missing
references to the dry-run reftable folder. This series, achieves that.

The first commit adds the required changes to create a 'ref_store' for a
given path. The second commit parses the URI if available when creating
the main ref store.

This is based on top of 9a2fb147f2 (Git 2.52, 2025-11-17).

[1]: https://gitlab.com/gitlab-org/gitaly

---
Changes in v3:
- Cleanup some stale code which wasn't removed.
- Localize strings which will be output to the user.
- Remove additional defensive checks which are not needed.
- Link to v2: https://patch.msgid.link/20251126-kn-alternate-ref-dir-v2-0-8b9f6f18f635@gmail.com

Changes in v2:
- Added more clarification and proper intent in the cover message.
- Changed the format from '<ref_backend>://<path>' to
  `<ref_backend>://<URI-for-resource>` as it much clearer.
- Added logic to check for the '//' in the provided URI and a test for
  the same.
- In the tests:
  - Use test_must_fail() instead of ! git
  - Fix looped tests not using the variables correctly and ensure that
    the test description is correct.
- Link to v1: https://patch.msgid.link/20251119-kn-alternate-ref-dir-v1-0-4cf4a94c8bed@gmail.com

---
 Documentation/git.adoc |   8 ++++
 environment.h          |   1 +
 refs.c                 |  68 +++++++++++++++++++++++++--
 t/meson.build          |   1 +
 t/t1423-ref-backend.sh | 121 +++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 196 insertions(+), 3 deletions(-)

Karthik Nayak (2):
      refs: support obtaining ref_store for given dir
      refs: add GIT_REF_URI to specify reference backend and directory

Range-diff versus v2:

1:  5d37b2f0be = 1:  2b65f93e56 refs: support obtaining ref_store for given dir
2:  493c7ca098 ! 2:  d5dbb2f112 refs: add GIT_REF_URI to specify reference backend and directory
    @@ refs.c: static struct ref_store *get_ref_store_for_dir(struct repository *r,
     +	char *format_string;
     +	char *dir;
     +
    -+	if (!uri || !uri[0]) {
    -+		error("reference backend uri is empty");
    ++	if (!uri) {
    ++		error(_("reference backend uri is not provided"));
     +		goto cleanup;
     +	}
     +
     +	if (string_list_split(&ref_backend_info, uri, ":", 2) != 2) {
    -+		error("invalid reference backend uri format '%s'", uri);
    ++		error(_("invalid reference backend uri format '%s'"), uri);
     +		goto cleanup;
     +	}
     +
     +	format_string = ref_backend_info.items[0].string;
     +	if (!starts_with(ref_backend_info.items[1].string, "//")) {
    -+		error("invalid reference backend uri format '%s'", uri);
    ++		error(_("invalid reference backend uri format '%s'"), uri);
     +		goto cleanup;
     +	}
     +	dir = ref_backend_info.items[1].string + 2;
     +
    -+	format_string = ref_backend_info.items[0].string;
    -+	dir = ref_backend_info.items[1].string + 2;
    -+
    -+	if (!dir || !dir[0]) {
    -+		error("invalid path in uri '%s'", uri);
    ++	if (!dir[0]) {
    ++		error(_("invalid path in uri '%s'"), uri);
     +		goto cleanup;
     +	}
     +
     +	format = ref_storage_format_by_name(format_string);
     +	if (format == REF_STORAGE_FORMAT_UNKNOWN) {
    -+		error("unknown reference backend '%s'", format_string);
    ++		error(_("unknown reference backend '%s'"), format_string);
     +		goto cleanup;
     +	}
     +
    @@ t/t1423-ref-backend.sh (new)
     +		GIT_REF_URI="" &&
     +		export GIT_REF_URI &&
     +		test_must_fail git refs list 2>err &&
    -+		test_grep "reference backend uri is empty" err
    ++		test_grep "invalid reference backend uri format" err
     +	)
     +'
     +


base-commit: 9a2fb147f2c61d0cab52c883e7e26f5b7948e3ed
change-id: 20251105-kn-alternate-ref-dir-3e572e8cd0ef

Thanks
- Karthik