[PATCH v7 0/4] fetch: avoid fetching every branch of a new remote in a shallow repo

Avoid fetching every branch of a new remote in a shallow repo.

Changes in v7:

 * Fixed fatal: --refmap option is only meaningful with command-line
   refspec(s) firing just from configuring remote.<name>.refmap alone, with
   nothing else to fetch.
 * When both remote.<name>.fetch and remote.<name>.refmap are configured, a
   refspec-less fetch follows .fetch as usual, .refmap only remaps something
   already being fetched by name. Added tests.

Changes in v6:

 * Remove leftover reference to deleted default-branch logic in commit
   message.

Changes in v5:

 * Renumber test file from t5585 to t5586.

Changes in v4:

 * Removed the automatic default-branch fetch. A fresh remote fetches
   nothing until you track a branch explicitly.
 * Fixed fetch report showing "new ref HEAD" instead of "new branch "
 * Reworded remote..refmap docs and commit message.

Changes in v3:

 * Replace the special ":"/"+:" fetch refspec with remote.<name>.refmap,
   reusing git's existing --refmap mechanism instead of inventing new
   refspec syntax.
 * Split the change into 4 commits.

Changes in v2:

 * Replaced the opt-in fetch.shallow config entirely with a new special
   fetch refspec (+:) that git remote add now defaults new remotes to in a
   shallow repository. The new refspec fetches whichever branches any local
   branch tracks at that remote, plus the remote's default branch.

Harald Nordgren (4):
  fetch: add remote.<name>.refmap
  fetch: infer branches to fetch from a refmap-only remote
  remote: add "git remote add --limited-fetch"
  remote: default to --limited-fetch in a shallow repository

 Documentation/config/remote.adoc |   8 +++
 Documentation/fetch-options.adoc |   5 ++
 Documentation/git-remote.adoc    |  14 +++-
 builtin/fetch.c                  |  58 ++++++++++++---
 builtin/remote.c                 |  31 ++++++--
 remote.c                         |  41 ++++++++++-
 remote.h                         |   9 +++
 t/meson.build                    |   1 +
 t/t5505-remote.sh                |  76 ++++++++++++++++++++
 t/t5510-fetch.sh                 |  63 ++++++++++++++++
 t/t5586-fetch-refmap.sh          | 119 +++++++++++++++++++++++++++++++
 11 files changed, 408 insertions(+), 17 deletions(-)
 create mode 100755 t/t5586-fetch-refmap.sh


base-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2412%2FHaraldNordgren%2Ffetch-shallow-narrow-refspec-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2412/HaraldNordgren/fetch-shallow-narrow-refspec-v7
Pull-Request: https://github.com/git/git/pull/2412

Range-diff vs v6:

 1:  5c31a51bb7 ! 1:  b5db64d56f fetch: add remote.<name>.refmap
     @@ Commit message
          to do when there is nothing explicit to fetch, on the command line or
          via remote.<name>.fetch.
      
     +    A refspec-less fetch with a refmap configured but nothing else to say
     +    what to fetch falls back to the same defaults as a remote with no
     +    refspec or refmap at all. Only an explicit --refmap given on the
     +    command line with no command-line refspec to go with it is still
     +    rejected, since that combination is a plain mistake to type, unlike a
     +    remote.<name>.refmap configured on its own. A remote.<name>.fetch
     +    that is also configured is used as before.
     +
          Signed-off-by: Harald Nordgren [off-list ref]
      
       ## Documentation/config/remote.adoc ##
     @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote,
       		else
       			fetch_refspec = &remote->fetch;
       
     - 		for (i = 0; i < fetch_refspec->nr; i++)
     - 			get_fetch_map(ref_map, &fetch_refspec->items[i], &oref_tail, 1);
     --	} else if (refmap.nr) {
     -+	} else if (effective_refmap && effective_refmap->nr) {
     - 		die("--refmap option is only meaningful with command-line refspec(s)");
     - 	} else {
     - 		/* Use the defaults */
      
       ## remote.c ##
      @@ remote.c: static struct remote *make_remote(struct remote_state *remote_state,
     @@ t/t5510-fetch.sh: test_expect_success 'explicit --refmap option overrides remote
      +		git rev-parse --verify refs/remotes/other/main
      +	)
      +'
     ++
     ++check_fetched_refs () {
     ++	git for-each-ref --format="%(refname)" refs/remotes/ >actual &&
     ++	cat >expect &&
     ++	test_cmp expect actual
     ++}
     ++
     ++test_expect_success 'remote.<name>.refmap without tracking (baseline)' '
     ++	test_when_finished "rm -fr fetch-refmap-baseline fetch-refmap-upstream" &&
     ++	git init -b main fetch-refmap-upstream &&
     ++	test_commit -C fetch-refmap-upstream base &&
     ++	git -C fetch-refmap-upstream branch other &&
     ++	git init fetch-refmap-baseline &&
     ++	(
     ++		cd fetch-refmap-baseline &&
     ++		git remote add origin ../fetch-refmap-upstream &&
     ++
     ++		# Without fetch refspec, but with fetch refmap.
     ++		git config --unset-all remote.origin.fetch &&
     ++		git config remote.origin.refmap "+refs/heads/*:refs/remotes/origin/*" &&
     ++
     ++		# Nothing tracked, nothing fetched, no error.
     ++		git fetch origin &&
     ++		check_fetched_refs <<-\EOF &&
     ++		EOF
     ++
     ++		# Nothing tracked, explicit ref on the command line.
     ++		git fetch origin main &&
     ++		check_fetched_refs <<-\EOF &&
     ++		refs/remotes/origin/main
     ++		EOF
     ++
     ++		# With both refmap and fetch configured, remote.<name>.fetch
     ++		# wins: a refspec-less fetch follows it as usual, and
     ++		# remote.<name>.refmap plays no part in deciding what to
     ++		# fetch, only in remapping something that is already being
     ++		# fetched by name.
     ++		git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" &&
     ++		git fetch origin &&
     ++		check_fetched_refs <<-\EOF
     ++		refs/remotes/origin/HEAD
     ++		refs/remotes/origin/main
     ++		refs/remotes/origin/other
     ++		EOF
     ++	)
     ++'
      +
       test_expect_success 'explicitly empty --refmap option disables remote.*.fetch' '
       	git branch -f side &&
 2:  74c177e3e2 ! 2:  fd6864daaf fetch: infer branches to fetch from a refmap-only remote
     @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote,
       			get_fetch_map(ref_map, &fetch_refspec->items[i], &oref_tail, 1);
      +	} else if (inferred_branches) {
      +		/* Already fully handled above. */
     - 	} else if (effective_refmap && effective_refmap->nr) {
     + 	} else if (refmap.nr) {
       		die("--refmap option is only meaningful with command-line refspec(s)");
       	} else {
      @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote,
 3:  3abcc8915b = 3:  a2208875b6 remote: add "git remote add --limited-fetch"
 4:  84d192445c = 4:  c8fd073de3 remote: default to --limited-fetch in a shallow repository

-- 
gitgitgadget