Re: [PATCH 2/2] fetch-pack: do not check links for partial fetch

2 messages, 2 authors, 2018-03-14 · open the first message on its own page

Re: [PATCH 2/2] fetch-pack: do not check links for partial fetch

From: Junio C Hamano <hidden>
Date: 2018-03-14 21:55:40

Jonathan Tan [off-list ref] writes:
When doing a partial clone or fetch with transfer.fsckobjects=1, use the
--fsck-objects instead of the --strict flag when invoking index-pack so
that links are not checked, only objects. This is because incomplete
links are expected when doing a partial clone or fetch.
It is expected that _some_ links are missing, but this makes me
wonder if we can do better than disabling the connectivity check
altogether.  Does "git fetch" lack sufficient information to attempt
the connectivity check, and when (and only when) it hits a broken
link, see if that is because the connectivity check traversal is
crossing a "partial" fetch boundary, or something along that line?
quoted hunk
Signed-off-by: Jonathan Tan <redacted>
---
 fetch-pack.c             | 13 +++++++++++--
 t/t5616-partial-clone.sh | 11 +++++++++++
 2 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index d97461296..1d6117565 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -886,8 +886,17 @@ static int get_pack(struct fetch_pack_args *args,
 	    ? fetch_fsck_objects
 	    : transfer_fsck_objects >= 0
 	    ? transfer_fsck_objects
-	    : 0)
-		argv_array_push(&cmd.args, "--strict");
+	    : 0) {
+		if (args->from_promisor)
+			/*
+			 * We cannot use --strict in index-pack because it
+			 * checks both broken objects and links, but we only
+			 * want to check for broken objects.
+			 */
+			argv_array_push(&cmd.args, "--fsck-objects");
+		else
+			argv_array_push(&cmd.args, "--strict");
+	}
 
 	cmd.in = demux.out;
 	cmd.git_cmd = 1;
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index 29d863118..cee556536 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -143,4 +143,15 @@ test_expect_success 'manual prefetch of missing objects' '
 	test_line_count = 0 observed.oids
 '
 
+test_expect_success 'partial clone with transfer.fsckobjects=1 uses index-pack --fsck-objects' '
+	git init src &&
+	test_commit -C src x &&
+	test_config -C src uploadpack.allowfilter 1 &&
+	test_config -C src uploadpack.allowanysha1inwant 1 &&
+
+	GIT_TRACE="$(pwd)/trace" git -c transfer.fsckobjects=1 \
+		clone --filter="blob:none" "file://$(pwd)/src" dst &&
+	grep "git index-pack.*--fsck-objects" trace
+'
+
 test_done

Re: [PATCH 2/2] fetch-pack: do not check links for partial fetch

From: Jonathan Tan <hidden>
Date: 2018-03-14 22:24:37

On Wed, 14 Mar 2018 14:55:31 -0700
Junio C Hamano [off-list ref] wrote:
Jonathan Tan [off-list ref] writes:
quoted
When doing a partial clone or fetch with transfer.fsckobjects=1, use the
--fsck-objects instead of the --strict flag when invoking index-pack so
that links are not checked, only objects. This is because incomplete
links are expected when doing a partial clone or fetch.
It is expected that _some_ links are missing, but this makes me
wonder if we can do better than disabling the connectivity check
altogether.  Does "git fetch" lack sufficient information to attempt
the connectivity check, and when (and only when) it hits a broken
link, see if that is because the connectivity check traversal is
crossing a "partial" fetch boundary, or something along that line?
Our only definition (currently) for the "partial" fetch boundary is
whether an object in a promisor packfile (a packfile obtained from the
promisor remote) references it, so I think that checking for crossing a
"partial" fetch boundary does not add anything. This is because by that
definition, any missing links observed from objects newly fetched from
the promisor remote cross a "partial" fetch boundary (since all objects
fetched in this way "promise" all objects that they refer to).

But it is true that we might be able to do better in checking, for
example, that a packfile fetched using a blob size limit contains all
referenced trees (that is, only blobs are allowed to be missing).
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help