Re: Cloning from sites with 404 overridden
From: Junio C Hamano <hidden>
Date: 2016-06-15 22:42:21
Subsystem:
the rest · Maintainer:
Linus Torvalds
Possibly related (same subject, not in this thread)
- 2016-06-15 · Re: Cloning from sites with 404 overridden · <hidden>
- 2016-06-15 · Re: Cloning from sites with 404 overridden · Shawn Pearce <hidden>
- 2016-06-15 · Re: Cloning from sites with 404 overridden · Linus Torvalds <torvalds@osdl.org>
- 2016-06-15 · Re: Cloning from sites with 404 overridden · Marco Costalba <hidden>
- 2016-06-15 · Re: Cloning from sites with 404 overridden · Junio C Hamano <hidden>
"Marco Costalba" [off-list ref] writes:
Finally accessing the missing object with a browser http://digilander.libero.it/mcostalba/ scm/qgit.git/objects/8d/ea03519e75f47da91108330dde3043defddd60 gives a pre-canned (in italian) 'Sorry page not found' stuff. So I really think the site "HTTP/1.0 200 OK" response it's a fake. Perhaps security related to avoid sniffing (just a guess because I have absolutely zero competence in security related things).
I think you are just rephrasing what I said. From the HTTP protocol perspective, you _do_ have that 8d/3a0351 thing on that server, because you do not correctly say "No we donot have it" using 404 response. Your inability to produce 404 is a different matter -- often the hosting server is not under your control. But that does not change the fact that the repository observed by your clients is "broken". That is why a workaround flag like I suggested may be needed for such a setup. This is totally untested, but maybe something like this? ---
diff --git a/http-fetch.c b/http-fetch.c
index 7de818b..d523798 100644
--- a/http-fetch.c
+++ b/http-fetch.c@@ -8,6 +8,7 @@ #define RANGE_HEADER_SIZE 30 static int got_alternates = -1; +static int unreliable_404 = 0; static struct curl_slist *no_pragma_header;
@@ -822,12 +823,18 @@ static int fetch_object(struct alt_base close(obj_req->local); obj_req->local = -1; } + + if (obj_req->state == ABORTED) { ret = error("Request for %s aborted", hex); - } else if (obj_req->curl_result != CURLE_OK && - obj_req->http_code != 416) { + } else if ((obj_req->curl_result != CURLE_OK && + obj_req->http_code != 416) || + (unreliable_404 && + obj_req->curl_result == CURLE_OK && + obj_req->zret != Z_STREAM_END)) { if (obj_req->http_code == 404 || - obj_req->curl_result == CURLE_FILE_COULDNT_READ_FILE) + obj_req->curl_result == CURLE_FILE_COULDNT_READ_FILE || + unreliable_404) ret = -1; /* Be silent, it is probably in a pack. */ else ret = error("%s (curl_result = %d, http_code = %ld, sha1 = %s)",
@@ -966,6 +973,8 @@ int main(int argc, char **argv) arg++; } else if (!strcmp(argv[arg], "--recover")) { get_recover = 1; + } else if (!strcmp(argv[arg], "--unreliable-404")) { + unreliable_404 = 1; } arg++; }