From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:29
This is an RFC series to restart the smart HTTP transport work.
Those familiar with the native git:// protocol should be able to
quickly understand what I'm doing here by looking at only the last
two patches.
This time around I actually have the whole thing fully implemented
in JGit (both client and server), and am now trying to port that
over to C git.git, as well as document it in depth.
The JGit series can be found here at Eclipse.org:
http://egit.eclipse.org/r/
git://egit.eclipse.org/egit/parallelip-jgit refs/changes/50/50/4
This RFC C Git series only implements the server side, and only
has partial documentation. I did some limited smoke testing with
the JGit client against this server, it seems to work as expected.
I plan on trying to write the C Git clients tomorrow. The
send-pack/receive-pack protocol is trivial and shouldn't be
that hard, but the fetch-pack/upload-pack protocol is going
to be somewhat interesting...
Shawn O. Pearce (4):
Document the HTTP transport protocol
Git-aware CGI to provide dumb HTTP transport
Add smart-http options to upload-pack, receive-pack
Smart fetch and push over HTTP: server side
.gitignore | 1 +
Documentation/technical/http-protocol.txt | 542 +++++++++++++++++++++++++++++
Makefile | 1 +
builtin-receive-pack.c | 26 +-
http-backend.c | 394 +++++++++++++++++++++
upload-pack.c | 40 ++-
6 files changed, 994 insertions(+), 10 deletions(-)
create mode 100644 Documentation/technical/http-protocol.txt
create mode 100644 http-backend.c
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:29
When --smart-http is passed as a command line parameter to
upload-pack or receive-pack the programs now assume they may
perform only a single read-write cycle with stdin and stdout.
This fits with the HTTP POST request processing model where a
program may read the request, write a response, and must exit.
When --advertise-refs is passed as a command line parameter only
the initial ref advertisement is output, and the program exits
immediately. This fits with the HTTP GET request model, where
no request content is received but a response must be produced.
HTTP headers and/or environment are not processed here, but
instead are assumed to be handled by the program invoking
either service backend.
Signed-off-by: Shawn O. Pearce <redacted>
---
builtin-receive-pack.c | 26 ++++++++++++++++++++------
upload-pack.c | 40 ++++++++++++++++++++++++++++++++++++----
2 files changed, 56 insertions(+), 10 deletions(-)
@@ -623,7 +625,15 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)constchar*arg=*argv++;if(*arg=='-'){-/* Do flag handling here */+if(!strcmp(arg,"--advertise-refs")){+advertise_refs=1;+continue;+}+if(!strcmp(arg,"--smart-http")){+smart_http=1;+continue;+}+usage(receive_pack_usage);}if(dir)
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:29
Requests for $GIT_URL/git-receive-pack and $GIT_URL/git-upload-pack
are forwarded to the corresponding backend process by directly
executing it and leaving stdin and stdout connected to the web
server. Prior to starting the backend HTTP headers are sent, thereby
freeing the backend from needing to know about the HTTP protocol.
Signed-off-by: Shawn O. Pearce <redacted>
---
http-backend.c | 135 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 134 insertions(+), 1 deletions(-)
@@ -77,6 +77,95 @@ static NORETURN void not_found(const char *err, ...)exit(0);}+staticNORETURNvoidforbidden(constchar*err,...)+{+va_listparams;++write_status(403,"Forbidden");+write_nocache();+end_headers();++va_start(params,err);+if(err&&*err){+vsnprintf(buffer,sizeof(buffer),err,params);+fprintf(stderr,"%s\n",buffer);+}+va_end(params);+exit(0);+}++structhttp_service{+constchar*name;+constchar*config_name;+intenabled;+};+staticstructhttp_service*service;++staticstructhttp_servicehttp_service[]={+{"upload-pack","uploadpack",1},+{"receive-pack","receivepack",0},+};++staticinthttp_config(constchar*var,constchar*value,void*cb)+{+if(!prefixcmp(var,"http.")&&+!strcmp(var+7,service->config_name)){+service->enabled=git_config_bool(var,value);+return0;+}++/* we are not interested in parsing any other configuration here */+return0;+}++staticvoidselect_service(constchar*name)+{+inti;++if(prefixcmp(name,"git-"))+forbidden("Unsupported service: '%s'",name);++for(i=0;i<ARRAY_SIZE(http_service);i++){+service=&http_service[i];+if(!strcmp(service->name,name+4)){+git_config(http_config,NULL);+if(!service->enabled)+forbidden("Service not enabled: '%s'",name);+return;+}+}+forbidden("Unsupported service: '%s'",name);+}++staticvoidrun_service(constchar**argv)+{+#ifndef WIN32+execv_git_cmd(argv);+#else+structchild_processcld;++memset(&cld,0,sizeof(cld));+cld.argv=argv;+cld.git_cmd=1;+if(start_command(&cld))+die("Cannot start git-%s service",service->name);+close(0);+close(1);+finish_command(&cld);+#endif+}++staticvoidrequire_content_type(constchar*need_type)+{+constchar*input_type=getenv("CONTENT_TYPE");+if(!input_type||strcmp(input_type,need_type)){+write_status(415,"Unsupported Media Type");+write_nocache();+end_headers();+exit(0);+}+}+staticvoidwrite_file(constchar*the_type,constchar*name){constchar*p=git_path("%s",name);
@@ -151,6 +240,25 @@ static int show_text_ref(const char *name, const unsigned char *sha1,staticvoidget_info_refs(char*arg){+char*query=getenv("QUERY_STRING");++if(query&&!prefixcmp(query,"service=")){+constchar*argv[]={NULL/* service name */,+"--smart-http","--advertise-refs",+".",NULL};++select_service(query+8);++write_nocache();+format_write("%s: application/x-git-%s-advertisement\r\n",+content_type,service->name);+end_headers();+packet_write(1,"# service=git-%s\n",service->name);++argv[0]=service->name;+run_service(argv);+}+write_nocache();write_header(content_type,"text/plain; charset=utf-8");end_headers();
@@ -176,6 +284,28 @@ static void get_info_packs(char *arg)safe_write(1,"\n",1);}+staticvoidpost_to_service(char*service_name)+{+constchar*argv[]={NULL,"--smart-http",".",NULL};+unsignedn;++select_service(service_name);++n=snprintf(buffer,sizeof(buffer),+"application/x-git-%s-request",service->name);+if(n>=sizeof(buffer))+die("impossibly long service name");+require_content_type(buffer);++write_nocache();+format_write("%s: application/x-git-%s-result\r\n",+content_type,service->name);+end_headers();++argv[0]=service->name;+run_service(argv);+}+staticNORETURNvoiddie_webcgi(constchar*err,va_listparams){write_status(500,"Internal Server Error");
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:29
The git-http-backend CGI can be configured into any Apache server
using ScriptAlias, such as with the following configuration:
LoadModule cgi_module /usr/libexec/apache2/mod_cgi.so
LoadModule alias_module /usr/libexec/apache2/mod_alias.so
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Repositories are accessed via the translated PATH_INFO.
The CGI is backwards compatible with the dumb client, allowing all
older HTTP clients to continue to download repositories which are
managed by the CGI.
Signed-off-by: Shawn O. Pearce <redacted>
---
.gitignore | 1 +
Makefile | 1 +
http-backend.c | 261 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 263 insertions(+), 0 deletions(-)
create mode 100644 http-backend.c
@@ -361,6 +361,7 @@ PROGRAMS += git-show-index$XPROGRAMS+=git-unpack-file$XPROGRAMS+=git-upload-pack$XPROGRAMS+=git-var$X+PROGRAMS+=git-http-backend$X# List built-in command $C whose implementation cmd_$C() is not in# builtin-$C.o but is linked in as part of some other command.
@@ -0,0 +1,261 @@+#include"cache.h"+#include"refs.h"+#include"pkt-line.h"+#include"object.h"+#include"tag.h"+#include"exec_cmd.h"+#include"run-command.h"++staticconstcharcontent_type[]="Content-Type";+staticconstcharcontent_length[]="Content-Length";++staticcharbuffer[1024];++staticconstchar*http_date(unsignedlongtime)+{+returnshow_date(time,0,DATE_RFC2822);+}++staticvoidformat_write(constchar*fmt,...)+{+va_listargs;+unsignedn;++va_start(args,fmt);+n=vsnprintf(buffer,sizeof(buffer),fmt,args);+va_end(args);+if(n>=sizeof(buffer))+die("protocol error: impossibly long line");++safe_write(1,buffer,n);+}++staticvoidwrite_status(unsignedcode,constchar*msg)+{+format_write("Status: %u %s\r\n",code,msg);+}++staticvoidwrite_header(constchar*name,constchar*value)+{+format_write("%s: %s\r\n",name,value);+}++staticvoidend_headers(void)+{+safe_write(1,"\r\n",2);+}++staticvoidwrite_nocache(void)+{+write_header("Expires","Fri, 01 Jan 1980 00:00:00 GMT");+write_header("Pragma","no-cache");+write_header("Cache-Control","no-cache, max-age=0, must-revalidate");+}++staticvoidwrite_cache_forever(void)+{+unsignedlongnow=time(NULL);+write_header("Date",http_date(now));+write_header("Expires",http_date(now+31536000));+write_header("Cache-Control","public, max-age=31536000");+}++staticNORETURNvoidnot_found(constchar*err,...)+{+va_listparams;++write_status(404,"Not Found");+write_nocache();+end_headers();++va_start(params,err);+if(err&&*err){+vsnprintf(buffer,sizeof(buffer),err,params);+fprintf(stderr,"%s\n",buffer);+}+va_end(params);+exit(0);+}++staticvoidwrite_file(constchar*the_type,constchar*name)+{+constchar*p=git_path("%s",name);+intfd;+structstatsb;+uintmax_tremaining;++fd=open(p,O_RDONLY);+if(fd<0)+not_found("Cannot open '%s': %s",p,strerror(errno));+if(fstat(fd,&sb)<0)+die_errno("Cannot stat '%s'",p);+remaining=(uintmax_t)sb.st_size;++write_header(content_type,the_type);+write_header("Last-Modified",http_date(sb.st_mtime));+format_write("Content-Length: %"PRIuMAX"\r\n",remaining);+end_headers();++while(remaining){+ssize_tn=xread(fd,buffer,sizeof(buffer));+if(n<0)+die_errno("Cannot read '%s'",p);+n=safe_write(1,buffer,n);+if(n<=0)+break;+}+close(fd);+}++staticvoidget_text_file(char*name)+{+write_nocache();+write_file("text/plain; charset=utf-8",name);+}++staticvoidget_loose_object(char*name)+{+write_cache_forever();+write_file("application/x-git-loose-object",name);+}++staticvoidget_pack_file(char*name)+{+write_cache_forever();+write_file("application/x-git-packed-objects",name);+}++staticvoidget_idx_file(char*name)+{+write_cache_forever();+write_file("application/x-git-packed-objects-toc",name);+}++staticintshow_text_ref(constchar*name,constunsignedchar*sha1,+intflag,void*cb_data)+{+structobject*o=parse_object(sha1);+if(!o)+return0;++format_write("%s\t%s\n",sha1_to_hex(sha1),name);+if(o->type==OBJ_TAG){+o=deref_tag(o,name,0);+if(!o)+return0;+format_write("%s\t%s^{}\n",sha1_to_hex(o->sha1),name);+}++return0;+}++staticvoidget_info_refs(char*arg)+{+write_nocache();+write_header(content_type,"text/plain; charset=utf-8");+end_headers();++for_each_ref(show_text_ref,NULL);+}++staticvoidget_info_packs(char*arg)+{+size_tobjdirlen=strlen(get_object_directory());+structpacked_git*p;++write_nocache();+write_header(content_type,"text/plain; charset=utf-8");+end_headers();++prepare_packed_git();+for(p=packed_git;p;p=p->next){+if(!p->pack_local)+continue;+format_write("P %s\n",p->pack_name+objdirlen+6);+}+safe_write(1,"\n",1);+}++staticNORETURNvoiddie_webcgi(constchar*err,va_listparams)+{+write_status(500,"Internal Server Error");+write_nocache();+end_headers();++vsnprintf(buffer,sizeof(buffer),err,params);+fprintf(stderr,"fatal: %s\n",buffer);+exit(0);+}++staticstructservice_cmd{+constchar*method;+constchar*pattern;+void(*imp)(char*);+}services[]={+{"GET","/HEAD$",get_text_file},+{"GET","/info/refs$",get_info_refs},+{"GET","/objects/info/packs$",get_info_packs},+{"GET","/objects/info/[^/]*$",get_text_file},+{"GET","/objects/[0-9a-f]{2}/[0-9a-f]{38}$",get_loose_object},+{"GET","/objects/pack/pack-[0-9a-f]{40}\\.pack$",get_pack_file},+{"GET","/objects/pack/pack-[0-9a-f]{40}\\.idx$",get_idx_file}+};++intmain(intargc,char**argv)+{+char*dir=getenv("PATH_TRANSLATED");+char*input_method=getenv("REQUEST_METHOD");+structservice_cmd*cmd=NULL;+char*cmd_arg=NULL;+inti;++set_die_routine(die_webcgi);++if(!dir)+die("No PATH_TRANSLATED from server");+if(!input_method)+die("No REQUEST_METHOD from server");+if(!strcmp(input_method,"HEAD"))+input_method="GET";++for(i=0;i<ARRAY_SIZE(services);i++){+structservice_cmd*c=&services[i];+regex_tre;+regmatch_tout[1];++if(regcomp(&re,c->pattern,REG_EXTENDED))+die("Bogus regex in service table: %s",c->pattern);+if(!regexec(&re,dir,1,out,0)){+size_tn=out[0].rm_eo-out[0].rm_so;++if(strcmp(input_method,c->method)){+constchar*proto=getenv("SERVER_PROTOCOL");+if(proto&&!strcmp(proto,"HTTP/1.1"))+write_status(405,"Method Not Allowed");+else+write_status(400,"Bad Request");+write_nocache();+end_headers();+return0;+}++cmd=c;+cmd_arg=xmalloc(n);+strncpy(cmd_arg,dir+out[0].rm_so+1,n);+cmd_arg[n]='\0';+dir[out[0].rm_so]=0;+break;+}+regfree(&re);+}++if(!cmd)+not_found("Request not supported: '%s'",dir);++setup_path();+if(!enter_repo(dir,0))+not_found("Not a git repository: '%s'",dir);++cmd->imp(cmd_arg);+return0;+}
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================++Git supports two HTTP based transfer protocols. A "dumb" protocol+which requires only a standard HTTP server on the server end of the+connection, and a "smart" protocol which requires a Git aware CGI+(or server module). This document describes both protocols.++As a design feature smart clients can automatically upgrade "dumb"+protocol URLs to smart URLs. This permits all users to have the+same published URL, and the peers automatically select the most+efficient transport available to them.+++URL Format+----------++URLs for Git repositories accessed by HTTP use the standard HTTP+URL syntax documented by RFC 1738, so they are of the form:++ http://<host>:<port>/<path>++Within this documentation the placeholder $GIT_URL will stand for+the http:// repository URL entered by the end-user.++Both the "smart" and "dumb" HTTP protocols used by Git operate+by appending additional path components onto the end of the user+supplied $GIT_URL string.++Clients MUST strip a trailing '/', if present, from the user supplied+$GIT_URL string to prevent empty path tokens ('//') from appearing+in any URL sent to a server. Compatible clients must expand+'$GIT_URL/info/refs' as 'foo/info/refs' and not 'foo//info/refs'.+++Authentication+--------------++Standard HTTP authentication is used if authentication is required+to access a repository, and MAY be configured and enforced by the+HTTP server software.++Because Git repositories are accessed by standard path components+server administrators MAY use directory based permissions within+their HTTP server to control repository access.++Clients SHOULD support Basic authentication as described by RFC 2616.+Servers SHOULD support Basic authentication by relying upon the+HTTP server placed in front of the Git server software.++Servers MUST NOT require HTTP cookies for the purposes of+authentication or access control.++Clients and servers MAY support other common forms of HTTP based+authentication, such as Digest authentication.+++SSL+---++Clients and servers SHOULD support SSL, particularly to protect+passwords when relying on Basic HTTP authentication.+++Session State+-------------++The Git over HTTP protocol (much like HTTP itself) is stateless+from the perspective of the HTTP server side. All state must be+retained and managed by the client process. This permits simple+round-robin load-balancing on the server side, without needing to+worry about state mangement.++Clients MUST NOT require state management on the server side in+order to function correctly.++Servers MUST NOT require HTTP cookies in order to function correctly.+Clients MAY store and forward HTTP cookies during request processing+as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any+cookies sent by a client.+++pkt-line Format+---------------++Much (but not all) of the payload is described around pkt-lines.++A pkt-line is a variable length binary string. The first four bytes+of the line indicates the total length of the line, in hexadecimal.+The total length includes the 4 bytes used to denote the length.+A line SHOULD BE terminated by an LF, which if present MUST be+included in the total length.++A pkt-line MAY contain binary data, so implementors MUST ensure all+pkt-line parsing/formatting routines are 8-bit clean. The maximum+length of a pkt-line's data is 65532 bytes (65536 - 4).++Examples (as C-style strings):++ pkt-line actual value+ ---------------------------------+ "0006a\n" "a\n"+ "0005a" "a"+ "000bfoobar\n" "foobar\n"+ "0004" ""++A pkt-line with a length of 0 ("0000") is a special case and MUST+be treated as a message break or terminator in the payload.+++General Request Processing+--------------------------++Except where noted, all standard HTTP behavior SHOULD be assumed+by both client and server. This includes (but is not necessarily+limited to):++If there is no repository at $GIT_URL, the server MUST respond with+the '404 Not Found' HTTP status code.++If there is a repository at $GIT_URL, but access is not currently+permitted, the server MUST respond with the '403 Forbidden' HTTP+status code.++Servers SHOULD support both HTTP 1.0 and HTTP 1.1.+Servers SHOULD support chunked encoding for both+request and response bodies.++Clients SHOULD support both HTTP 1.0 and HTTP 1.1.+Clients SHOULD support chunked encoding for both+request and response bodies.++Servers MAY return ETag and/or Last-Modified headers.++Clients MAY revalidate cached entities by including If-Modified-Since+and/or If-None-Match request headers.++Servers MAY return '304 Not Modified' if the relevant headers appear+in the request and the entity has not changed. Clients MUST treat+'304 Not Modified' identical to '200 OK' by reusing the cached entity.++Clients MAY reuse a cached entity without revalidation if the+Cache-Control and/or Expires header permits caching. Clients and+servers MUST follow RFC 2616 for cache controls.+++Discovering References+----------------------++All HTTP clients MUST begin either a fetch or a push exchange by+discovering the references available on the remote repository.++Dumb Clients+~~~~~~~~~~~~++HTTP clients that only support the "dumb" protocol MUST discover+references by making a request for the special info/refs file of+the repository.++Dumb HTTP clients MUST NOT include search/query parameters when+fetching the info/refs file. (That is, '?' must not appear in the+requested URL.)++ C: GET $GIT_URL/info/refs HTTP/1.0++ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++The Content-Type of the returned info/refs entity SHOULD be+"text/plain; charset=utf-8", but MAY be any content type.+Clients MUST NOT attempt to validate the returned Content-Type.+Dumb servers MUST NOT return a return type starting with+"application/x-git-".++Cache-Control headers MAY be returned to disable caching of the+returned entity.++When examining the response clients SHOULD only examine the HTTP+status code. Valid responses are '200 OK', or '304 Not Modified'.++The returned content is a UNIX formatted text file describing+each ref and its known value. The file SHOULD be sorted by name+according to the C locale ordering. The file SHOULD NOT include+the default ref named 'HEAD'.++ info_refs = *( ref_record )+ ref_record = any_ref | peeled_ref++ any_ref = id HT name LF+ peeled_ref = id HT name LF+ id HT name "^{}" LF+ id = 40*HEX++ HEX = "0".."9" | "a".."f"+ LF = <US-ASCII LF, linefeed (10)>+ HT = <US-ASCII HT, horizontal-tab (9)>++Smart Clients+~~~~~~~~~~~~~++HTTP clients that support the "smart" protocol (or both the+"smart" and "dumb" protocols) MUST discover references by making+a paramterized request for the info/refs file of the repository.++The request MUST contain exactly one query parameter,+'service=$servicename', where $servicename MUST be the service+name the client wishes to contact to complete the operation.+The request MUST NOT contain additional query parameters.++ C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0++ dumb server reply:+ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++ smart server reply:+ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-advertisement+ S: Cache-Control: no-cache+ S:+ S: ....# service=git-upload-pack+ S: ....95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0 multi_ack+ S: ....d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: ....2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: ....a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++Dumb Server Response+^^^^^^^^^^^^^^^^^^^^+Dumb servers MUST respond with the dumb server reply format.++See the prior section under dumb clients for a more detailed+description of the dumb server response.++Smart Server Response+^^^^^^^^^^^^^^^^^^^^^+Smart servers MUST respond with the smart server reply format.++If the server does not recognize the requested service name, or the+requested service name has been disabled by the server administrator,+the server MUST respond with the '403 Forbidden' HTTP status code.++Cache-Control headers SHOULD be used to disable caching of the+returned entity.++The Content-Type MUST be 'application/x-$servicename-advertisement'.+Clients SHOULD fall back to the dumb protocol if another content+type is returned. When falling back to the dumb protocol clients+SHOULD NOT make an additional request to $GIT_URL/info/refs, but+instead SHOULD use the response already in hand. Clients MUST NOT+continue if they do not support the dumb protocol.++Clients MUST validate the status code is either '200 OK' or+'304 Not Modified'.++Clients MUST validate the first five bytes of the response entity+matches the regex "^[0-9a-f]{4}#". If this test fails, clients+MUST NOT continue.++Clients MUST parse the entire response as a sequence of pkt-line+records.++Clients MUST verify the first pkt-line is "# service=$servicename".+Servers MUST set $servicename to be the request parameter value.+Servers SHOULD include an LF at the end of this line.+Clients MUST ignore an LF at the end of the line.++Servers MUST terminate the response with the magic "0000" end+pkt-line marker.++The returned response is a pkt-line stream describing each ref and+its known value. The stream SHOULD be sorted by name according to+the C locale ordering. The stream SHOULD include the default ref+named 'HEAD' as the first ref. The stream MUST include capability+declarations behind a NUL on the first ref.++ smart_reply = PKT-LINE("# service=$servicename" LF)+ ref_list+ "0000"+ ref_list = empty_list | populated_list++ empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)++ non_empty_list = PKT-LINE(id SP name NUL cap_list LF)+ *ref_record++ cap_list = *(SP capability) SP+ ref_record = any_ref | peeled_ref++ any_ref = PKT-LINE(id SP name LF)+ peeled_ref = PKT-LINE(id SP name LF)+ PKT-LINE(id SP name "^{}" LF+ id = 40*HEX++ HEX = "0".."9" | "a".."f"+ NL = <US-ASCII NUL, null (0)>+ LF = <US-ASCII LF, linefeed (10)>+ SP = <US-ASCII SP, horizontal-tab (9)>+++Smart Service git-upload-pack+------------------------------+This service reads from the remote repository.++Clients MUST first perform ref discovery with+'$GIT_URL/info/refs?service=git-upload-pack'.++ C: POST $GIT_URL/git-upload-pack HTTP/1.0+ C: Content-Type: application/x-git-upload-pack-request+ C:+ C: ....want 0a53e9ddeaddad63ad106860237bbf53411d11a7+ C: ....have 441b40d833fdfa93eb2908e52742248faf0ee993+ C: 0000++ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-result+ S: Cache-Control: no-cache+ S:+ S: ....ACK %s, continue+ S: ....NAK++Clients MUST NOT reuse or revalidate a cached reponse.+Servers MUST include sufficient Cache-Control headers+to prevent caching of the response.++Servers SHOULD support all capabilities defined here.++Clients MUST send at least one 'want' command in the request body.+Clients MUST NOT reference an id in a 'want' command which did not+appear in the response obtained through ref discovery.++ compute_request = want_list+ have_list+ request_end+ request_end = "0000" | "done"++ want_list = PKT-LINE(want NUL cap_list LF)+ *(want_pkt)+ want_pkt = PKT-LINE(want LF)+ want = "want" SP id+ cap_list = *(SP capability) SP++ have_list = *PKT-LINE("have" SP id LF)++ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name++TODO: Document this further.+TODO: Don't use uppercase for variable names below.++Capability include-tag+~~~~~~~~~~~~~~~~~~~~~~++When packing an object that an annotated tag points at, include the+tag object too. Clients can request this if they want to fetch+tags, but don't know which tags they will need until after they+receive the branch data. By enabling include-tag an entire call+to upload-pack can be avoided.++Capability thin-pack+~~~~~~~~~~~~~~~~~~~~++When packing a deltified object the base is not included if the base+is reachable from an object listed in the COMMON set by the client.+This reduces the bandwidth required to transfer, but it does slightly+increase processing time for the client to save the pack to disk.++The Negotiation Algorithm+~~~~~~~~~~~~~~~~~~~~~~~~~+The computation to select the minimal pack proceeds as follows+(c = client, s = server):++ init step:+ (c) Use ref discovery to obtain the advertised refs.+ (c) Place any object seen into set ADVERTISED.++ (c) Build an empty set, COMMON, to hold the objects that are later+ determined to be on both ends.+ (c) Build a set, WANT, of the objects from ADVERTISED the client+ wants to fetch, based on what it saw during ref discovery.++ (c) Start a queue, C_PENDING, ordered by commit time (popping newest+ first). Add all client refs. When a commit is popped from+ the queue its parents should be automatically inserted back.+ Commits MUST only enter the queue once.++ one compute step:+ (c) Send one $GIT_URL/git-upload-pack request:++ C: 0032want <WANT #1>...............................+ C: 0032want <WANT #2>...............................+ ....+ C: 0032have <COMMON #1>.............................+ C: 0032have <COMMON #2>.............................+ ....+ C: 0032have <HAVE #1>...............................+ C: 0032have <HAVE #2>...............................+ ....+ C: 0000++ The stream is organized into "commands", with each command+ appearing by itself in a pkt-line. Within a command line+ the text leading up to the first space is the command name,+ and the remainder of the line to the first LF is the value.+ Command lines are terminated with an LF as the last byte of+ the pkt-line value.++ Commands MUST appear in the following order, if they appear+ at all in the request stream:++ * want+ * have++ The stream is terminated by a pkt-line flush ("0000").++ A single "want" or "have" command MUST have one hex formatted+ SHA-1 as its value. Multiple SHA-1s MUST be sent by sending+ multiple commands.++ The HAVE list is created by popping the first 32 commits+ from C_PENDING. Less can be supplied if C_PENDING empties.++ If the client has sent 256 HAVE commits and has not yet+ received one of those back from S_COMMON, or the client has+ emptied C_PENDING it should include a "done" command to let+ the server know it won't proceed:++ C: 0009done++ (s) Parse the git-upload-pack request:++ Verify all objects in WANT are directly reachable from refs.++ The server MAY walk backwards through history or through+ the reflog to permit slightly stale requests.++ If no WANT objects are received, send an error:++TODO: Define error if no want lines are requested.++ If any WANT object is not reachable, send an error:++TODO: Define error if an invalid want is requested.++ Create an empty list, S_COMMON.++ If 'have' was sent:++ Loop through the objects in the order supplied by the client.+ For each object, if the server has the object reachable from+ a ref, add it to S_COMMON. If a commit is added to S_COMMON,+ do not add any ancestors, even if they also appear in HAVE.++ (s) Send the git-upload-pack response:++ If the server has found a closed set of objects to pack or the+ request ends with "done", it replies with the pack.++TODO: Document the pack based response+ S: PACK...++ The returned stream is the side-band-64k protocol supported+ by the git-upload-pack service, and the pack is embedded into+ stream 1. Progress messages from the server side may appear+ in stream 2.++ Here a "closed set of objects" is defined to have at least+ one path from every WANT to at least one COMMON object.++ If the server needs more information, it replies with a+ status continue response:++TODO: Document the non-pack response++ (c) Parse the upload-pack response:++TODO: Document parsing response++ Do another compute step.+++Smart Service git-receive-pack+------------------------------+This service modifies the remote repository.++Clients MUST first perform ref discovery with+'$GIT_URL/info/refs?service=git-receive-pack'.++ C: POST $GIT_URL/git-receive-pack HTTP/1.0+ C: Content-Type: application/x-git-receive-pack-request+ C:+ C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status+ C: 0000+ C: PACK....++ S: 200 OK+ S: Content-Type: application/x-git-receive-pack-result+ S: Cache-Control: no-cache+ S:+ S: ....++Clients MUST NOT reuse or revalidate a cached reponse.+Servers MUST include sufficient Cache-Control headers+to prevent caching of the response.++Servers SHOULD support all capabilities defined here.++Clients MUST send at least one command in the request body.+Within the command portion of the request body clients SHOULD send+the id obtained through ref discovery as old_id.++ update_request = command_list+ "PACK" <binary data>++ command_list = PKT-LINE(command NUL cap_list LF)+ *(command_pkt)+ command_pkt = PKT-LINE(command LF)+ cap_list = *(SP capability) SP++ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name++TODO: Document this further.+++References+----------++link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)]+link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]+
I dunno I kinda object to it being called http-backend, personally I'd
rather it be called git-smart since this is the smart http protocol ;-)
- John 'Warthog9' Hawley
Shawn O. Pearce wrote:
quoted hunk
The git-http-backend CGI can be configured into any Apache server
using ScriptAlias, such as with the following configuration:
LoadModule cgi_module /usr/libexec/apache2/mod_cgi.so
LoadModule alias_module /usr/libexec/apache2/mod_alias.so
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Repositories are accessed via the translated PATH_INFO.
The CGI is backwards compatible with the dumb client, allowing all
older HTTP clients to continue to download repositories which are
managed by the CGI.
Signed-off-by: Shawn O. Pearce <redacted>
---
.gitignore | 1 +
Makefile | 1 +
http-backend.c | 261 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 263 insertions(+), 0 deletions(-)
create mode 100644 http-backend.c
@@ -361,6 +361,7 @@ PROGRAMS += git-show-index$XPROGRAMS+=git-unpack-file$XPROGRAMS+=git-upload-pack$XPROGRAMS+=git-var$X+PROGRAMS+=git-http-backend$X# List built-in command $C whose implementation cmd_$C() is not in# builtin-$C.o but is linked in as part of some other command.
@@ -0,0 +1,261 @@+#include"cache.h"+#include"refs.h"+#include"pkt-line.h"+#include"object.h"+#include"tag.h"+#include"exec_cmd.h"+#include"run-command.h"++staticconstcharcontent_type[]="Content-Type";+staticconstcharcontent_length[]="Content-Length";++staticcharbuffer[1024];++staticconstchar*http_date(unsignedlongtime)+{+returnshow_date(time,0,DATE_RFC2822);+}++staticvoidformat_write(constchar*fmt,...)+{+va_listargs;+unsignedn;++va_start(args,fmt);+n=vsnprintf(buffer,sizeof(buffer),fmt,args);+va_end(args);+if(n>=sizeof(buffer))+die("protocol error: impossibly long line");++safe_write(1,buffer,n);+}++staticvoidwrite_status(unsignedcode,constchar*msg)+{+format_write("Status: %u %s\r\n",code,msg);+}++staticvoidwrite_header(constchar*name,constchar*value)+{+format_write("%s: %s\r\n",name,value);+}++staticvoidend_headers(void)+{+safe_write(1,"\r\n",2);+}++staticvoidwrite_nocache(void)+{+write_header("Expires","Fri, 01 Jan 1980 00:00:00 GMT");+write_header("Pragma","no-cache");+write_header("Cache-Control","no-cache, max-age=0, must-revalidate");+}++staticvoidwrite_cache_forever(void)+{+unsignedlongnow=time(NULL);+write_header("Date",http_date(now));+write_header("Expires",http_date(now+31536000));+write_header("Cache-Control","public, max-age=31536000");+}++staticNORETURNvoidnot_found(constchar*err,...)+{+va_listparams;++write_status(404,"Not Found");+write_nocache();+end_headers();++va_start(params,err);+if(err&&*err){+vsnprintf(buffer,sizeof(buffer),err,params);+fprintf(stderr,"%s\n",buffer);+}+va_end(params);+exit(0);+}++staticvoidwrite_file(constchar*the_type,constchar*name)+{+constchar*p=git_path("%s",name);+intfd;+structstatsb;+uintmax_tremaining;++fd=open(p,O_RDONLY);+if(fd<0)+not_found("Cannot open '%s': %s",p,strerror(errno));+if(fstat(fd,&sb)<0)+die_errno("Cannot stat '%s'",p);+remaining=(uintmax_t)sb.st_size;++write_header(content_type,the_type);+write_header("Last-Modified",http_date(sb.st_mtime));+format_write("Content-Length: %"PRIuMAX"\r\n",remaining);+end_headers();++while(remaining){+ssize_tn=xread(fd,buffer,sizeof(buffer));+if(n<0)+die_errno("Cannot read '%s'",p);+n=safe_write(1,buffer,n);+if(n<=0)+break;+}+close(fd);+}++staticvoidget_text_file(char*name)+{+write_nocache();+write_file("text/plain; charset=utf-8",name);+}++staticvoidget_loose_object(char*name)+{+write_cache_forever();+write_file("application/x-git-loose-object",name);+}++staticvoidget_pack_file(char*name)+{+write_cache_forever();+write_file("application/x-git-packed-objects",name);+}++staticvoidget_idx_file(char*name)+{+write_cache_forever();+write_file("application/x-git-packed-objects-toc",name);+}++staticintshow_text_ref(constchar*name,constunsignedchar*sha1,+intflag,void*cb_data)+{+structobject*o=parse_object(sha1);+if(!o)+return0;++format_write("%s\t%s\n",sha1_to_hex(sha1),name);+if(o->type==OBJ_TAG){+o=deref_tag(o,name,0);+if(!o)+return0;+format_write("%s\t%s^{}\n",sha1_to_hex(o->sha1),name);+}++return0;+}++staticvoidget_info_refs(char*arg)+{+write_nocache();+write_header(content_type,"text/plain; charset=utf-8");+end_headers();++for_each_ref(show_text_ref,NULL);+}++staticvoidget_info_packs(char*arg)+{+size_tobjdirlen=strlen(get_object_directory());+structpacked_git*p;++write_nocache();+write_header(content_type,"text/plain; charset=utf-8");+end_headers();++prepare_packed_git();+for(p=packed_git;p;p=p->next){+if(!p->pack_local)+continue;+format_write("P %s\n",p->pack_name+objdirlen+6);+}+safe_write(1,"\n",1);+}++staticNORETURNvoiddie_webcgi(constchar*err,va_listparams)+{+write_status(500,"Internal Server Error");+write_nocache();+end_headers();++vsnprintf(buffer,sizeof(buffer),err,params);+fprintf(stderr,"fatal: %s\n",buffer);+exit(0);+}++staticstructservice_cmd{+constchar*method;+constchar*pattern;+void(*imp)(char*);+}services[]={+{"GET","/HEAD$",get_text_file},+{"GET","/info/refs$",get_info_refs},+{"GET","/objects/info/packs$",get_info_packs},+{"GET","/objects/info/[^/]*$",get_text_file},+{"GET","/objects/[0-9a-f]{2}/[0-9a-f]{38}$",get_loose_object},+{"GET","/objects/pack/pack-[0-9a-f]{40}\\.pack$",get_pack_file},+{"GET","/objects/pack/pack-[0-9a-f]{40}\\.idx$",get_idx_file}+};++intmain(intargc,char**argv)+{+char*dir=getenv("PATH_TRANSLATED");+char*input_method=getenv("REQUEST_METHOD");+structservice_cmd*cmd=NULL;+char*cmd_arg=NULL;+inti;++set_die_routine(die_webcgi);++if(!dir)+die("No PATH_TRANSLATED from server");+if(!input_method)+die("No REQUEST_METHOD from server");+if(!strcmp(input_method,"HEAD"))+input_method="GET";++for(i=0;i<ARRAY_SIZE(services);i++){+structservice_cmd*c=&services[i];+regex_tre;+regmatch_tout[1];++if(regcomp(&re,c->pattern,REG_EXTENDED))+die("Bogus regex in service table: %s",c->pattern);+if(!regexec(&re,dir,1,out,0)){+size_tn=out[0].rm_eo-out[0].rm_so;++if(strcmp(input_method,c->method)){+constchar*proto=getenv("SERVER_PROTOCOL");+if(proto&&!strcmp(proto,"HTTP/1.1"))+write_status(405,"Method Not Allowed");+else+write_status(400,"Bad Request");+write_nocache();+end_headers();+return0;+}++cmd=c;+cmd_arg=xmalloc(n);+strncpy(cmd_arg,dir+out[0].rm_so+1,n);+cmd_arg[n]='\0';+dir[out[0].rm_so]=0;+break;+}+regfree(&re);+}++if(!cmd)+not_found("Request not supported: '%s'",dir);++setup_path();+if(!enter_repo(dir,0))+not_found("Not a git repository: '%s'",dir);++cmd->imp(cmd_arg);+return0;+}
Heya,
I had some spare time, I hope these comments from someone that is not
too familiar with the protocol are helpful :).
On Fri, Oct 9, 2009 at 07:22, Shawn O. Pearce [off-list ref] wrote:
+Compatible clients must expand
+'$GIT_URL/info/refs' as 'foo/info/refs' and not 'foo//info/refs'.
Does this need a s/non_empty_list/populated_list/ ?
+ cap_list = *(SP capability) SP
You never define capability.
+ (c) Send one $GIT_URL/git-upload-pack request:
I don't think you documented what $GIT_URL/git-upload-pack means.
+ If the client has sent 256 HAVE commits and has not yet
+ received one of those back from S_COMMON, or the client has
+ emptied C_PENDING it should include a "done" command to let
+ the server know it won't proceed:
+
+ C: 0009done
This should probably move down to after you define what S_COMMON is in
the first place.
+ Here a "closed set of objects" is defined to have at least
+ one path from every WANT to at least one COMMON object.
A 'path from' is perhaps a bit unclear.
--
Cheers,
Sverre Rabbelier
Heya,
On Fri, Oct 9, 2009 at 10:01, Sverre Rabbelier [off-list ref] wrote:
quoted
+ (c) Send one $GIT_URL/git-upload-pack request:
I don't think you documented what $GIT_URL/git-upload-pack means.
Ah, I didn't realize until I read 4/4 that this is just a regular
request to the 'http://<host>:<port>/git-upload-pack' url, I was
confused by the need to query
"http://<host>:<port>/info/refs?service=git-upload-pack".
--
Cheers,
Sverre Rabbelier
From: Alex Blewitt <hidden> Date: 2016-06-15 22:47:29
Shawn O. Pearce <spearce <at> spearce.org> writes:
+URL Format
+----------
+
+URLs for Git repositories accessed by HTTP use the standard HTTP
+URL syntax documented by RFC 1738, so they are of the form:
+
+ http://<host>:<port>/<path>
+
+Within this documentation the placeholder $GIT_URL will stand for
+the http:// repository URL entered by the end-user.
It's worth making clear here that $GIT_URL will be the path to the repository,
rather than necessarily just the host upon which the server sits. Perhaps
including an example, like http://example:8080/repos/example.git
would make it clearer that there can be a path (and so leading to
a request like http://example:8080/repos/example.git/info/refs?service=...
It's also worth clarifying, therefore, that multiple repositories can be served
by the same process (as with the git server today) by using different path(s).
And for those that are interested in submodules, it's worth confirming that
http://example/repos/master.git/child.git/info/refs?service= will ensure
that the repository is the 'child' git rather than anything else.
HEX = [0-9a-f]
Is there any reason not to support A-F as well in the hex spec, even if they
SHOULD use a-f? This may limit the appeal for some case-insensitive systems.
It would also be good to document, like with the git daemon, whether all
repositories under a path are exported or only those that have the magic
setting in the config like git-daemon-export-ok.
Lastly, it would be good to clarify when the result of this GET/POST exchange
is a text-based (and encoded in UTF-8) vs when binary data is returned; we
don't want to get into the state where we're returning binary data and
pretending that it's UTF-8.
Alex
Errr... are you sure? Because from examples it looks like cap_list
(capabilities list) is a list of space *separated* capabilities, while
the above requires also both leading and trailing space. Shouldn't it
be
cap_list = capability *(SP capability)
Also the format for capability is not defined; I guess only
a-z, 0-9, '-' and '_' are allowed in capability name.
BTW. is it possible to not have capability list?
+ HEX = "0".."9" | "a".."f"
Do you plan allowing also upper case letters, while server and client
SHOULD use lowercase? Because if you do, then RFC 5234 which defines
ABNF you seem to be using here has HEXDIG defined.
You should also reference the following RFCs:
* "RFC 5234: Augmented BNF for Syntax Specifications: ABNF"
* "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels"
--
Jakub Narebski
Poland
ShadeHawk on #git
From: Jeff King <hidden> Date: 2016-06-15 22:47:30
On Thu, Oct 08, 2009 at 10:22:45PM -0700, Shawn O. Pearce wrote:
+Servers MUST NOT require HTTP cookies for the purposes of
+authentication or access control.
[...]
+Servers MUST NOT require HTTP cookies in order to function correctly.
+Clients MAY store and forward HTTP cookies during request processing
+as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any
+cookies sent by a client.
Why not? I can grant that the current git implementation probably can't
handle it, but keep in mind this is talking about the protocol and not
the implementation. And I can see it being useful for sites like github
which already have a cookie-based login. Adapting the client to handle
this case would not be too difficult (it would just mean keeping cookie
state in a file between runs, or even just pulling it out of the normal
browser's cookie store). And people whose client didn't do this would
simply get an "access denied" response code.
Is there a technical reason not to allow it?
-Peff
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================
...
+As a design feature smart clients can automatically upgrade "dumb"
+protocol URLs to smart URLs. This permits all users to have the
+same published URL, and the peers automatically select the most
+efficient transport available to them.
The first sentence feels backwards although the conclusion in the second
sentence is true. It is more like smart ones trying smart protocol first,
and downgrading to "dumb" after noticing that the server is not smart.
+Authentication
+--------------
...
+Clients SHOULD support Basic authentication as described by RFC 2616.
+Servers SHOULD support Basic authentication by relying upon the
+HTTP server placed in front of the Git server software.
It is perfectly fine to make it a requirement for a server to support the
Basic authentication, but should you make it a requirement that the
support is done by a specific implementation, i.e. "by relying upon..."?
+Session State
+-------------
...
+retained and managed by the client process. This permits simple
+round-robin load-balancing on the server side, without needing to
+worry about state mangement.
s/mangement/management/;
+pkt-line Format
+---------------
...
+Examples (as C-style strings):
+
+ pkt-line actual value
+ ---------------------------------
+ "0006a\n" "a\n"
+ "0005a" "a"
+ "000bfoobar\n" "foobar\n"
+ "0004" ""
+
+A pkt-line with a length of 0 ("0000") is a special case and MUST
+be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
+General Request Processing
+--------------------------
+
+Except where noted, all standard HTTP behavior SHOULD be assumed
+by both client and server. This includes (but is not necessarily
+limited to):
+
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
+Dumb Clients
+~~~~~~~~~~~~
+
+HTTP clients that only support the "dumb" protocol MUST discover
+references by making a request for the special info/refs file of
+the repository.
+
+Dumb HTTP clients MUST NOT include search/query parameters when
+fetching the info/refs file. (That is, '?' must not appear in the
+requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
+ C: GET $GIT_URL/info/refs HTTP/1.0
+
+ S: 200 OK
...
+When examining the response clients SHOULD only examine the HTTP
+status code. Valid responses are '200 OK', or '304 Not Modified'.
Isn't 401 ("Ah, I was given a wrong URL") and 403 ("Ok, I do not have an
access to this repository") also valid?
+The returned content is a UNIX formatted text file describing
+each ref and its known value. The file SHOULD be sorted by name
+according to the C locale ordering. The file SHOULD NOT include
+the default ref named 'HEAD'.
I know you said "known" to imply "concurrent operations may change it
while the server is serving this client", but it feels rather awkward.
+Smart Server Response
+^^^^^^^^^^^^^^^^^^^^^
+
+Smart servers MUST respond with the smart server reply format.
+If the server does not recognize the requested service name, or the
+requested service name has been disabled by the server administrator,
+the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
+The Content-Type MUST be 'application/x-$servicename-advertisement'.
+Clients SHOULD fall back to the dumb protocol if another content
+type is returned. When falling back to the dumb protocol clients
+SHOULD NOT make an additional request to $GIT_URL/info/refs, but
+instead SHOULD use the response already in hand. Clients MUST NOT
+continue if they do not support the dumb protocol.
The part I commented on (the beginning of Smart Server Response) was
written as a generic description, not specific to git-upload-pack service,
and the beginning of this paragraph also pretends to be a generic
description, but it is misleading. This is a specific instruction to the
clients that asked for git-upload-pack service and got a dumb server
response (if the above were talking about something other than upload-pack
service, there is no guarantee that "response already in hand" is useful
to talk to dumb servers).
+The returned response is a pkt-line stream describing each ref and
+its known value. The stream SHOULD be sorted by name according to
+the C locale ordering. The stream SHOULD include the default ref
+named 'HEAD' as the first ref. The stream MUST include capability
+declarations behind a NUL on the first ref.
+
+ smart_reply = PKT-LINE("# service=$servicename" LF)
+ ref_list
+ "0000"
+ ref_list = empty_list | populated_list
+
+ empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)
+
+ non_empty_list = PKT-LINE(id SP name NUL cap_list LF)
+ *ref_record
+
+ cap_list = *(SP capability) SP
+ ref_record = any_ref | peeled_ref
+
+ any_ref = PKT-LINE(id SP name LF)
+ peeled_ref = PKT-LINE(id SP name LF)
+ PKT-LINE(id SP name "^{}" LF
+ id = 40*HEX
+
+ HEX = "0".."9" | "a".."f"
+ NL = <US-ASCII NUL, null (0)>
+ LF = <US-ASCII LF, linefeed (10)>
+ SP = <US-ASCII SP, horizontal-tab (9)>
Did you define what "populated_list" is?
+Smart Service git-upload-pack
+------------------------------
+This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
+Capability include-tag
+~~~~~~~~~~~~~~~~~~~~~~
+
+When packing an object that an annotated tag points at, include the
+tag object too. Clients can request this if they want to fetch
+tags, but don't know which tags they will need until after they
+receive the branch data. By enabling include-tag an entire call
+to upload-pack can be avoided.
+
I think you are avoiding an "extra" call; you would need one entire call
to upload-pack anyway for the primary transfer.
On 2009-10-09, Junio C Hamano [off-list ref] wrote:
quoted
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
In both cases - is it really necessary to forbid the use of 410 (Gone)?
--
Mr. Antti-Juhani Kaijanaho, Jyvaskyla, Finland
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:47:30
Hi,
On Fri, Oct 9, 2009 at 1:22 PM, Shawn O. Pearce [off-list ref] wrote:
+Smart Clients
+~~~~~~~~~~~~~
+
+HTTP clients that support the "smart" protocol (or both the
+"smart" and "dumb" protocols) MUST discover references by making
+a paramterized request for the info/refs file of the repository.
s/paramterized/parameterized/ -- missing 'e'.
--
Cheers,
Ray Chuan
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:33
Alex Blewitt [off-list ref] wrote:
Shawn O. Pearce <spearce <at> spearce.org> writes:
quoted
+URL Format
+----------
It's worth making clear here that $GIT_URL will be the path to the repository,
...
Thanks, noted.
quoted
HEX = [0-9a-f]
Is there any reason not to support A-F as well in the hex spec, even if they
SHOULD use a-f?
Consistency. I'd rather be strict and say HEX is [0-9a-f] and
demand that everyone try to standardize on the lower case form.
This may limit the appeal for some case-insensitive systems.
Given that this particular notation of HEX is *only* used within
the protocol body to describe SHA-1 IDs, it won't make it to the
file system as-is.
A conforming Git implementation would first validate that this is in
fact a SHA-1 ID, likely translate it into a binary representation
(that is collapse the 40 byte hex to a 20 byte binary), and then
reformat it as a file system path if its looking for a loose object.
It would also be good to document, like with the git daemon, whether all
repositories under a path are exported or only those that have the magic
setting in the config like git-daemon-export-ok.
This isn't something that matters to the protocol specification.
Its a server access control, not protocol detail.
Really, its an implementation detail of git-http-backend in git.git,
or of the RepositoryResolver and UploadPackFactory in JGit.
Therefore, its not going to be documented in this document.
Lastly, it would be good to clarify when the result of this GET/POST exchange
is a text-based (and encoded in UTF-8) vs when binary data is returned; we
don't want to get into the state where we're returning binary data and
pretending that it's UTF-8.
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:33
Jeff King [off-list ref] wrote:
On Thu, Oct 08, 2009 at 10:22:45PM -0700, Shawn O. Pearce wrote:
quoted
+Servers MUST NOT require HTTP cookies for the purposes of
+authentication or access control.
[...]
+Servers MUST NOT require HTTP cookies in order to function correctly.
Why not? I can grant that the current git implementation probably can't
handle it, but keep in mind this is talking about the protocol and not
the implementation.
Good point... this document is about trying to explain the common
functionality that everyone can agree on.
And I can see it being useful for sites like github
which already have a cookie-based login.
What I'm concerned about is using the cookie jar. My Mac OS X
laptop has 5 browsers installed, each with their own #@!*! cookie
jar: Safari, Opera, Firefox, Camino, Google Chrome. How the hell
is the git client going to be able to use those cookies in order
to interact with a website that requires cookie authentication?
Adapting the client to handle
this case would not be too difficult (it would just mean keeping cookie
state in a file between runs,
Saving our own cookie jar is easy, libcurl has some limited cookie
jar support already built in. We just have to enable it.
or even just pulling it out of the normal
browser's cookie store).
See above, I don't think this will be very easy.
And people whose client didn't do this would
simply get an "access denied" response code.
And then they will email git ML or ask on #git why their git client
can't speak to some random website... and its because they used
"lynx" or yet-another-browser whose cookie jar format we can't read.
Is there a technical reason not to allow it?
Not technical, but I want to reduce the amount of complexity that
a conforming client has to deal with to reduce support costs for
everyone involved.
I weakend the sections on cookies:
+ Authentication
+ --------------
....
+ Servers SHOULD NOT require HTTP cookies for the purposes of
+ authentication or access control.
and that's all we say on the matter. I took out the Servers MUST
NOT line under session state.
--
Shawn.
From: Jeff King <hidden> Date: 2016-06-15 22:47:33
On Thu, Oct 15, 2009 at 09:52:28AM -0700, Shawn O. Pearce wrote:
quoted
And I can see it being useful for sites like github
which already have a cookie-based login.
What I'm concerned about is using the cookie jar. My Mac OS X
laptop has 5 browsers installed, each with their own #@!*! cookie
jar: Safari, Opera, Firefox, Camino, Google Chrome. How the hell
is the git client going to be able to use those cookies in order
to interact with a website that requires cookie authentication?
Sure, it is obviously something that an implementation will have to deal
with. Either through manual configuration by the user or some
auto-detection magic that tries to cover every case (and I suspect if we
really wanted to do this, a patch to libcurl to handle different cookie
jar formats would probably be the best way to go).
But my main point was that it is an implementation issue, not a protocol
issue. The lines are a little blurry for us because there really aren't
very many git implementations, but I think your document is an attempt
to document just the protocol to allow interoperability between clients.
But I think you got my point:
Not technical, but I want to reduce the amount of complexity that
a conforming client has to deal with to reduce support costs for
everyone involved.
I weakend the sections on cookies:
+ Authentication
+ --------------
....
+ Servers SHOULD NOT require HTTP cookies for the purposes of
+ authentication or access control.
and that's all we say on the matter. I took out the Servers MUST
NOT line under session state.
I think this is a good compromise. It's not recommended at this point,
but there is no reason to disallow it if both sides can handle the
non-protocol part (i.e., storing and managing cookies). Thanks.
-Peff
From: "H. Peter Anvin" <hpa@zytor.com> Date: 2016-06-15 22:47:34
On 10/10/2009 03:12 AM, Antti-Juhani Kaijanaho wrote:
On 2009-10-09, Junio C Hamano [off-list ref] wrote:
quoted
quoted
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
In both cases - is it really necessary to forbid the use of 410 (Gone)?
410 means "we once had it, it's no longer here, no idea where it went."
It's a largely useless code...
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.
From: Mike Hommey <hidden> Date: 2016-06-15 22:47:34
On Thu, Oct 15, 2009 at 10:59:25PM -0700, H. Peter Anvin wrote:
On 10/10/2009 03:12 AM, Antti-Juhani Kaijanaho wrote:
quoted
On 2009-10-09, Junio C Hamano [off-list ref] wrote:
quoted
quoted
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
In both cases - is it really necessary to forbid the use of 410 (Gone)?
410 means "we once had it, it's no longer here, no idea where it went."
It's a largely useless code...
There is an additional meaning to it, that is "it will never ever
return". It thus has a stronger meaning than 404. Sadly, not even search
engine spiders consider it as a hint to not crawl there in the future...
Mike
From: Shawn O. Pearce <hidden> Date: 2016-06-15 22:47:34
Mike Hommey [off-list ref] wrote:
On Thu, Oct 15, 2009 at 10:59:25PM -0700, H. Peter Anvin wrote:
quoted
On 10/10/2009 03:12 AM, Antti-Juhani Kaijanaho wrote:
quoted
On 2009-10-09, Junio C Hamano [off-list ref] wrote:
quoted
quoted
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
In both cases - is it really necessary to forbid the use of 410 (Gone)?
My original text got taken a bit out of context here. I guess MUST
was too strong of a word. I more ment something like:
If there is no repository at $GIT_URL, the server MUST NOT respond
with '200 OK' and a valid info/refs response. A server SHOULD
respond with '404 Not Found', '410 Gone', or any other suitable
HTTP status code which does not imply the resource exists as
requested.
quoted
410 means "we once had it, it's no longer here, no idea where it went."
It's a largely useless code...
There is an additional meaning to it, that is "it will never ever
return". It thus has a stronger meaning than 404. Sadly, not even search
engine spiders consider it as a hint to not crawl there in the future...
I know. I broke a URL on a site back in Janurary, MSN keeps crawling
it anyway. F'king spiders.
--
Shawn.
On 2009-10-16, H. Peter Anvin [off-list ref] wrote:
410 means "we once had it, it's no longer here, no idea where it went."
It's a largely useless code...
That's not a reason to forbid it methinks. And I quite like the difference
between "oops, mistyped the URI" and "oops, that URI is no longer valid".
--
Mr. Antti-Juhani Kaijanaho, Jyvaskyla, Finland
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:48:34
Hi,
(I'm reviving this thread to complete the document. What I have right
now is available at my github repo; you can see it at
http://github.com/rctay/git/compare/git/next...feature/http-doc#files_bucket
.
An inlined patch should be sent in soon.)
On Fri, 09 Oct 2009 13:44:53 -0700
Junio C Hamano [off-list ref] wrote:
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================
...
+As a design feature smart clients can automatically upgrade "dumb"
+protocol URLs to smart URLs. This permits all users to have the
+same published URL, and the peers automatically select the most
+efficient transport available to them.
The first sentence feels backwards although the conclusion in the second
sentence is true. It is more like smart ones trying smart protocol first,
and downgrading to "dumb" after noticing that the server is not smart.
I think Shawn is trying to describe this from the persepective of the
client implementation - a "dumb" url is first constructed, then
"upgraded" to a "smart" one.
quoted
+Authentication
+--------------
...
+Clients SHOULD support Basic authentication as described by RFC 2616.
+Servers SHOULD support Basic authentication by relying upon the
+HTTP server placed in front of the Git server software.
It is perfectly fine to make it a requirement for a server to support the
Basic authentication, but should you make it a requirement that the
support is done by a specific implementation, i.e. "by relying upon..."?
I think the term "Server" in this document is implied as an amalgam of
the HTTP server, and a CGI script/program ("Git server software"). I
think what Shawn meant was for Basic authentication to be implemented
at the server layer, not in the CGI scripts/program.
quoted
+Session State
+-------------
...
+retained and managed by the client process. This permits simple
+round-robin load-balancing on the server side, without needing to
+worry about state mangement.
s/mangement/management/;
Done.
quoted
+pkt-line Format
+---------------
...
+Examples (as C-style strings):
+
+ pkt-line actual value
+ ---------------------------------
+ "0006a\n" "a\n"
+ "0005a" "a"
+ "000bfoobar\n" "foobar\n"
+ "0004" ""
+
+A pkt-line with a length of 0 ("0000") is a special case and MUST
+be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
This section has been purged; we already have this in
Documentation/technical/protocol-common.txt.
quoted
+General Request Processing
+--------------------------
+
+Except where noted, all standard HTTP behavior SHOULD be assumed
+by both client and server. This includes (but is not necessarily
+limited to):
+
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
Proposed re-wording:
If there is no repository at $GIT_URL, or the resource pointed to by a
location containing $GIT_URL does not exist, the server MUST NOT respond
with '200 OK' response. A server SHOULD respond with
'404 Not Found', '410 Gone', or any other suitable HTTP status code
which does not imply the resource exists as requested.
(The 'valid info/refs response' part has been dropped.)
quoted
+Dumb Clients
+~~~~~~~~~~~~
+
+HTTP clients that only support the "dumb" protocol MUST discover
+references by making a request for the special info/refs file of
+the repository.
+
+Dumb HTTP clients MUST NOT include search/query parameters when
+fetching the info/refs file. (That is, '?' must not appear in the
+requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
Done.
quoted
+ C: GET $GIT_URL/info/refs HTTP/1.0
+
+ S: 200 OK
...
+When examining the response clients SHOULD only examine the HTTP
+status code. Valid responses are '200 OK', or '304 Not Modified'.
Isn't 401 ("Ah, I was given a wrong URL") and 403 ("Ok, I do not have an
access to this repository") also valid?
I think "valid" for the client means "ok, continue processing
normally".
quoted
+The returned content is a UNIX formatted text file describing
+each ref and its known value. The file SHOULD be sorted by name
+according to the C locale ordering. The file SHOULD NOT include
+the default ref named 'HEAD'.
I know you said "known" to imply "concurrent operations may change it
while the server is serving this client", but it feels rather awkward.
TODO
quoted
+Smart Server Response
+^^^^^^^^^^^^^^^^^^^^^
+
+Smart servers MUST respond with the smart server reply format.
+If the server does not recognize the requested service name, or the
+requested service name has been disabled by the server administrator,
+the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
Proposed rewording:
If the server does not recognize the requested service name, or the
requested service name has been disabled by the server administrator,
the server MUST respond with the '403 Forbidden' HTTP status code.
Otherwise, smart servers MUST respond with the smart server reply
format for the requested service name.
quoted
+The Content-Type MUST be 'application/x-$servicename-advertisement'.
+Clients SHOULD fall back to the dumb protocol if another content
+type is returned. When falling back to the dumb protocol clients
+SHOULD NOT make an additional request to $GIT_URL/info/refs, but
+instead SHOULD use the response already in hand. Clients MUST NOT
+continue if they do not support the dumb protocol.
The part I commented on (the beginning of Smart Server Response) was
written as a generic description, not specific to git-upload-pack service,
and the beginning of this paragraph also pretends to be a generic
description, but it is misleading. This is a specific instruction to the
clients that asked for git-upload-pack service and got a dumb server
response (if the above were talking about something other than upload-pack
service, there is no guarantee that "response already in hand" is useful
to talk to dumb servers).
I think "non_empty_list" was meant.
Ideally, ref advertisements should be in protocol-common.txt.
quoted
+Smart Service git-upload-pack
+------------------------------
+This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
Would
This service reads from the repository pointed to by $GIT_URL.
be an improvement?
quoted
+Capability include-tag
+~~~~~~~~~~~~~~~~~~~~~~
+
+When packing an object that an annotated tag points at, include the
+tag object too. Clients can request this if they want to fetch
+tags, but don't know which tags they will need until after they
+receive the branch data. By enabling include-tag an entire call
+to upload-pack can be avoided.
+
I think you are avoiding an "extra" call; you would need one entire call
to upload-pack anyway for the primary transfer.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:48:34
Hi,
(I'm reviving this thread to complete the document. What I have right
now is available at my github repo; you can see it at
http://github.com/rctay/git/compare/git/next...feature/http-doc#files_bucket
.
An inlined patch should be sent in soon.)
On Fri, 09 Oct 2009 13:44:53 -0700
Junio C Hamano [off-list ref] wrote:
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================
...
+As a design feature smart clients can automatically upgrade "dumb"
+protocol URLs to smart URLs. This permits all users to have the
+same published URL, and the peers automatically select the most
+efficient transport available to them.
The first sentence feels backwards although the conclusion in the second
sentence is true. It is more like smart ones trying smart protocol first,
and downgrading to "dumb" after noticing that the server is not smart.
I think Shawn is trying to describe this from the persepective of the
client implementation - a "dumb" url is first constructed, then
"upgraded" to a "smart" one.
quoted
+Authentication
+--------------
...
+Clients SHOULD support Basic authentication as described by RFC 2616.
+Servers SHOULD support Basic authentication by relying upon the
+HTTP server placed in front of the Git server software.
It is perfectly fine to make it a requirement for a server to support the
Basic authentication, but should you make it a requirement that the
support is done by a specific implementation, i.e. "by relying upon..."?
I think the term "Server" in this document is implied as an amalgam of
the HTTP server, and a CGI script/program ("Git server software"). I
think what Shawn meant was for Basic authentication to be implemented
at the server layer, not in the CGI scripts/program.
quoted
+Session State
+-------------
...
+retained and managed by the client process. This permits simple
+round-robin load-balancing on the server side, without needing to
+worry about state mangement.
s/mangement/management/;
Done.
quoted
+pkt-line Format
+---------------
...
+Examples (as C-style strings):
+
+ pkt-line actual value
+ ---------------------------------
+ "0006a\n" "a\n"
+ "0005a" "a"
+ "000bfoobar\n" "foobar\n"
+ "0004" ""
+
+A pkt-line with a length of 0 ("0000") is a special case and MUST
+be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
This section has been purged; we already have this in
Documentation/technical/protocol-common.txt.
quoted
+General Request Processing
+--------------------------
+
+Except where noted, all standard HTTP behavior SHOULD be assumed
+by both client and server. This includes (but is not necessarily
+limited to):
+
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
Proposed re-wording:
If there is no repository at $GIT_URL, or the resource pointed to by a
location containing $GIT_URL does not exist, the server MUST NOT respond
with '200 OK' response. A server SHOULD respond with
'404 Not Found', '410 Gone', or any other suitable HTTP status code
which does not imply the resource exists as requested.
(The 'valid info/refs response' part has been dropped.)
quoted
+Dumb Clients
+~~~~~~~~~~~~
+
+HTTP clients that only support the "dumb" protocol MUST discover
+references by making a request for the special info/refs file of
+the repository.
+
+Dumb HTTP clients MUST NOT include search/query parameters when
+fetching the info/refs file. (That is, '?' must not appear in the
+requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
Done.
quoted
+ C: GET $GIT_URL/info/refs HTTP/1.0
+
+ S: 200 OK
...
+When examining the response clients SHOULD only examine the HTTP
+status code. Valid responses are '200 OK', or '304 Not Modified'.
Isn't 401 ("Ah, I was given a wrong URL") and 403 ("Ok, I do not have an
access to this repository") also valid?
I think "valid" for the client means "ok, continue processing
normally".
quoted
+The returned content is a UNIX formatted text file describing
+each ref and its known value. The file SHOULD be sorted by name
+according to the C locale ordering. The file SHOULD NOT include
+the default ref named 'HEAD'.
I know you said "known" to imply "concurrent operations may change it
while the server is serving this client", but it feels rather awkward.
TODO
quoted
+Smart Server Response
+^^^^^^^^^^^^^^^^^^^^^
+
+Smart servers MUST respond with the smart server reply format.
+If the server does not recognize the requested service name, or the
+requested service name has been disabled by the server administrator,
+the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
Proposed rewording:
If the server does not recognize the requested service name, or the
requested service name has been disabled by the server administrator,
the server MUST respond with the '403 Forbidden' HTTP status code.
Otherwise, smart servers MUST respond with the smart server reply
format for the requested service name.
quoted
+The Content-Type MUST be 'application/x-$servicename-advertisement'.
+Clients SHOULD fall back to the dumb protocol if another content
+type is returned. When falling back to the dumb protocol clients
+SHOULD NOT make an additional request to $GIT_URL/info/refs, but
+instead SHOULD use the response already in hand. Clients MUST NOT
+continue if they do not support the dumb protocol.
The part I commented on (the beginning of Smart Server Response) was
written as a generic description, not specific to git-upload-pack service,
and the beginning of this paragraph also pretends to be a generic
description, but it is misleading. This is a specific instruction to the
clients that asked for git-upload-pack service and got a dumb server
response (if the above were talking about something other than upload-pack
service, there is no guarantee that "response already in hand" is useful
to talk to dumb servers).
I think "non_empty_list" was meant.
Ideally, ref advertisements should be in protocol-common.txt.
quoted
+Smart Service git-upload-pack
+------------------------------
+This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
Would
This service reads from the repository pointed to by $GIT_URL.
be an improvement?
quoted
+Capability include-tag
+~~~~~~~~~~~~~~~~~~~~~~
+
+When packing an object that an annotated tag points at, include the
+tag object too. Clients can request this if they want to fetch
+tags, but don't know which tags they will need until after they
+receive the branch data. By enabling include-tag an entire call
+to upload-pack can be avoided.
+
I think you are avoiding an "extra" call; you would need one entire call
to upload-pack anyway for the primary transfer.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:48:34
(sorry for the multiple copies - something wrong with my MUA. Had to
give it a kiss in the a** to get it working.)
(v2 - added back headers. My apologies again.)
Hi,
(I'm reviving this thread to complete the document. What I have right
now is available at my github repo; you can see it at
http://github.com/rctay/git/compare/git/next...feature/http-doc#files_bucket
.
An inlined patch should be sent in soon.)
On Fri, 09 Oct 2009 13:44:53 -0700
Junio C Hamano [off-list ref] wrote:
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================
...
+As a design feature smart clients can automatically upgrade "dumb"
+protocol URLs to smart URLs. This permits all users to have the
+same published URL, and the peers automatically select the most
+efficient transport available to them.
The first sentence feels backwards although the conclusion in the second
sentence is true. It is more like smart ones trying smart protocol first,
and downgrading to "dumb" after noticing that the server is not smart.
I think Shawn is trying to describe this from the persepective of the
client implementation - a "dumb" url is first constructed, then
"upgraded" to a "smart" one.
quoted
+Authentication
+--------------
...
+Clients SHOULD support Basic authentication as described by RFC 2616.
+Servers SHOULD support Basic authentication by relying upon the
+HTTP server placed in front of the Git server software.
It is perfectly fine to make it a requirement for a server to support the
Basic authentication, but should you make it a requirement that the
support is done by a specific implementation, i.e. "by relying upon..."?
I think the term "Server" in this document is implied as an amalgam of
the HTTP server, and a CGI script/program ("Git server software"). I
think what Shawn meant was for Basic authentication to be implemented
at the server layer, not in the CGI scripts/program.
quoted
+Session State
+-------------
...
+retained and managed by the client process. This permits simple
+round-robin load-balancing on the server side, without needing to
+worry about state mangement.
s/mangement/management/;
Done.
quoted
+pkt-line Format
+---------------
...
+Examples (as C-style strings):
+
+ pkt-line actual value
+ ---------------------------------
+ "0006a\n" "a\n"
+ "0005a" "a"
+ "000bfoobar\n" "foobar\n"
+ "0004" ""
+
+A pkt-line with a length of 0 ("0000") is a special case and MUST
+be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
This section has been purged; we already have this in
Documentation/technical/protocol-common.txt.
quoted
+General Request Processing
+--------------------------
+
+Except where noted, all standard HTTP behavior SHOULD be assumed
+by both client and server. This includes (but is not necessarily
+limited to):
+
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
Proposed re-wording:
If there is no repository at $GIT_URL, or the resource pointed to by a
location containing $GIT_URL does not exist, the server MUST NOT respond
with '200 OK' response. A server SHOULD respond with
'404 Not Found', '410 Gone', or any other suitable HTTP status code
which does not imply the resource exists as requested.
(The 'valid info/refs response' part has been dropped.)
quoted
+Dumb Clients
+~~~~~~~~~~~~
+
+HTTP clients that only support the "dumb" protocol MUST discover
+references by making a request for the special info/refs file of
+the repository.
+
+Dumb HTTP clients MUST NOT include search/query parameters when
+fetching the info/refs file. (That is, '?' must not appear in the
+requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
Done.
quoted
+ C: GET $GIT_URL/info/refs HTTP/1.0
+
+ S: 200 OK
...
+When examining the response clients SHOULD only examine the HTTP
+status code. Valid responses are '200 OK', or '304 Not Modified'.
Isn't 401 ("Ah, I was given a wrong URL") and 403 ("Ok, I do not have an
access to this repository") also valid?
I think "valid" for the client means "ok, continue processing
normally".
quoted
+The returned content is a UNIX formatted text file describing
+each ref and its known value. The file SHOULD be sorted by name
+according to the C locale ordering. The file SHOULD NOT include
+the default ref named 'HEAD'.
I know you said "known" to imply "concurrent operations may change it
while the server is serving this client", but it feels rather awkward.
TODO
quoted
+Smart Server Response
+^^^^^^^^^^^^^^^^^^^^^
+
+Smart servers MUST respond with the smart server reply format.
+If the server does not recognize the requested service name, or the
+requested service name has been disabled by the server administrator,
+the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
Proposed rewording:
If the server does not recognize the requested service name, or the
requested service name has been disabled by the server administrator,
the server MUST respond with the '403 Forbidden' HTTP status code.
Otherwise, smart servers MUST respond with the smart server reply
format for the requested service name.
quoted
+The Content-Type MUST be 'application/x-$servicename-advertisement'.
+Clients SHOULD fall back to the dumb protocol if another content
+type is returned. When falling back to the dumb protocol clients
+SHOULD NOT make an additional request to $GIT_URL/info/refs, but
+instead SHOULD use the response already in hand. Clients MUST NOT
+continue if they do not support the dumb protocol.
The part I commented on (the beginning of Smart Server Response) was
written as a generic description, not specific to git-upload-pack service,
and the beginning of this paragraph also pretends to be a generic
description, but it is misleading. This is a specific instruction to the
clients that asked for git-upload-pack service and got a dumb server
response (if the above were talking about something other than upload-pack
service, there is no guarantee that "response already in hand" is useful
to talk to dumb servers).
I think "non_empty_list" was meant.
Ideally, ref advertisements should be in protocol-common.txt.
quoted
+Smart Service git-upload-pack
+------------------------------
+This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
Would
This service reads from the repository pointed to by $GIT_URL.
be an improvement?
quoted
+Capability include-tag
+~~~~~~~~~~~~~~~~~~~~~~
+
+When packing an object that an annotated tag points at, include the
+tag object too. Clients can request this if they want to fetch
+tags, but don't know which tags they will need until after they
+receive the branch data. By enabling include-tag an entire call
+to upload-pack can be avoided.
+
I think you are avoiding an "extra" call; you would need one entire call
to upload-pack anyway for the primary transfer.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:48:34
(sorry for the multiple copies - something wrong with my MUA. Had to
give it a kiss in the a** to get it working.)
(v2 - added back headers. My apologies again.)
Hi,
(I'm reviving this thread to complete the document. What I have right
now is available at my github repo; you can see it at
http://github.com/rctay/git/compare/git/next...feature/http-doc#files_bucket
.
An inlined patch should be sent in soon.)
On Fri, 09 Oct 2009 13:44:53 -0700
Junio C Hamano [off-list ref] wrote:
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================
...
+As a design feature smart clients can automatically upgrade "dumb"
+protocol URLs to smart URLs. This permits all users to have the
+same published URL, and the peers automatically select the most
+efficient transport available to them.
The first sentence feels backwards although the conclusion in the second
sentence is true. It is more like smart ones trying smart protocol first,
and downgrading to "dumb" after noticing that the server is not smart.
I think Shawn is trying to describe this from the persepective of the
client implementation - a "dumb" url is first constructed, then
"upgraded" to a "smart" one.
quoted
+Authentication
+--------------
...
+Clients SHOULD support Basic authentication as described by RFC 2616.
+Servers SHOULD support Basic authentication by relying upon the
+HTTP server placed in front of the Git server software.
It is perfectly fine to make it a requirement for a server to support the
Basic authentication, but should you make it a requirement that the
support is done by a specific implementation, i.e. "by relying upon..."?
I think the term "Server" in this document is implied as an amalgam of
the HTTP server, and a CGI script/program ("Git server software"). I
think what Shawn meant was for Basic authentication to be implemented
at the server layer, not in the CGI scripts/program.
quoted
+Session State
+-------------
...
+retained and managed by the client process. This permits simple
+round-robin load-balancing on the server side, without needing to
+worry about state mangement.
s/mangement/management/;
Done.
quoted
+pkt-line Format
+---------------
...
+Examples (as C-style strings):
+
+ pkt-line actual value
+ ---------------------------------
+ "0006a\n" "a\n"
+ "0005a" "a"
+ "000bfoobar\n" "foobar\n"
+ "0004" ""
+
+A pkt-line with a length of 0 ("0000") is a special case and MUST
+be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
This section has been purged; we already have this in
Documentation/technical/protocol-common.txt.
quoted
+General Request Processing
+--------------------------
+
+Except where noted, all standard HTTP behavior SHOULD be assumed
+by both client and server. This includes (but is not necessarily
+limited to):
+
+If there is no repository at $GIT_URL, the server MUST respond with
+the '404 Not Found' HTTP status code.
We may also want to add
If there is no object at $GIT_URL/some/path, the server MUST respond
with the '404 Not Found' HTTP status code.
to help dumb clients.
Proposed re-wording:
If there is no repository at $GIT_URL, or the resource pointed to by a
location containing $GIT_URL does not exist, the server MUST NOT respond
with '200 OK' response. A server SHOULD respond with
'404 Not Found', '410 Gone', or any other suitable HTTP status code
which does not imply the resource exists as requested.
(The 'valid info/refs response' part has been dropped.)
quoted
+Dumb Clients
+~~~~~~~~~~~~
+
+HTTP clients that only support the "dumb" protocol MUST discover
+references by making a request for the special info/refs file of
+the repository.
+
+Dumb HTTP clients MUST NOT include search/query parameters when
+fetching the info/refs file. (That is, '?' must not appear in the
+requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
Done.
quoted
+ C: GET $GIT_URL/info/refs HTTP/1.0
+
+ S: 200 OK
...
+When examining the response clients SHOULD only examine the HTTP
+status code. Valid responses are '200 OK', or '304 Not Modified'.
Isn't 401 ("Ah, I was given a wrong URL") and 403 ("Ok, I do not have an
access to this repository") also valid?
I think "valid" for the client means "ok, continue processing
normally".
quoted
+The returned content is a UNIX formatted text file describing
+each ref and its known value. The file SHOULD be sorted by name
+according to the C locale ordering. The file SHOULD NOT include
+the default ref named 'HEAD'.
I know you said "known" to imply "concurrent operations may change it
while the server is serving this client", but it feels rather awkward.
TODO
quoted
+Smart Server Response
+^^^^^^^^^^^^^^^^^^^^^
+
+Smart servers MUST respond with the smart server reply format.
+If the server does not recognize the requested service name, or the
+requested service name has been disabled by the server administrator,
+the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
Proposed rewording:
If the server does not recognize the requested service name, or the
requested service name has been disabled by the server administrator,
the server MUST respond with the '403 Forbidden' HTTP status code.
Otherwise, smart servers MUST respond with the smart server reply
format for the requested service name.
quoted
+The Content-Type MUST be 'application/x-$servicename-advertisement'.
+Clients SHOULD fall back to the dumb protocol if another content
+type is returned. When falling back to the dumb protocol clients
+SHOULD NOT make an additional request to $GIT_URL/info/refs, but
+instead SHOULD use the response already in hand. Clients MUST NOT
+continue if they do not support the dumb protocol.
The part I commented on (the beginning of Smart Server Response) was
written as a generic description, not specific to git-upload-pack service,
and the beginning of this paragraph also pretends to be a generic
description, but it is misleading. This is a specific instruction to the
clients that asked for git-upload-pack service and got a dumb server
response (if the above were talking about something other than upload-pack
service, there is no guarantee that "response already in hand" is useful
to talk to dumb servers).
I think "non_empty_list" was meant.
Ideally, ref advertisements should be in protocol-common.txt.
quoted
+Smart Service git-upload-pack
+------------------------------
+This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
Would
This service reads from the repository pointed to by $GIT_URL.
be an improvement?
quoted
+Capability include-tag
+~~~~~~~~~~~~~~~~~~~~~~
+
+When packing an object that an annotated tag points at, include the
+tag object too. Clients can request this if they want to fetch
+tags, but don't know which tags they will need until after they
+receive the branch data. By enabling include-tag an entire call
+to upload-pack can be avoided.
+
I think you are avoiding an "extra" call; you would need one entire call
to upload-pack anyway for the primary transfer.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
This patch series are the changes based on the discussion on Shawn's
original text [1]. Some of them are minor, while some may potentially
change behaviour; see below for a classification of the changes.
Hopefully they can be examined by the git contributors here.
An earlier iteration of this patch series [2], including additional
changes by Nguyen [3], had been merged in 36d8020 (Merge branch
'sp/doc-smart-http', Aug 30). Since that iteration, the changes have
been corrected and consolidated. Effort has also been made to provide
the context for the changes; hopefully it helps with the review.
[1] http://mid.gmane.org/[off-list ref]
[2] https://github.com/rctay/git/blob/rc/http-doc/v1/p/Documentation/technical/http-protocol.txt
[3] http://mid.gmane.org/[off-list ref]
(For convenience, a diff against 36d8020 is included at the end of this
message; it is in word-diff form, hopefully for better clarity of the
changes.)
Given that an earlier iteration had already been merged, perhaps that
could be replaced with merge -Xtheirs (just throwing ideas, my git-fu is
not that strong). This would make the changes on the original RFC
available eg. via git-blame, which may be helpful for implementations
made based on the original RFC, especially since these "early"
implementations may now be in violation of the recently-included copy of
the spec.
The patches have been grouped based on their "safeness" (with regard to
potentially changing the protocol spec), with a bias towards caution, as
follows:
Trivial changes (eg formatting, style):
[PATCH 01/14] Document the HTTP transport protocol
[PATCH 02/14] normalize indentation with protcol-common.txt
[PATCH 03/14] capitalize key words according to RFC 2119
[PATCH 04/14] normalize rules with RFC 5234
[PATCH 05/14] drop rules, etc. common to the pack protocol
[PATCH 10/14] fix example request/responses
[PATCH 13/14] shift dumb server response details
Rewords based on discussions that have been settled, or seem safe:
[PATCH 07/14] weaken specification over cookies for authentication
[PATCH 09/14] reduce ambiguity over '?' in $GIT_URL for dumb clients
[PATCH 11/14] be clearer in place of 'remote repository' phrase
Potentially behaviour-changes, may need of discussion:
[PATCH 06/14] reword behaviour on missing repository or objects
[PATCH 08/14] mention different variations around $GIT_URL
[PATCH 12/14] reduce confusion over smart server response behaviour
[PATCH 14/14] mention effect of "allow-tip-sha1-in-want" capability
Full, ordered listing:
[PATCH 01/14] Document the HTTP transport protocol
[PATCH 02/14] normalize indentation with protcol-common.txt
[PATCH 03/14] capitalize key words according to RFC 2119
[PATCH 04/14] normalize rules with RFC 5234
[PATCH 05/14] drop rules, etc. common to the pack protocol
[PATCH 06/14] reword behaviour on missing repository or objects
[PATCH 07/14] weaken specification over cookies for authentication
[PATCH 08/14] mention different variations around $GIT_URL
[PATCH 09/14] reduce ambiguity over '?' in $GIT_URL for dumb clients
[PATCH 10/14] fix example request/responses
[PATCH 11/14] be clearer in place of 'remote repository' phrase
[PATCH 12/14] reduce confusion over smart server response behaviour
[PATCH 13/14] shift dumb server response details
[PATCH 14/14] mention effect of "allow-tip-sha1-in-want" capability
This patch series is queued at:
https://github.com/rctay/git/commits/rc/http-doc/v2/q
--
1.8.4.rc4.527.g303b16c
output of
$ git diff -b --word-diff 36d8020 -- Documentation/technical/http-protocol.txt
@@ -11,6 +11,10 @@ protocol URLs to smart URLs. This permits all users to have the
same published URL, and the peers automatically select the most
efficient transport available to them.
{+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL+}
{+NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and+}
{+"OPTIONAL" in this document are to be interpreted as described in+}
{+RFC 2119.+}
URL Format
----------
@@ -33,16 +37,13 @@ An example of a dumb client requesting for a loose object: $GIT_URL: http://example.com:8080/git/repo.git URL request: http://example.com:8080/git/repo.git/objects/d0/49f6c27a2244e12041955e262a404c7faba355
@@ -103,9 +104,10 @@ Except where noted, all standard HTTP behavior SHOULD be assumed
by both client and server. This includes (but is not necessarily
limited to):
If there is no repository at $GIT_URL, [-or-]{+the server MUST NOT respond with+}
{+'200 OK' and a valid info/refs response. Also, if+} the resource pointed
to by a location matching $GIT_URL does not exist, the server MUST NOT
respond with '200 [-OK' response.-]{+OK'.+} A server SHOULD respond with
'404 Not Found', '410 Gone', or any other suitable HTTP status code
which does not imply the resource exists as requested.
@@ -114,12 +116,12 @@ permitted, the server MUST respond with the '403 Forbidden' HTTP
status code.
Servers SHOULD support both HTTP 1.0 and HTTP 1.1.
Servers SHOULD support chunked encoding for both
request and response bodies.
Clients SHOULD support both HTTP 1.0 and HTTP 1.1.
Clients SHOULD support chunked encoding for both
request and response bodies.
Servers MAY return ETag and/or Last-Modified headers.
@@ -149,40 +151,16 @@ references by making a request for the special info/refs file of
the repository.
Dumb HTTP clients MUST make a GET request to $GIT_URL/info/refs,
without any search/query parameters. {+E.g.+}
C: GET $GIT_URL/info/refs HTTP/1.0
S: 200 OK
S:
S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 [-refs/heads/maint-]{+refs/heads/maint\n+}
S: d049f6c27a2244e12041955e262a404c7faba355 [-refs/heads/master-]{+refs/heads/master\n+}
S: 2cb58b79488a98d2721cea644875a8dd0026b115 [-refs/tags/v1.0-]{+refs/tags/v1.0\n+}
S: a3c2e2402b99163d1d59756e5f207ae21cccba4c [-refs/tags/v1.0^{}-]
[-The Content-Type of the returned info/refs entity SHOULD be-]
[-"text/plain; charset=utf-8", but MAY be any content type.-]
[-Clients MUST NOT attempt to validate the returned Content-Type.-]
[-Dumb servers MUST NOT return a return type starting with-]
[-"application/x-git-".-]
[-Cache-Control headers MAY be returned to disable caching of the-]
[-returned entity.-]
[-When examining the response clients SHOULD only examine the HTTP-]
[-status code. Valid responses are '200 OK', or '304 Not Modified'.-]
[-The returned content is a UNIX formatted text file describing-]
[-each ref and its known value. The file SHOULD be sorted by name-]
[-according to the C locale ordering. The file SHOULD NOT include-]
[-the default ref named 'HEAD'.-]
[- info_refs = *( ref_record )-]
[- ref_record = any_ref / peeled_ref-]
[- any_ref = obj-id HTAB refname LF-]
[- peeled_ref = obj-id HTAB refname LF-]
[- obj-id HTAB refname "^{}" LF-]{+refs/tags/v1.0^{}\n+}
Smart Clients
~~~~~~~~~~~~~
@@ -196,15 +174,20 @@ The request MUST contain exactly one query parameter,
name the client wishes to contact to complete the operation.
The request MUST NOT contain additional query parameters.
{+TODO: "exactly" one query parameter may be too strict; see the catch-all+}
{+gateway $GIT_URL for an example where more than one parameter is passed.+}
{+In fact, the http client implementation in Git can handle similar+}
{+$GIT_URLs, and thus may pass more than parameter to the server.+}
C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
dumb server reply:
S: 200 OK
S:
S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 [-refs/heads/maint-]{+refs/heads/maint\n+}
S: d049f6c27a2244e12041955e262a404c7faba355 [-refs/heads/master-]{+refs/heads/master\n+}
S: 2cb58b79488a98d2721cea644875a8dd0026b115 [-refs/tags/v1.0-]{+refs/tags/v1.0\n+}
S: a3c2e2402b99163d1d59756e5f207ae21cccba4c [-refs/tags/v1.0^{}-]{+refs/tags/v1.0^{}\n+}
smart server reply:
S: 200 OK
@@ -216,13 +199,35 @@ The request MUST NOT contain additional query parameters. S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n {+S: 0000+}
Dumb Server Response
^^^^^^^^^^^^^^^^^^^^
Dumb servers MUST respond with the dumb server reply format.
[-See-]{+The Content-Type of+} the [-prior section under dumb clients for-]{+returned info/refs entity SHOULD be+}
{+"text/plain; charset=utf-8", but MAY be any content type.+}
{+Clients MUST NOT attempt to validate the returned Content-Type.+}
{+Dumb servers MUST NOT return+} a [-more detailed-]
[-description-]{+return type starting with+}
{+"application/x-git-".+}
{+Cache-Control headers MAY be returned to disable caching+} of the
[-dumb server response.-]{+returned entity.+}
{+When examining the response clients SHOULD only examine the HTTP+}
{+status code. Valid responses are '200 OK', or '304 Not Modified'.+}
{+The returned content is a UNIX formatted text file describing+}
{+each ref and its known value. The file SHOULD be sorted by name+}
{+according to the C locale ordering. The file SHOULD NOT include+}
{+the default ref named 'HEAD'.+}
{+ info_refs = *( ref_record )+}
{+ ref_record = any_ref / peeled_ref+}
{+ any_ref = obj-id HTAB refname LF+}
{+ peeled_ref = obj-id HTAB refname LF+}
{+ obj-id HTAB refname "^{}" LF+}
Smart Server Response
^^^^^^^^^^^^^^^^^^^^^
@@ -268,23 +273,7 @@ named 'HEAD' as the first ref. The stream MUST include capability
@@ -394,7 +383,7 @@ The computation to select the minimal pack proceeds as follows emptied C_PENDING it SHOULD include a "done" command to let the server know it won't proceed: C: [-0009done-]{+0009done\n+} (s) Parse the git-upload-pack request:
Smart Service git-receive-pack
------------------------------
This service [-reads from-]{+modifies+} the repository pointed to by $GIT_URL.
Clients MUST first perform ref discovery with
'$GIT_URL/info/refs?service=git-receive-pack'.
@@ -458,7 +447,7 @@ Clients MUST first perform ref discovery with C: POST $GIT_URL/git-receive-pack HTTP/1.0 C: Content-Type: application/x-git-receive-pack-request C: C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 [-refs/heads/maint\0 report-status-]{+refs/heads/maint\0report-status+} C: 0000 C: PACK....
@@ -11,6 +11,10 @@ protocol URLs to smart URLs. This permits all users to have the same published URL, and the peers automatically select the most efficient transport available to them.+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL+NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and+"OPTIONAL" in this document are to be interpreted as described in+RFC 2119. URL Format ----------
@@ -29,7 +33,7 @@ supplied $GIT_URL string. Clients MUST strip a trailing '/', if present, from the user supplied $GIT_URL string to prevent empty path tokens ('//') from appearing-in any URL sent to a server. Compatible clients must expand+in any URL sent to a server. Compatible clients MUST expand '$GIT_URL/info/refs' as 'foo/info/refs' and not 'foo//info/refs'.
@@ -66,7 +70,7 @@ Session State ------------- The Git over HTTP protocol (much like HTTP itself) is stateless-from the perspective of the HTTP server side. All state must be+from the perspective of the HTTP server side. All state MUST be retained and managed by the client process. This permits simple round-robin load-balancing on the server side, without needing to worry about state management.
@@ -158,7 +162,7 @@ references by making a request for the special info/refs file of the repository. Dumb HTTP clients MUST NOT include search/query parameters when-fetching the info/refs file. (That is, '?' must not appear in the+fetching the info/refs file. (That is, '?' MUST NOT appear in the requested URL.) C: GET $GIT_URL/info/refs HTTP/1.0
@@ -390,7 +394,7 @@ The computation to select the minimal pack proceeds as follows (c) Start a queue, C_PENDING, ordered by commit time (popping newest first). Add all client refs. When a commit is popped from- the queue its parents should be automatically inserted back.+ the queue its parents SHOULD be automatically inserted back. Commits MUST only enter the queue once. one compute step:
@@ -431,7 +435,7 @@ The computation to select the minimal pack proceeds as follows If the client has sent 256 HAVE commits and has not yet received one of those back from S_COMMON, or the client has- emptied C_PENDING it should include a "done" command to let+ emptied C_PENDING it SHOULD include a "done" command to let the server know it won't proceed: C: 0009done
@@ -470,7 +474,7 @@ TODO: Document the pack based response The returned stream is the side-band-64k protocol supported by the git-upload-pack service, and the pack is embedded into- stream 1. Progress messages from the server side may appear+ stream 1. Progress messages from the server side MAY appear in stream 2. Here a "closed set of objects" is defined to have at least
@@ -538,5 +542,6 @@ References ---------- link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)]+link:http://www.ietf.org/rfc/rfc2119.txt[RFC 2119: Key words for use in RFCs to Indicate Requirement Levels] link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Drop LF, SP which are defined in RFC 5234.
Replace HT with HTAB (also defined in the RFC).
Use '/' instead of '|', as the RFC does.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 26 +++++++++-----------------
1 file changed, 9 insertions(+), 17 deletions(-)
@@ -192,16 +192,13 @@ according to the C locale ordering. The file SHOULD NOT include the default ref named 'HEAD'. info_refs = *( ref_record )- ref_record = any_ref | peeled_ref+ ref_record = any_ref / peeled_ref- any_ref = id HT name LF- peeled_ref = id HT name LF- id HT name "^{}" LF+ any_ref = id HTAB name LF+ peeled_ref = id HTAB name LF+ id HTAB name "^{}" LF id = 40*HEX- HEX = "0".."9" | "a".."f"- LF = <US-ASCII LF, linefeed (10)>- HT = <US-ASCII HT, horizontal-tab (9)> Smart Clients ~~~~~~~~~~~~~
@@ -288,7 +285,7 @@ declarations behind a NUL on the first ref. smart_reply = PKT-LINE("# service=$servicename" LF) ref_list "0000"- ref_list = empty_list | non_empty_list+ ref_list = empty_list / non_empty_list empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)
@@ -296,18 +293,13 @@ declarations behind a NUL on the first ref. *ref_record cap_list = *(SP capability) SP- ref_record = any_ref | peeled_ref+ ref_record = any_ref / peeled_ref any_ref = PKT-LINE(id SP name LF) peeled_ref = PKT-LINE(id SP name LF) PKT-LINE(id SP name "^{}" LF id = 40*HEX- HEX = "0".."9" | "a".."f"- NL = <US-ASCII NUL, null (0)>- LF = <US-ASCII LF, linefeed (10)>- SP = <US-ASCII SP, horizontal-tab (9)>- Smart Service git-upload-pack ------------------------------
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Based on
From: Alex Blewitt [off-list ref]
Message-ID: [off-list ref]
Shawn O. Pearce <spearce <at> spearce.org> writes:
> +URL Format
> +----------
> +
> +URLs for Git repositories accessed by HTTP use the standard HTTP
> +URL syntax documented by RFC 1738, so they are of the form:
> +
> + http://<host>:<port>/<path>
> +
> +Within this documentation the placeholder $GIT_URL will stand for
> +the http:// repository URL entered by the end-user.
It's worth making clear here that $GIT_URL will be the path to the repository,
rather than necessarily just the host upon which the server sits. Perhaps
including an example, like http://example:8080/repos/example.git
would make it clearer that there can be a path (and so leading to
a request like http://example:8080/repos/example.git/info/refs?service=...
It's also worth clarifying, therefore, that multiple repositories can be served
by the same process (as with the git server today) by using different path(s).
And for those that are interested in submodules, it's worth confirming that
http://example/repos/master.git/child.git/info/refs?service= will ensure
that the repository is the 'child' git rather than anything else.
The submodule example (/master.git/child.git) seems potentially
confusing - it suggests a setup where the server has a route to a git
repo (child.git) with a parent path containing another git repo
(master.git). It is excluded lest we be seen as encouraging such
mind-boggling setups.
While providing an example $GIT_URL containing a '?' (the catch-all
gateway one), also mention a possible contradiction between the
exactly-one-param requirement and the http client implementation in Git.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
@@ -22,15 +22,28 @@ URL Format URLs for Git repositories accessed by HTTP use the standard HTTP URL syntax documented by RFC 1738, so they are of the form:- http://<host>:<port>/<path>+ http://<host>:<port>/<path>?<searchpart> Within this documentation the placeholder $GIT_URL will stand for the http:// repository URL entered by the end-user.-Both the "smart" and "dumb" HTTP protocols used by Git operate+Servers SHOULD handle all requests to locations matching $GIT_URL, as+both the "smart" and "dumb" HTTP protocols used by Git operate by appending additional path components onto the end of the user supplied $GIT_URL string.+An example of a dumb client requesting for a loose object:++ $GIT_URL: http://example.com:8080/git/repo.git+ URL request: http://example.com:8080/git/repo.git/objects/d0/49f6c27a2244e12041955e262a404c7faba355++An example of a smart request to a catch-all gateway (notice how the+'service' parameter is passed with '&', since a '?' was detected in+$GIT_URL):++ $GIT_URL: http://example.com/daemon.cgi?svc=git&q=+ URL request: http://example.com/daemon.cgi?svc=git&q=/info/refs&service=git-receive-pack+ Clients MUST strip a trailing '/', if present, from the user supplied $GIT_URL string to prevent empty path tokens ('//') from appearing in any URL sent to a server. Compatible clients MUST expand
@@ -186,6 +199,11 @@ The request MUST contain exactly one query parameter, name the client wishes to contact to complete the operation. The request MUST NOT contain additional query parameters.+TODO: "exactly" one query parameter may be too strict; see the catch-all+gateway $GIT_URL for an example where more than one parameter is passed.+In fact, the http client implementation in Git can handle similar+$GIT_URLs, and thus may pass more than parameter to the server.+ C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 dumb server reply:
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
From: "Shawn O. Pearce" <redacted>
Signed-off-by: Tay Ray Chuan <redacted>
--
To Shawn: sign-off-by needed.
Based on:
From: "Shawn O. Pearce" [off-list ref]
Message-ID: [off-list ref]
Mike Hommey [off-list ref] wrote:
> On Thu, Oct 15, 2009 at 10:59:25PM -0700, H. Peter Anvin wrote:
> > On 10/10/2009 03:12 AM, Antti-Juhani Kaijanaho wrote:
> > > On 2009-10-09, Junio C Hamano [off-list ref] wrote:
> > >>> +If there is no repository at $GIT_URL, the server MUST respond with
> > >>> +the '404 Not Found' HTTP status code.
> > >>
> > >> We may also want to add
> > >>
> > >> If there is no object at $GIT_URL/some/path, the server MUST respond
> > >> with the '404 Not Found' HTTP status code.
> > >>
> > >> to help dumb clients.
> > >
> > > In both cases - is it really necessary to forbid the use of 410 (Gone)?
My original text got taken a bit out of context here. I guess MUST
was too strong of a word. I more ment something like:
If there is no repository at $GIT_URL, the server MUST NOT respond
with '200 OK' and a valid info/refs response. A server SHOULD
respond with '404 Not Found', '410 Gone', or any other suitable
HTTP status code which does not imply the resource exists as
requested.
In addition, address behaviour on missing objects, as suggested by
Junio. His text (see quote in above excerpt) was not used, in favour of
a more general treatment (locations matching $GIT_URL, not just
objects).
---
Documentation/technical/http-protocol.txt | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
@@ -91,8 +91,12 @@ Except where noted, all standard HTTP behavior SHOULD be assumed by both client and server. This includes (but is not necessarily limited to):-If there is no repository at $GIT_URL, the server MUST respond with-the '404 Not Found' HTTP status code.+If there is no repository at $GIT_URL, the server MUST NOT respond with+'200 OK' and a valid info/refs response. Also, if the resource pointed+to by a location matching $GIT_URL does not exist, the server MUST NOT+respond with '200 OK'. A server SHOULD respond with+'404 Not Found', '410 Gone', or any other suitable HTTP status code+which does not imply the resource exists as requested. If there is a repository at $GIT_URL, but access is not currently permitted, the server MUST respond with the '403 Forbidden' HTTP
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
From: "Shawn O. Pearce" <redacted>
Signed-off-by: Tay Ray Chuan <redacted>
--
To Shawn: sign-off-by needed.
Based on the discussion in
[off-list ref],
[off-list ref] (patch),
[off-list ref] (agreement)
From: "Shawn O. Pearce" [off-list ref]
Message-ID: [off-list ref]
I weakend the sections on cookies:
+ Authentication
+ --------------
....
+ Servers SHOULD NOT require HTTP cookies for the purposes of
+ authentication or access control.
and that's all we say on the matter. I took out the Servers MUST
NOT line under session state.
---
Documentation/technical/http-protocol.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -52,7 +52,7 @@ Clients SHOULD support Basic authentication as described by RFC 2616. Servers SHOULD support Basic authentication by relying upon the HTTP server placed in front of the Git server software.-Servers MUST NOT require HTTP cookies for the purposes of+Servers SHOULD NOT require HTTP cookies for the purposes of authentication or access control. Clients and servers MAY support other common forms of HTTP based
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
From: Junio C Hamano <redacted>
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
Signed-off-by: Tay Ray Chuan <redacted>
--
Notes:
- said "request to" instead of Junio's "request against", for
consistency with the rest of the document.
- said "E.g." instead of "I.e." since it's an example request and
response
Based on:
From: Junio C Hamano [off-list ref]
Message-ID: [off-list ref]
> +Dumb Clients
> +~~~~~~~~~~~~
> +
> +HTTP clients that only support the "dumb" protocol MUST discover
> +references by making a request for the special info/refs file of
> +the repository.
> +
> +Dumb HTTP clients MUST NOT include search/query parameters when
> +fetching the info/refs file. (That is, '?' must not appear in the
> +requested URL.)
It is unclear if '?' can be part of $GIT_URL. E.g.
$ wget http://example.xz/serve.cgi?path=git.git/info/refs
$ git clone http://example.xz/serve.cgi?path=git.git
It might be clearer to just say
Dumb HTTP clients MUST make a GET request against $GIT_URL/info/refs,
without any search/query parameters. I.e.
C: GET $GIT_URL/info/refs HTTP/1.0
to also exclude methods other than GET.
---
Documentation/technical/http-protocol.txt | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
@@ -150,9 +150,8 @@ HTTP clients that only support the "dumb" protocol MUST discover references by making a request for the special info/refs file of the repository.-Dumb HTTP clients MUST NOT include search/query parameters when-fetching the info/refs file. (That is, '?' MUST NOT appear in the-requested URL.)+Dumb HTTP clients MUST make a GET request to $GIT_URL/info/refs,+without any search/query parameters. E.g. C: GET $GIT_URL/info/refs HTTP/1.0
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Add LF for responses.
For smart interactions, add pkt-line lengths and the flush-pkt (0000) line.
Drop the SP that followed NUL before capability list.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 35 ++++++++++++++++---------------
1 file changed, 18 insertions(+), 17 deletions(-)
@@ -157,10 +157,10 @@ without any search/query parameters. E.g. S: 200 OK S:- S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint- S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\n+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n The Content-Type of the returned info/refs entity SHOULD be "text/plain; charset=utf-8", but MAY be any content type.
@@ -208,21 +208,22 @@ $GIT_URLs, and thus may pass more than parameter to the server. dumb server reply: S: 200 OK S:- S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint- S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\n+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n smart server reply: S: 200 OK S: Content-Type: application/x-git-upload-pack-advertisement S: Cache-Control: no-cache S:- S: ....# service=git-upload-pack- S: ....95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0 multi_ack- S: ....d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: ....2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: ....a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}+ S: 001e# service=git-upload-pack\n+ S: 004895dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0multi_ack\n+ S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n+ S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n+ S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n+ S: 0000 Dumb Server Response ^^^^^^^^^^^^^^^^^^^^
@@ -286,8 +287,8 @@ Clients MUST first perform ref discovery with C: POST $GIT_URL/git-upload-pack HTTP/1.0 C: Content-Type: application/x-git-upload-pack-request C:- C: ....want 0a53e9ddeaddad63ad106860237bbf53411d11a7- C: ....have 441b40d833fdfa93eb2908e52742248faf0ee993+ C: 0032want 0a53e9ddeaddad63ad106860237bbf53411d11a7\n+ C: 0032have 441b40d833fdfa93eb2908e52742248faf0ee993\n C: 0000 S: 200 OK
@@ -383,7 +384,7 @@ The computation to select the minimal pack proceeds as follows emptied C_PENDING it SHOULD include a "done" command to let the server know it won't proceed:- C: 0009done+ C: 0009done\n (s) Parse the git-upload-pack request:
@@ -447,7 +448,7 @@ Clients MUST first perform ref discovery with C: POST $GIT_URL/git-receive-pack HTTP/1.0 C: Content-Type: application/x-git-receive-pack-request C:- C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status+ C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0report-status C: 0000 C: PACK....
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Based on:
From: Junio C Hamano [off-list ref]
Message-ID: [off-list ref]
> +Smart Service git-upload-pack
> +------------------------------
> +This service reads from the remote repository.
The wording "remote repository" felt confusing. I know it is "from the
repository served by the server", but if it were named without
"upload-pack", I might have mistaken that you are allowing to proxy a
request to access a third-party repository by this server. The same
comment applies to the git-receive-pack service.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
@@ -279,7 +279,7 @@ declarations behind a NUL on the first ref. Smart Service git-upload-pack -------------------------------This service reads from the remote repository.+This service reads from the repository pointed to by $GIT_URL. Clients MUST first perform ref discovery with '$GIT_URL/info/refs?service=git-upload-pack'.
@@ -440,7 +440,7 @@ TODO: Document parsing response Smart Service git-receive-pack -------------------------------This service modifies the remote repository.+This service modifies the repository pointed to by $GIT_URL. Clients MUST first perform ref discovery with '$GIT_URL/info/refs?service=git-receive-pack'.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Shift details like ABNF from the client section to server section. This
is in line with the smart analogue.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 49 +++++++++++++++----------------
1 file changed, 23 insertions(+), 26 deletions(-)
@@ -162,30 +162,6 @@ without any search/query parameters. E.g. S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n-The Content-Type of the returned info/refs entity SHOULD be-"text/plain; charset=utf-8", but MAY be any content type.-Clients MUST NOT attempt to validate the returned Content-Type.-Dumb servers MUST NOT return a return type starting with-"application/x-git-".--Cache-Control headers MAY be returned to disable caching of the-returned entity.--When examining the response clients SHOULD only examine the HTTP-status code. Valid responses are '200 OK', or '304 Not Modified'.--The returned content is a UNIX formatted text file describing-each ref and its known value. The file SHOULD be sorted by name-according to the C locale ordering. The file SHOULD NOT include-the default ref named 'HEAD'.-- info_refs = *( ref_record )- ref_record = any_ref / peeled_ref-- any_ref = obj-id HTAB refname LF- peeled_ref = obj-id HTAB refname LF- obj-id HTAB refname "^{}" LF- Smart Clients ~~~~~~~~~~~~~
@@ -229,8 +205,29 @@ Dumb Server Response ^^^^^^^^^^^^^^^^^^^^ Dumb servers MUST respond with the dumb server reply format.-See the prior section under dumb clients for a more detailed-description of the dumb server response.+The Content-Type of the returned info/refs entity SHOULD be+"text/plain; charset=utf-8", but MAY be any content type.+Clients MUST NOT attempt to validate the returned Content-Type.+Dumb servers MUST NOT return a return type starting with+"application/x-git-".++Cache-Control headers MAY be returned to disable caching of the+returned entity.++When examining the response clients SHOULD only examine the HTTP+status code. Valid responses are '200 OK', or '304 Not Modified'.++The returned content is a UNIX formatted text file describing+each ref and its known value. The file SHOULD be sorted by name+according to the C locale ordering. The file SHOULD NOT include+the default ref named 'HEAD'.++ info_refs = *( ref_record )+ ref_record = any_ref / peeled_ref++ any_ref = obj-id HTAB refname LF+ peeled_ref = obj-id HTAB refname LF+ obj-id HTAB refname "^{}" LF Smart Server Response ^^^^^^^^^^^^^^^^^^^^^
@@ -304,7 +304,8 @@ Servers SHOULD support all capabilities defined here. Clients MUST send at least one 'want' command in the request body. Clients MUST NOT reference an id in a 'want' command which did not-appear in the response obtained through ref discovery.+appear in the response obtained through ref discovery unless the+server advertises capability "allow-tip-sha1-in-want". compute_request = want_list have_list
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
From: "Shawn O. Pearce" <redacted>
Signed-off-by: Shawn O. Pearce <redacted>
Signed-off-by: Tay Ray Chuan <redacted>
--
This is the original
[off-list ref]
with some minor changes, as follows:
- fix mis-spelling 'paramterized'
- fix mis-spelling 'mangement' (spotted by Junio)
- fix missing ABNF reference for smart replies (spotted by Sverre, Junio)
---
Documentation/technical/http-protocol.txt | 542 ++++++++++++++++++++++++++++++
1 file changed, 542 insertions(+)
create mode 100644 Documentation/technical/http-protocol.txt
@@ -0,0 +1,542 @@+HTTP transfer protocols+=======================++Git supports two HTTP based transfer protocols. A "dumb" protocol+which requires only a standard HTTP server on the server end of the+connection, and a "smart" protocol which requires a Git aware CGI+(or server module). This document describes both protocols.++As a design feature smart clients can automatically upgrade "dumb"+protocol URLs to smart URLs. This permits all users to have the+same published URL, and the peers automatically select the most+efficient transport available to them.+++URL Format+----------++URLs for Git repositories accessed by HTTP use the standard HTTP+URL syntax documented by RFC 1738, so they are of the form:++ http://<host>:<port>/<path>++Within this documentation the placeholder $GIT_URL will stand for+the http:// repository URL entered by the end-user.++Both the "smart" and "dumb" HTTP protocols used by Git operate+by appending additional path components onto the end of the user+supplied $GIT_URL string.++Clients MUST strip a trailing '/', if present, from the user supplied+$GIT_URL string to prevent empty path tokens ('//') from appearing+in any URL sent to a server. Compatible clients must expand+'$GIT_URL/info/refs' as 'foo/info/refs' and not 'foo//info/refs'.+++Authentication+--------------++Standard HTTP authentication is used if authentication is required+to access a repository, and MAY be configured and enforced by the+HTTP server software.++Because Git repositories are accessed by standard path components+server administrators MAY use directory based permissions within+their HTTP server to control repository access.++Clients SHOULD support Basic authentication as described by RFC 2616.+Servers SHOULD support Basic authentication by relying upon the+HTTP server placed in front of the Git server software.++Servers MUST NOT require HTTP cookies for the purposes of+authentication or access control.++Clients and servers MAY support other common forms of HTTP based+authentication, such as Digest authentication.+++SSL+---++Clients and servers SHOULD support SSL, particularly to protect+passwords when relying on Basic HTTP authentication.+++Session State+-------------++The Git over HTTP protocol (much like HTTP itself) is stateless+from the perspective of the HTTP server side. All state must be+retained and managed by the client process. This permits simple+round-robin load-balancing on the server side, without needing to+worry about state management.++Clients MUST NOT require state management on the server side in+order to function correctly.++Servers MUST NOT require HTTP cookies in order to function correctly.+Clients MAY store and forward HTTP cookies during request processing+as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any+cookies sent by a client.+++pkt-line Format+---------------++Much (but not all) of the payload is described around pkt-lines.++A pkt-line is a variable length binary string. The first four bytes+of the line indicates the total length of the line, in hexadecimal.+The total length includes the 4 bytes used to denote the length.+A line SHOULD BE terminated by an LF, which if present MUST be+included in the total length.++A pkt-line MAY contain binary data, so implementors MUST ensure all+pkt-line parsing/formatting routines are 8-bit clean. The maximum+length of a pkt-line's data is 65532 bytes (65536 - 4).++Examples (as C-style strings):++ pkt-line actual value+ ---------------------------------+ "0006a\n" "a\n"+ "0005a" "a"+ "000bfoobar\n" "foobar\n"+ "0004" ""++A pkt-line with a length of 0 ("0000") is a special case and MUST+be treated as a message break or terminator in the payload.+++General Request Processing+--------------------------++Except where noted, all standard HTTP behavior SHOULD be assumed+by both client and server. This includes (but is not necessarily+limited to):++If there is no repository at $GIT_URL, the server MUST respond with+the '404 Not Found' HTTP status code.++If there is a repository at $GIT_URL, but access is not currently+permitted, the server MUST respond with the '403 Forbidden' HTTP+status code.++Servers SHOULD support both HTTP 1.0 and HTTP 1.1.+Servers SHOULD support chunked encoding for both+request and response bodies.++Clients SHOULD support both HTTP 1.0 and HTTP 1.1.+Clients SHOULD support chunked encoding for both+request and response bodies.++Servers MAY return ETag and/or Last-Modified headers.++Clients MAY revalidate cached entities by including If-Modified-Since+and/or If-None-Match request headers.++Servers MAY return '304 Not Modified' if the relevant headers appear+in the request and the entity has not changed. Clients MUST treat+'304 Not Modified' identical to '200 OK' by reusing the cached entity.++Clients MAY reuse a cached entity without revalidation if the+Cache-Control and/or Expires header permits caching. Clients and+servers MUST follow RFC 2616 for cache controls.+++Discovering References+----------------------++All HTTP clients MUST begin either a fetch or a push exchange by+discovering the references available on the remote repository.++Dumb Clients+~~~~~~~~~~~~++HTTP clients that only support the "dumb" protocol MUST discover+references by making a request for the special info/refs file of+the repository.++Dumb HTTP clients MUST NOT include search/query parameters when+fetching the info/refs file. (That is, '?' must not appear in the+requested URL.)++ C: GET $GIT_URL/info/refs HTTP/1.0++ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++The Content-Type of the returned info/refs entity SHOULD be+"text/plain; charset=utf-8", but MAY be any content type.+Clients MUST NOT attempt to validate the returned Content-Type.+Dumb servers MUST NOT return a return type starting with+"application/x-git-".++Cache-Control headers MAY be returned to disable caching of the+returned entity.++When examining the response clients SHOULD only examine the HTTP+status code. Valid responses are '200 OK', or '304 Not Modified'.++The returned content is a UNIX formatted text file describing+each ref and its known value. The file SHOULD be sorted by name+according to the C locale ordering. The file SHOULD NOT include+the default ref named 'HEAD'.++ info_refs = *( ref_record )+ ref_record = any_ref | peeled_ref++ any_ref = id HT name LF+ peeled_ref = id HT name LF+ id HT name "^{}" LF+ id = 40*HEX++ HEX = "0".."9" | "a".."f"+ LF = <US-ASCII LF, linefeed (10)>+ HT = <US-ASCII HT, horizontal-tab (9)>++Smart Clients+~~~~~~~~~~~~~++HTTP clients that support the "smart" protocol (or both the+"smart" and "dumb" protocols) MUST discover references by making+a parameterized request for the info/refs file of the repository.++The request MUST contain exactly one query parameter,+'service=$servicename', where $servicename MUST be the service+name the client wishes to contact to complete the operation.+The request MUST NOT contain additional query parameters.++ C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0++ dumb server reply:+ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++ smart server reply:+ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-advertisement+ S: Cache-Control: no-cache+ S:+ S: ....# service=git-upload-pack+ S: ....95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0 multi_ack+ S: ....d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: ....2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: ....a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++Dumb Server Response+^^^^^^^^^^^^^^^^^^^^+Dumb servers MUST respond with the dumb server reply format.++See the prior section under dumb clients for a more detailed+description of the dumb server response.++Smart Server Response+^^^^^^^^^^^^^^^^^^^^^+Smart servers MUST respond with the smart server reply format.++If the server does not recognize the requested service name, or the+requested service name has been disabled by the server administrator,+the server MUST respond with the '403 Forbidden' HTTP status code.++Cache-Control headers SHOULD be used to disable caching of the+returned entity.++The Content-Type MUST be 'application/x-$servicename-advertisement'.+Clients SHOULD fall back to the dumb protocol if another content+type is returned. When falling back to the dumb protocol clients+SHOULD NOT make an additional request to $GIT_URL/info/refs, but+instead SHOULD use the response already in hand. Clients MUST NOT+continue if they do not support the dumb protocol.++Clients MUST validate the status code is either '200 OK' or+'304 Not Modified'.++Clients MUST validate the first five bytes of the response entity+matches the regex "^[0-9a-f]{4}#". If this test fails, clients+MUST NOT continue.++Clients MUST parse the entire response as a sequence of pkt-line+records.++Clients MUST verify the first pkt-line is "# service=$servicename".+Servers MUST set $servicename to be the request parameter value.+Servers SHOULD include an LF at the end of this line.+Clients MUST ignore an LF at the end of the line.++Servers MUST terminate the response with the magic "0000" end+pkt-line marker.++The returned response is a pkt-line stream describing each ref and+its known value. The stream SHOULD be sorted by name according to+the C locale ordering. The stream SHOULD include the default ref+named 'HEAD' as the first ref. The stream MUST include capability+declarations behind a NUL on the first ref.++ smart_reply = PKT-LINE("# service=$servicename" LF)+ ref_list+ "0000"+ ref_list = empty_list | non_empty_list++ empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)++ non_empty_list = PKT-LINE(id SP name NUL cap_list LF)+ *ref_record++ cap_list = *(SP capability) SP+ ref_record = any_ref | peeled_ref++ any_ref = PKT-LINE(id SP name LF)+ peeled_ref = PKT-LINE(id SP name LF)+ PKT-LINE(id SP name "^{}" LF+ id = 40*HEX++ HEX = "0".."9" | "a".."f"+ NL = <US-ASCII NUL, null (0)>+ LF = <US-ASCII LF, linefeed (10)>+ SP = <US-ASCII SP, horizontal-tab (9)>+++Smart Service git-upload-pack+------------------------------+This service reads from the remote repository.++Clients MUST first perform ref discovery with+'$GIT_URL/info/refs?service=git-upload-pack'.++ C: POST $GIT_URL/git-upload-pack HTTP/1.0+ C: Content-Type: application/x-git-upload-pack-request+ C:+ C: ....want 0a53e9ddeaddad63ad106860237bbf53411d11a7+ C: ....have 441b40d833fdfa93eb2908e52742248faf0ee993+ C: 0000++ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-result+ S: Cache-Control: no-cache+ S:+ S: ....ACK %s, continue+ S: ....NAK++Clients MUST NOT reuse or revalidate a cached reponse.+Servers MUST include sufficient Cache-Control headers+to prevent caching of the response.++Servers SHOULD support all capabilities defined here.++Clients MUST send at least one 'want' command in the request body.+Clients MUST NOT reference an id in a 'want' command which did not+appear in the response obtained through ref discovery.++ compute_request = want_list+ have_list+ request_end+ request_end = "0000" | "done"++ want_list = PKT-LINE(want NUL cap_list LF)+ *(want_pkt)+ want_pkt = PKT-LINE(want LF)+ want = "want" SP id+ cap_list = *(SP capability) SP++ have_list = *PKT-LINE("have" SP id LF)++ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name++TODO: Document this further.+TODO: Don't use uppercase for variable names below.++Capability include-tag+~~~~~~~~~~~~~~~~~~~~~~++When packing an object that an annotated tag points at, include the+tag object too. Clients can request this if they want to fetch+tags, but don't know which tags they will need until after they+receive the branch data. By enabling include-tag an entire call+to upload-pack can be avoided.++Capability thin-pack+~~~~~~~~~~~~~~~~~~~~++When packing a deltified object the base is not included if the base+is reachable from an object listed in the COMMON set by the client.+This reduces the bandwidth required to transfer, but it does slightly+increase processing time for the client to save the pack to disk.++The Negotiation Algorithm+~~~~~~~~~~~~~~~~~~~~~~~~~+The computation to select the minimal pack proceeds as follows+(c = client, s = server):++ init step:+ (c) Use ref discovery to obtain the advertised refs.+ (c) Place any object seen into set ADVERTISED.++ (c) Build an empty set, COMMON, to hold the objects that are later+ determined to be on both ends.+ (c) Build a set, WANT, of the objects from ADVERTISED the client+ wants to fetch, based on what it saw during ref discovery.++ (c) Start a queue, C_PENDING, ordered by commit time (popping newest+ first). Add all client refs. When a commit is popped from+ the queue its parents should be automatically inserted back.+ Commits MUST only enter the queue once.++ one compute step:+ (c) Send one $GIT_URL/git-upload-pack request:++ C: 0032want <WANT #1>...............................+ C: 0032want <WANT #2>...............................+ ....+ C: 0032have <COMMON #1>.............................+ C: 0032have <COMMON #2>.............................+ ....+ C: 0032have <HAVE #1>...............................+ C: 0032have <HAVE #2>...............................+ ....+ C: 0000++ The stream is organized into "commands", with each command+ appearing by itself in a pkt-line. Within a command line+ the text leading up to the first space is the command name,+ and the remainder of the line to the first LF is the value.+ Command lines are terminated with an LF as the last byte of+ the pkt-line value.++ Commands MUST appear in the following order, if they appear+ at all in the request stream:++ * want+ * have++ The stream is terminated by a pkt-line flush ("0000").++ A single "want" or "have" command MUST have one hex formatted+ SHA-1 as its value. Multiple SHA-1s MUST be sent by sending+ multiple commands.++ The HAVE list is created by popping the first 32 commits+ from C_PENDING. Less can be supplied if C_PENDING empties.++ If the client has sent 256 HAVE commits and has not yet+ received one of those back from S_COMMON, or the client has+ emptied C_PENDING it should include a "done" command to let+ the server know it won't proceed:++ C: 0009done++ (s) Parse the git-upload-pack request:++ Verify all objects in WANT are directly reachable from refs.++ The server MAY walk backwards through history or through+ the reflog to permit slightly stale requests.++ If no WANT objects are received, send an error:++TODO: Define error if no want lines are requested.++ If any WANT object is not reachable, send an error:++TODO: Define error if an invalid want is requested.++ Create an empty list, S_COMMON.++ If 'have' was sent:++ Loop through the objects in the order supplied by the client.+ For each object, if the server has the object reachable from+ a ref, add it to S_COMMON. If a commit is added to S_COMMON,+ do not add any ancestors, even if they also appear in HAVE.++ (s) Send the git-upload-pack response:++ If the server has found a closed set of objects to pack or the+ request ends with "done", it replies with the pack.++TODO: Document the pack based response+ S: PACK...++ The returned stream is the side-band-64k protocol supported+ by the git-upload-pack service, and the pack is embedded into+ stream 1. Progress messages from the server side may appear+ in stream 2.++ Here a "closed set of objects" is defined to have at least+ one path from every WANT to at least one COMMON object.++ If the server needs more information, it replies with a+ status continue response:++TODO: Document the non-pack response++ (c) Parse the upload-pack response:++TODO: Document parsing response++ Do another compute step.+++Smart Service git-receive-pack+------------------------------+This service modifies the remote repository.++Clients MUST first perform ref discovery with+'$GIT_URL/info/refs?service=git-receive-pack'.++ C: POST $GIT_URL/git-receive-pack HTTP/1.0+ C: Content-Type: application/x-git-receive-pack-request+ C:+ C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status+ C: 0000+ C: PACK....++ S: 200 OK+ S: Content-Type: application/x-git-receive-pack-result+ S: Cache-Control: no-cache+ S:+ S: ....++Clients MUST NOT reuse or revalidate a cached reponse.+Servers MUST include sufficient Cache-Control headers+to prevent caching of the response.++Servers SHOULD support all capabilities defined here.++Clients MUST send at least one command in the request body.+Within the command portion of the request body clients SHOULD send+the id obtained through ref discovery as old_id.++ update_request = command_list+ "PACK" <binary data>++ command_list = PKT-LINE(command NUL cap_list LF)+ *(command_pkt)+ command_pkt = PKT-LINE(command LF)+ cap_list = *(SP capability) SP++ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name++TODO: Document this further.+++References+----------++link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)]+link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]+
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
The MUST and the following 'If' scenario may seem contradictory at first
glance; swap their order to alleviate this.
Also mention that the response should specifically be for the requested
service, for clarity's sake.
Based on:
From: Junio C Hamano [off-list ref]
Message-ID: [off-list ref]
> +Smart Server Response
> +^^^^^^^^^^^^^^^^^^^^^
> +
> +Smart servers MUST respond with the smart server reply format.
> +If the server does not recognize the requested service name, or the
> +requested service name has been disabled by the server administrator,
> +the server MUST respond with the '403 Forbidden' HTTP status code.
This is a bit confusing.
If you as a server administrator want to disable the smart upload-pack for
one repository (but not for other repositories), you would not be able to
force smart clients to fall back to the dumb protocol by giving "403" for
that repository.
Maybe in 2 years somebody smarter than us will have invented a more
efficient git-upload-pack-2 service, which is the only fetch protocol his
server supports other than dumb. If your v1 smart client asks for the
original git-upload-pack service and gets a "403", you won't be able to
fall back to "dumb".
The solution for such cases likely is to pretend as if you are a dumb
server for the smart request. That unfortunately means that the first
sentence is misleading, and the second sentence is also an inappropriate
advice.
Signed-off-by: Tay Ray Chuan <redacted>
---
Documentation/technical/http-protocol.txt | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
@@ -234,12 +234,13 @@ description of the dumb server response. Smart Server Response ^^^^^^^^^^^^^^^^^^^^^-Smart servers MUST respond with the smart server reply format.- If the server does not recognize the requested service name, or the requested service name has been disabled by the server administrator, the server MUST respond with the '403 Forbidden' HTTP status code.+Otherwise, smart servers MUST respond with the smart server reply+format for the requested service name.+ Cache-Control headers SHOULD be used to disable caching of the returned entity.
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Use obj-id in lieu of id (defined as 40*HEX).
Use zero-id in lieu of 40*"0".
Use refname in lieu of name (not defined).
Drop section on capabilities, since they are already available in
protocol-capabilities.txt.
Signed-off-by: Tay Ray Chuan <redacted>
--
pkt-line format section was dropped in response to Junio's comments:
From: Junio C Hamano [off-list ref]
Message-ID: [off-list ref]
> +pkt-line Format
> +---------------
> ...
> +Examples (as C-style strings):
> +
> + pkt-line actual value
> + ---------------------------------
> + "0006a\n" "a\n"
> + "0005a" "a"
> + "000bfoobar\n" "foobar\n"
> + "0004" ""
> +
> +A pkt-line with a length of 0 ("0000") is a special case and MUST
> +be treated as a message break or terminator in the payload.
Isn't this "MUST be" wrong?
It is not an advice to the implementors, but the protocol specification
itself defines what the flush packet means. IOW, "The author of this
specification, Shawn, MUST treat a flush packet as a message break or
terminator in the payload, when designing this protocol."
Capabilities and 'command' ABNF rules under git-upload-pack were
dropped by Nguyễn:
Message-ID: [off-list ref]
---
Documentation/technical/http-protocol.txt | 85 ++++---------------------------
1 file changed, 10 insertions(+), 75 deletions(-)
@@ -84,34 +84,6 @@ as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any cookies sent by a client.-pkt-line Format------------------Much (but not all) of the payload is described around pkt-lines.--A pkt-line is a variable length binary string. The first four bytes-of the line indicates the total length of the line, in hexadecimal.-The total length includes the 4 bytes used to denote the length.-A line SHOULD BE terminated by an LF, which if present MUST be-included in the total length.--A pkt-line MAY contain binary data, so implementors MUST ensure all-pkt-line parsing/formatting routines are 8-bit clean. The maximum-length of a pkt-line's data is 65532 bytes (65536 - 4).--Examples (as C-style strings):-- pkt-line actual value- ---------------------------------- "0006a\n" "a\n"- "0005a" "a"- "000bfoobar\n" "foobar\n"- "0004" ""--A pkt-line with a length of 0 ("0000") is a special case and MUST-be treated as a message break or terminator in the payload.-- General Request Processing --------------------------
@@ -194,11 +166,9 @@ the default ref named 'HEAD'. info_refs = *( ref_record ) ref_record = any_ref / peeled_ref- any_ref = id HTAB name LF- peeled_ref = id HTAB name LF- id HTAB name "^{}" LF- id = 40*HEX-+ any_ref = obj-id HTAB refname LF+ peeled_ref = obj-id HTAB refname LF+ obj-id HTAB refname "^{}" LF Smart Clients ~~~~~~~~~~~~~
@@ -283,23 +253,7 @@ named 'HEAD' as the first ref. The stream MUST include capability declarations behind a NUL on the first ref. smart_reply = PKT-LINE("# service=$servicename" LF)- ref_list- "0000"- ref_list = empty_list / non_empty_list-- empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)-- non_empty_list = PKT-LINE(id SP name NUL cap_list LF)- *ref_record-- cap_list = *(SP capability) SP- ref_record = any_ref / peeled_ref-- any_ref = PKT-LINE(id SP name LF)- peeled_ref = PKT-LINE(id SP name LF)- PKT-LINE(id SP name "^{}" LF- id = 40*HEX-+ advertised-refs Smart Service git-upload-pack ------------------------------
@@ -345,31 +299,9 @@ appear in the response obtained through ref discovery. have_list = *PKT-LINE("have" SP id LF)- command = create / delete / update- create = 40*"0" SP new_id SP name- delete = old_id SP 40*"0" SP name- update = old_id SP new_id SP name- TODO: Document this further. TODO: Don't use uppercase for variable names below.-Capability include-tag-~~~~~~~~~~~~~~~~~~~~~~--When packing an object that an annotated tag points at, include the-tag object too. Clients can request this if they want to fetch-tags, but don't know which tags they will need until after they-receive the branch data. By enabling include-tag an entire call-to upload-pack can be avoided.--Capability thin-pack-~~~~~~~~~~~~~~~~~~~~--When packing a deltified object the base is not included if the base-is reachable from an object listed in the COMMON set by the client.-This reduces the bandwidth required to transfer, but it does slightly-increase processing time for the client to save the pack to disk.- The Negotiation Algorithm ~~~~~~~~~~~~~~~~~~~~~~~~~ The computation to select the minimal pack proceeds as follows
@@ -536,4 +468,7 @@ References link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)] link:http://www.ietf.org/rfc/rfc2119.txt[RFC 2119: Key words for use in RFCs to Indicate Requirement Levels] link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]+link:technical/pack-protocol.txt+link:technical/protocol-common.txt+link:technical/protocol-capabilities.txt
From: Tay Ray Chuan <hidden> Date: 2016-06-15 22:58:42
Indent client/server query examples with 3 spaces.
Indent ABNF rules with 2 spaces.
Signed-off-by: Tay Ray Chuan <redacted>
--
This is in its own patch to minimize noise in diffs.
---
Documentation/technical/http-protocol.txt | 226 +++++++++++++++---------------
1 file changed, 113 insertions(+), 113 deletions(-)
@@ -161,14 +161,14 @@ Dumb HTTP clients MUST NOT include search/query parameters when fetching the info/refs file. (That is, '?' must not appear in the requested URL.)- C: GET $GIT_URL/info/refs HTTP/1.0+ C: GET $GIT_URL/info/refs HTTP/1.0- S: 200 OK- S:- S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint- S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}+ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{} The Content-Type of the returned info/refs entity SHOULD be "text/plain; charset=utf-8", but MAY be any content type.
@@ -187,17 +187,17 @@ each ref and its known value. The file SHOULD be sorted by name according to the C locale ordering. The file SHOULD NOT include the default ref named 'HEAD'.- info_refs = *( ref_record )- ref_record = any_ref | peeled_ref+ info_refs = *( ref_record )+ ref_record = any_ref | peeled_ref- any_ref = id HT name LF- peeled_ref = id HT name LF- id HT name "^{}" LF- id = 40*HEX+ any_ref = id HT name LF+ peeled_ref = id HT name LF+ id HT name "^{}" LF+ id = 40*HEX- HEX = "0".."9" | "a".."f"- LF = <US-ASCII LF, linefeed (10)>- HT = <US-ASCII HT, horizontal-tab (9)>+ HEX = "0".."9" | "a".."f"+ LF = <US-ASCII LF, linefeed (10)>+ HT = <US-ASCII HT, horizontal-tab (9)> Smart Clients ~~~~~~~~~~~~~
@@ -211,26 +211,26 @@ The request MUST contain exactly one query parameter, name the client wishes to contact to complete the operation. The request MUST NOT contain additional query parameters.- C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0-- dumb server reply:- S: 200 OK- S:- S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint- S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}-- smart server reply:- S: 200 OK- S: Content-Type: application/x-git-upload-pack-advertisement- S: Cache-Control: no-cache- S:- S: ....# service=git-upload-pack- S: ....95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0 multi_ack- S: ....d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master- S: ....2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0- S: ....a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}+ C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0++ dumb server reply:+ S: 200 OK+ S:+ S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint+ S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}++ smart server reply:+ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-advertisement+ S: Cache-Control: no-cache+ S:+ S: ....# service=git-upload-pack+ S: ....95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0 multi_ack+ S: ....d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master+ S: ....2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0+ S: ....a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{} Dumb Server Response ^^^^^^^^^^^^^^^^^^^^
@@ -281,28 +281,28 @@ the C locale ordering. The stream SHOULD include the default ref named 'HEAD' as the first ref. The stream MUST include capability declarations behind a NUL on the first ref.- smart_reply = PKT-LINE("# service=$servicename" LF)- ref_list- "0000"- ref_list = empty_list | non_empty_list+ smart_reply = PKT-LINE("# service=$servicename" LF)+ ref_list+ "0000"+ ref_list = empty_list | non_empty_list- empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)+ empty_list = PKT-LINE(id SP "capabilities^{}" NUL cap_list LF)- non_empty_list = PKT-LINE(id SP name NUL cap_list LF)- *ref_record+ non_empty_list = PKT-LINE(id SP name NUL cap_list LF)+ *ref_record- cap_list = *(SP capability) SP- ref_record = any_ref | peeled_ref+ cap_list = *(SP capability) SP+ ref_record = any_ref | peeled_ref- any_ref = PKT-LINE(id SP name LF)- peeled_ref = PKT-LINE(id SP name LF)- PKT-LINE(id SP name "^{}" LF- id = 40*HEX+ any_ref = PKT-LINE(id SP name LF)+ peeled_ref = PKT-LINE(id SP name LF)+ PKT-LINE(id SP name "^{}" LF+ id = 40*HEX- HEX = "0".."9" | "a".."f"- NL = <US-ASCII NUL, null (0)>- LF = <US-ASCII LF, linefeed (10)>- SP = <US-ASCII SP, horizontal-tab (9)>+ HEX = "0".."9" | "a".."f"+ NL = <US-ASCII NUL, null (0)>+ LF = <US-ASCII LF, linefeed (10)>+ SP = <US-ASCII SP, horizontal-tab (9)> Smart Service git-upload-pack
@@ -312,19 +312,19 @@ This service reads from the remote repository. Clients MUST first perform ref discovery with '$GIT_URL/info/refs?service=git-upload-pack'.- C: POST $GIT_URL/git-upload-pack HTTP/1.0- C: Content-Type: application/x-git-upload-pack-request- C:- C: ....want 0a53e9ddeaddad63ad106860237bbf53411d11a7- C: ....have 441b40d833fdfa93eb2908e52742248faf0ee993- C: 0000+ C: POST $GIT_URL/git-upload-pack HTTP/1.0+ C: Content-Type: application/x-git-upload-pack-request+ C:+ C: ....want 0a53e9ddeaddad63ad106860237bbf53411d11a7+ C: ....have 441b40d833fdfa93eb2908e52742248faf0ee993+ C: 0000- S: 200 OK- S: Content-Type: application/x-git-upload-pack-result- S: Cache-Control: no-cache- S:- S: ....ACK %s, continue- S: ....NAK+ S: 200 OK+ S: Content-Type: application/x-git-upload-pack-result+ S: Cache-Control: no-cache+ S:+ S: ....ACK %s, continue+ S: ....NAK Clients MUST NOT reuse or revalidate a cached reponse. Servers MUST include sufficient Cache-Control headers
@@ -336,23 +336,23 @@ Clients MUST send at least one 'want' command in the request body. Clients MUST NOT reference an id in a 'want' command which did not appear in the response obtained through ref discovery.- compute_request = want_list- have_list- request_end- request_end = "0000" | "done"+ compute_request = want_list+ have_list+ request_end+ request_end = "0000" | "done"- want_list = PKT-LINE(want NUL cap_list LF)- *(want_pkt)- want_pkt = PKT-LINE(want LF)- want = "want" SP id- cap_list = *(SP capability) SP+ want_list = PKT-LINE(want NUL cap_list LF)+ *(want_pkt)+ want_pkt = PKT-LINE(want LF)+ want = "want" SP id+ cap_list = *(SP capability) SP- have_list = *PKT-LINE("have" SP id LF)+ have_list = *PKT-LINE("have" SP id LF)- command = create | delete | update- create = 40*"0" SP new_id SP name- delete = old_id SP 40*"0" SP name- update = old_id SP new_id SP name+ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name TODO: Document this further. TODO: Don't use uppercase for variable names below.
@@ -396,16 +396,16 @@ The computation to select the minimal pack proceeds as follows one compute step: (c) Send one $GIT_URL/git-upload-pack request:- C: 0032want <WANT #1>...............................- C: 0032want <WANT #2>...............................- ....- C: 0032have <COMMON #1>.............................- C: 0032have <COMMON #2>.............................- ....- C: 0032have <HAVE #1>...............................- C: 0032have <HAVE #2>...............................- ....- C: 0000+ C: 0032want <WANT #1>...............................+ C: 0032want <WANT #2>...............................+ ....+ C: 0032have <COMMON #1>.............................+ C: 0032have <COMMON #2>.............................+ ....+ C: 0032have <HAVE #1>...............................+ C: 0032have <HAVE #2>...............................+ ....+ C: 0000 The stream is organized into "commands", with each command appearing by itself in a pkt-line. Within a command line
@@ -434,13 +434,13 @@ The computation to select the minimal pack proceeds as follows emptied C_PENDING it should include a "done" command to let the server know it won't proceed:- C: 0009done+ C: 0009done (s) Parse the git-upload-pack request: Verify all objects in WANT are directly reachable from refs.- The server MAY walk backwards through history or through+ The server MAY walk backwards through history or through the reflog to permit slightly stale requests. If no WANT objects are received, send an error:
@@ -466,7 +466,7 @@ TODO: Define error if an invalid want is requested. request ends with "done", it replies with the pack. TODO: Document the pack based response- S: PACK...+ S: PACK... The returned stream is the side-band-64k protocol supported by the git-upload-pack service, and the pack is embedded into
@@ -495,18 +495,18 @@ This service modifies the remote repository. Clients MUST first perform ref discovery with '$GIT_URL/info/refs?service=git-receive-pack'.- C: POST $GIT_URL/git-receive-pack HTTP/1.0- C: Content-Type: application/x-git-receive-pack-request- C:- C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status- C: 0000- C: PACK....+ C: POST $GIT_URL/git-receive-pack HTTP/1.0+ C: Content-Type: application/x-git-receive-pack-request+ C:+ C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status+ C: 0000+ C: PACK....- S: 200 OK- S: Content-Type: application/x-git-receive-pack-result- S: Cache-Control: no-cache- S:- S: ....+ S: 200 OK+ S: Content-Type: application/x-git-receive-pack-result+ S: Cache-Control: no-cache+ S:+ S: .... Clients MUST NOT reuse or revalidate a cached reponse. Servers MUST include sufficient Cache-Control headers
@@ -518,18 +518,18 @@ Clients MUST send at least one command in the request body. Within the command portion of the request body clients SHOULD send the id obtained through ref discovery as old_id.- update_request = command_list- "PACK" <binary data>+ update_request = command_list+ "PACK" <binary data>- command_list = PKT-LINE(command NUL cap_list LF)- *(command_pkt)- command_pkt = PKT-LINE(command LF)- cap_list = *(SP capability) SP+ command_list = PKT-LINE(command NUL cap_list LF)+ *(command_pkt)+ command_pkt = PKT-LINE(command LF)+ cap_list = *(SP capability) SP- command = create | delete | update- create = 40*"0" SP new_id SP name- delete = old_id SP 40*"0" SP name- update = old_id SP new_id SP name+ command = create | delete | update+ create = 40*"0" SP new_id SP name+ delete = old_id SP 40*"0" SP name+ update = old_id SP new_id SP name TODO: Document this further.