"Shawn O. Pearce" [off-list ref] writes:
When --stateless-rpc is passed as a command line parameter to
upload-pack or receive-pack the programs now assume they may
perform only a single read-write cycle with stdin and stdout.
This fits with the HTTP POST request processing model where a
program may read the request, write a response, and must exit.
When --advertise-refs is passed as a command line parameter only
the initial ref advertisement is output, and the program exits
immediately. This fits with the HTTP GET request model, where
no request content is received but a response must be produced.
Is the idea to first run with --advertise-refs to get the set of refs, and
then doing another, separate run with --stateless-rpc, assuming that the
refs the other end advertised does not change in the meantime, to conclude
the business?
I suspect that two separate invocations on a (supposedly) single
repository made back-to-back can produce an inconsistent response in
verious situations (e.g. somebody pushing in the middle, or the same
hostname served by more than one mirrors) and the other end can get
confused.
I do not think there is any way to avoid it, short of adding to the second
request some "cookie" that allows the second requestee to verify that the
request is based on what he would give to the requester if this request
were the first step of the request made to him, iow, his state did not
change in the middle since the first request was made (either to him or to
the equivalent mirror sitting next to himm).
I wouldn't worry too much about this if the only negative effect could be
that the requestor's second request may result in an error, but I am
wondering if this can be used to attack the requestee.
Junio C Hamano [off-list ref] wrote:
"Shawn O. Pearce" [off-list ref] writes:
quoted
When --stateless-rpc is passed [...]
When --advertise-refs is passed [...]
Is the idea to first run with --advertise-refs to get the set of refs, and
then doing another, separate run with --stateless-rpc, assuming that the
refs the other end advertised does not change in the meantime, to conclude
the business?
Yes.
I suspect that two separate invocations on a (supposedly) single
repository made back-to-back can produce an inconsistent response in
verious situations (e.g. somebody pushing in the middle, or the same
hostname served by more than one mirrors) and the other end can get
confused.
Yes, that can happen.
I do not think there is any way to avoid it, short of adding to the second
request some "cookie" that allows the second requestee to verify that the
request is based on what he would give to the requester if this request
were the first step of the request made to him, iow, his state did not
change in the middle since the first request was made (either to him or to
the equivalent mirror sitting next to himm).
There isn't a lot we can do, you are right.
One approach is to use an HMAC and sign each advertised SHA-1
during the initial --advertise-refs phase. Requesters then present
the SHA-1 and the HAMC signature in each "want" line, and the
--stateless-rpc phase validates the signatures to ensure they came
from a trusted party.
The major problem with this approach is the private key management.
All mirrors of that repository need to have a common private key
so they can generate and later verify that HMAC signature. This is
additional complexity, for perhaps not much gain.
A different approach is to have the --stateless-rpc phase validate
the want lines against its refs (like we do now), but if the validate
fails (no ref exactly matches) support walking backwards through the
last few records in the reflog, or through that ref's own history
for a few hundred commits, to see if the want SHA-1 appears in the
recent prior past.
Obviously when walking the reflog we would need to use a time bound,
e.g. "only examine last record if more recent than 5 minutes ago".
This way a force push to erase something will still erase it, but
a client who had just recently observed the prior SHA-1 can still
complete their fetch request, just as with native git:// they could
do that due to the prior SHA-1 being held in the git-upload-pack's
private memory.
Also, obviously when walking the commit history of a ref (to see
if the want'd SHA-1 is merged into one or more reachable refs)
we don't want to walk backwards too far, as it costs CPU time on
the server side, but we also don't want to walk too few commits.
E.g. pushes for me tend to be in the 20 commit range, while Linus
probably pushes a thousand commits or more in a single merge.
Finding the right balance may be tricky.
I wouldn't worry too much about this if the only negative effect could be
that the requestor's second request may result in an error, but I am
wondering if this can be used to attack the requestee.
I don't think it can be used to attack the server. The only impact
I can see is the client gets confused and gets an error response
from the server when the server aborts due to the invalid "want"
line sent during that 2nd (or any subsequent) request.
--
Shawn.
"Shawn O. Pearce" [off-list ref] writes:
One approach is to use an HMAC and sign each advertised SHA-1
during the initial --advertise-refs phase. Requesters then present
the SHA-1 and the HAMC signature in each "want" line, and the
--stateless-rpc phase validates the signatures to ensure they came
from a trusted party.
The major problem with this approach is the private key management.
All mirrors of that repository need to have a common private key
so they can generate and later verify that HMAC signature. This is
additional complexity, for perhaps not much gain.
I am not worried so much about malicious clients getting themselves
confused. One simple alternative would be to internally recreate the
response the requestee would send if it were the first phase request upon
receiving the request for the second phase---then you take an SHA-1 hash
of it. In the second phase request have the requestor include the SHA-1
hash of what it received in the response to its first phase request, and
the requestee can make sure they match. No per-ref signing nor secret key
management is necessary, and it would let the requestor retry if you allow
the response to the second phase request to be "your request is stale, try
again".
A different approach is to have the --stateless-rpc phase validate
the want lines against its refs (like we do now), but if the validate
fails (no ref exactly matches) support walking backwards through the
last few records in the reflog, or through that ref's own history
for a few hundred commits, to see if the want SHA-1 appears in the
recent prior past.
Obviously when walking the reflog we would need to use a time bound,
e.g. "only examine last record if more recent than 5 minutes ago".
I think that is probably too much complexity for too little gain. I think
detecting stale request and having requestor retry would be sufficient,
and validating the want lines as we already do would give the same level
of assurance as "check against the hash of first phase response" I
outlined above, and would be much simpler thus more robust.
Junio C Hamano [off-list ref] wrote:
I think that is probably too much complexity for too little gain. I think
detecting stale request and having requestor retry would be sufficient,
and validating the want lines as we already do would give the same level
of assurance as "check against the hash of first phase response" I
outlined above, and would be much simpler thus more robust.
Ack.
I think what we want here is to just add an "ERR invalid want"
message just before disconnecting when the client gives us a SHA-1
which we don't point to directly. Client implementations can choose
how to handle this error. They could retry from the beginning,
or they could abort.
--
Shawn.