From: Junio C Hamano <hidden> Date: 2011-09-13 16:45:44
[administrivia] This message is also Cc'ed to the kernel mailing list in
order to ask for opinions from members of one of the most important user
communities of Git, but people may want to drop the kernel list when
responding to this message to reduce the noise level over there. Thanks.
In the light of what happened to k.org recently, we've been discussing
things Git can do to help raising confidence levels perceived by the
general public on integrity of the source trees, especially for the kernel
community. As the article by Jonathan Corbet on lwn.net nicely described,
projects managed with Git are already pretty resistant from tampering, and
it is not my (nor anybody in the Git community's) intention to propose any
more unnecessary bureaucracy to the development process without merit.
There are two updates that may change the end user experience I would like
to ask your opinions on, both as the Git designer (emeritus?) and as the
top kernel developer.
1. Improved pull requests.
Currently a typical pull-request begins like this:
The following changes since commit f696543dad6c7ba27b0c4fab167a5687263a9ba0:
Flobar 2.4.3 (2011-09-13 12:34:56 +0900)
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ master
which is followed by the shortlog and expected diffstat. This tells you
where the requester based his work on in excruciating detail, but does not
tell you what you should expect to fetch, any more than "whatever happened
to be at the named branch when you happened to notice the request."
We have a tentative patch to add an extra line after the "URL branch" line
that is for your cut & paste that looks like:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ master
for you to fetch changes up to 5738c9c21e53356ab5020912116e7f82fd2d428f
I often see you respond to a pull request on the kernel mailing list with
"I see nothing new; forgot to push?", and having this extra line may also
help communication.
Would it be just an added and useless noise that you nor your requesters
would not care much about?
An alternative that I am considering is to let the requester say this
instead:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
without adding the extra line.
That is, to allow fetching the history up to an explicitly named commit
object. This would only involve a change to fetch-pack at the receiving
end; just match the commit object name given from the command line against
the ls-remote response and ask upload-pack to give the history leading to
it. The released versions of Git already will happily oblige, as long as
the commit object named in the request message still sits at the tip of
the intended branch.
Do you think it is worthwhile to pursue this alternative?
2. Signed pushes.
You tag official releases and release candidates with your GPG key, and
everybody who works within the kernel ecosystem trusts the history behind
the commits pointed by them, but there is no easy way to verify that
commits and merges between the last tagged commit and the tip of your
branch(es) are indeed from you, or if an intruder piled fake ones on top
of your commits (until you try to push again and discover that the history
does not fast-forward, that is).
We have been discussing an addition of "git push -s" to let people sign
their pushes (instead of having to sign every commit or add signed
tag). The implementation alternatives were being bikeshed but not of much
interest in this message, but the user experience would go like this:
* You push out your work with "git push -s";
* "git push" prepares a "push certificate" (it is meant to certify "these
are the commits I place at the tips of these refs"), which is a human
and machine readable text file in core, that may look like this:
Push-Certificate-Version: 0
Pusher: Junio C Hamano [off-list ref]
Update: 3793ac56b4c4f9bf0bddc306a0cec21118683728 refs/heads/master
Update: 12850bec0c24b529c9a9df6a95ad4bdeea39373e refs/heads/next
and asks you to GPG sign it. You only unlock your GPG key and the
command internally runs GPG, just like "tag -s".
* When "git push" finishes, the receiving end has this record in its
refs/notes/signed-push notes tree, together with your previous pushes
(as this is not a shared repository, it will record only your pushes).
The notes annnotate the commits named on the "Update:" lines above.
* People who want to verify commits that are not yet tagged near the tip
in their clone of your tree can fetch refs/notes/signed-push and run
$ git log --show-notes=signed-push --branches --not --tags
to see your push certificates as annotations on commits that are not
yet tagged. They can verify them using a tool (yet to be written) that
acts like "git tag --verify".
It is hoped that it would help downstream with warm and fuzzy assurances
that all commits including the ones that are not yet tagged are genuine
(disclaimer: my employer is among the "downstream" that wants to have that
warm and fuzzy assurance) if we can see these push certificates published
at your public repository.
A few questions.
* As a user, do you think "signed push" is a good idea, or is it merely
an unnecessary bureaucracy, having to sign all pushes?
* As a user, do you think it is a good thing that you could also verify
the commits you receive from the Git-managed repositories of your
lieutenants using this mechanism, or you wouldn't bother, perhaps
because you are applying many patches sent via unsigned e-mail from
Andrew anyway?
* If the answers to the above points are both "yes", do you think it
would make sense to also propagate the push certificates you obtain
from your lieutenants to your public repository when you make your
"push -s"? It will allow your downstream to follow the chain of trust
in one-go (if you are pulling from public places, they can fetch the
push certificates from your lieutenants themselves and merge them, so
this is merely a convenience feature) by simply fetching from the
refs/notes/signed-push notes tree from your public repository. Do you
think it is a useful and worthwhile thing to do?
From: Sam Vilain <hidden> Date: 2011-09-14 00:39:20
On 9/13/11 9:45 AM, Junio C Hamano wrote:
* You push out your work with "git push -s";
* "git push" prepares a "push certificate" (it is meant to certify "these
are the commits I place at the tips of these refs"), which is a human
and machine readable text file in core, that may look like this:
Push-Certificate-Version: 0
Pusher: Junio C Hamano[off-list ref]
Update: 3793ac56b4c4f9bf0bddc306a0cec21118683728 refs/heads/master
Update: 12850bec0c24b529c9a9df6a95ad4bdeea39373e refs/heads/next
and asks you to GPG sign it. You only unlock your GPG key and the
command internally runs GPG, just like "tag -s".
* When "git push" finishes, the receiving end has this record in its
refs/notes/signed-push notes tree, together with your previous pushes
(as this is not a shared repository, it will record only your pushes).
The notes annnotate the commits named on the "Update:" lines above.
If the push certificate also has the previous commit IDs for the changed
refs, then you actually have an audit log. Otherwise, it does not
certify the commit range they pushed.
This is an important prerequisite for a fully distributed, peer to peer
git. For this case it would also need something to distinguish which
repository is to be updated; such as a canonical repository URL (or list
of URLs), or just a short project name. A P2P protocol can then know
projects as (KEYID, projectname).
Sam
From: Andy Lutomirski <hidden> Date: 2011-09-14 19:35:26
On 09/13/2011 09:45 AM, Junio C Hamano wrote:
An alternative that I am considering is to let the requester say this
instead:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
without adding the extra line.
That is, to allow fetching the history up to an explicitly named commit
object. This would only involve a change to fetch-pack at the receiving
end; just match the commit object name given from the command line against
the ls-remote response and ask upload-pack to give the history leading to
it. The released versions of Git already will happily oblige, as long as
the commit object named in the request message still sits at the tip of
the intended branch.
I would love this feature on the pull/fetch interface, but for a
completely different reason. Sometimes I want to pull a particular
object (usually a commit, but sometimes just a tree or blob) from
*myself*, and having to stick it on a branch is annoying.
One use-case is when applying a patch in git's extended format. If I
know where it came from, I ought to be able to pull the blobs it depends
on to enable three-way merge. I think that this is essentially
impossible remotely right now.
Of course, merging with the result of the pull will result in terrible
automatically-generated messages, but it's easy to fix that up manually.
This is one thing that I think Mercurial handles better than git. (And
apologies for the noise if I've missed a way to do this with current
git. I've looked, but maybe I missed some magic way to do this.)
--Andy
From: Junio C Hamano <hidden> Date: 2016-06-15 22:52:01
This teaches "git fetch" (hence "git pull") to accept an explicit commit
object name in the LHS of the refspec, as long as the named commit is at
the tip of an advertised ref. E.g.
$ git pull origin 5738c9c21e53356ab5020912116e7f82fd2d428f
$ git fetch origin 5738c9c21e53356ab5020912116e7f82fd2d428f:refs/remotes/origin
would behave exactly as if you asked
$ git pull origin refs/heads/master
$ git fetch origin refs/heads/master:refs/remotes/origin
when the output from "git ls-remote origin" said the remote side has the
commit object whose name is 5738c9c21e53356ab5020912116e7f82fd2d428f at
the tip of refs/heads/master branch ref.
This does not allow asking for a random object that may or may not exist
in the repository (this has been a longstanding security feature).
Signed-off-by: Junio C Hamano <redacted>
---
remote.c | 25 +++++++++++++++++++++++--
1 files changed, 23 insertions(+), 2 deletions(-)
From: Junio C Hamano <hidden> Date: 2016-06-15 22:52:01
A typical pull-request begins like this:
The following changes since commit f696543dad6c7ba27b0c4fab167a5687263a9ba0:
Flobar 2.4.3 (2011-09-13 12:34:56 +0900)
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ master
which is followed by the shortlog and expected diffstat. This tells you
where the requester based his work on in excruciating detail, but does not
tell you what you should expect to fetch, any more than "whatever happened
to be at the named branch when you happened to notice the request."
Update the message slightly to say:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f ;# master
so that the line still can be cut&pasted after "git fetch" (or "git
pull"), to form a command line that looks like:
$ git <repository> <full commit object name> ;# branch
Signed-off-by: Junio C Hamano <redacted>
---
git-request-pull.sh | 2 +-
t/t5150-request-pull.sh | 11 +++++++----
2 files changed, 8 insertions(+), 5 deletions(-)
From: Junio C Hamano <hidden> Date: 2016-06-15 22:52:01
Here is an alternative approach to the earlier "request-pull" patch.
Junio C Hamano (2):
fetch: allow asking for an explicit commit object by name
request-pull: state exact commit object name
git-request-pull.sh | 2 +-
remote.c | 25 +++++++++++++++++++++++--
t/t5150-request-pull.sh | 11 +++++++----
3 files changed, 31 insertions(+), 7 deletions(-)
--
1.7.7.rc1.1.g1e5814
On Tue, Sep 13, 2011 at 17:31, Sam Vilain [off-list ref] wrote:
On 9/13/11 9:45 AM, Junio C Hamano wrote:
quoted
* "git push" prepares a "push certificate" (it is meant to certify "these
are the commits I place at the tips of these refs"), which is a human
and machine readable text file in core, that may look like this:
Push-Certificate-Version: 0
Pusher: Junio C Hamano[off-list ref]
Update: 3793ac56b4c4f9bf0bddc306a0cec21118683728 refs/heads/master
Update: 12850bec0c24b529c9a9df6a95ad4bdeea39373e refs/heads/next
If the push certificate also has the previous commit IDs for the changed
refs, then you actually have an audit log. Otherwise, it does not certify
the commit range they pushed.
Is that necessary? The range they are certifying is that commit, and
its entire ancestry. If the pusher doesn't trust his ancestry, why is
he working with it? Similar to an annotated tag. I make a signed
annotated tag, I am asserting that revision and its ancestry is
something I like as far as a project build goes. You don't need the
old revision to realize I like this commit.
If you want to get into the game of, maybe I push a branch, then
rewind it, and push something differently, and you want to be able to
verify that the 2nd push is the "right thing" and the 1st push should
be ignored, you can already see that by looking at the timestamp of
the push certificates (/me assumes there is a timestamp in there). If
you can create multiple signed pushes by yourself, using your GPG key,
within the same second, and they are conflicting... well, stop using
automated tools to create conflicting assertions as yourself. If you
are creating signed pushes on systems with clock skew, learn how to
configure NTP date.
This is an important prerequisite for a fully distributed, peer to peer git.
For this case it would also need something to distinguish which repository
is to be updated; such as a canonical repository URL (or list of URLs), or
just a short project name. A P2P protocol can then know projects as (KEYID,
projectname).
Why do we need a project name? Most Git based projects are uniquely
identified by the set of root commits they have. Why? Because most
root commits were created by different people, at different times,
with different commit messages, and different initial trees, resulting
in a unique commit SHA-1 for that root commit. Projects with more than
one root commit also disambiguate themselves from other projects that
maybe contain one of those roots (e.g. git.git vs. gitk).
If you wanted to identify a project on a P2P network, I think you
would want to do it based off the root commits, not some random name
people came up with and might try to publish forgeries under.
--
Shawn.
From: Sam Vilain <hidden> Date: 2016-06-15 22:52:02
On 9/13/11 5:39 PM, Shawn Pearce wrote:
quoted
If the push certificate also has the previous commit IDs for the changed
refs, then you actually have an audit log. Otherwise, it does not certify
the commit range they pushed.
Is that necessary? The range they are certifying is that commit, and
its entire ancestry. If the pusher doesn't trust his ancestry, why is
he working with it? Similar to an annotated tag. I make a signed
annotated tag, I am asserting that revision and its ancestry is
something I like as far as a project build goes. You don't need the
old revision to realize I like this commit.
Perhaps because they didn't notice what happened. Someone else pushed
to the server without a signed push somehow, and then they pulled,
pushed ... and now as far as you know, those commits are certified like
any other. Having this extra information, not much information, will
help figure out what happens in this sort of situation.
quoted
This is an important prerequisite for a fully distributed, peer to peer git.
For this case it would also need something to distinguish which repository
is to be updated; such as a canonical repository URL (or list of URLs), or
just a short project name. A P2P protocol can then know projects as (KEYID,
projectname).
Why do we need a project name? Most Git based projects are uniquely
identified by the set of root commits they have. Why? Because most
root commits were created by different people, at different times,
with different commit messages, and different initial trees, resulting
in a unique commit SHA-1 for that root commit. Projects with more than
one root commit also disambiguate themselves from other projects that
maybe contain one of those roots (e.g. git.git vs. gitk).
If you wanted to identify a project on a P2P network, I think you
would want to do it based off the root commits, not some random name
people came up with and might try to publish forgeries under.
Yes, this is true, but it also makes it a lot harder to figure out if
two projects are from the same real project, or whether they just shared
some history. In general, git repositories are partitioned by URL or
project, and so this makes a soft case for a distributed system to
partition itself by URL or project also.
Sam
On Wed, Sep 14, 2011 at 2:45 AM, Junio C Hamano [off-list ref] wrote:
1. Improved pull requests.
...
An alternative that I am considering is to let the requester say this
instead:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
without adding the extra line.
That is, to allow fetching the history up to an explicitly named commit
object. This would only involve a change to fetch-pack at the receiving
end; just match the commit object name given from the command line against
the ls-remote response and ask upload-pack to give the history leading to
it. The released versions of Git already will happily oblige, as long as
the commit object named in the request message still sits at the tip of
the intended branch.
Do you think it is worthwhile to pursue this alternative?
Stupid question, if we agree to go with signed push, can we also sign
pull requests and verify them when we pull? I suppose most of the
time, pulling can be done automatically by extracting pull url from
the request. This would make pull/push both signed.
BTW, there's a third way (rsync is obsolete) to carry changes away in
human-unreadable way: bundles. Should we also sign the bundles too (I
guess we could just do the same as in signed push).
--
Duy
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:52:02
Nguyen Thai Ngoc Duy wrote:
On Wed, Sep 14, 2011 at 2:45 AM, Junio C Hamano [off-list ref] wrote:
quoted
An alternative that I am considering is to let the requester say this
instead:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
[...]
Stupid question, if we agree to go with signed push, can we also sign
pull requests and verify them when we pull? I suppose most of the
time, pulling can be done automatically by extracting pull url from
the request. This would make pull/push both signed.
BTW, there's a third way (rsync is obsolete) to carry changes away in
human-unreadable way: bundles. Should we also sign the bundles too (I
guess we could just do the same as in signed push).
If I understand you correctly, then ordinary PGP email signing[1]
should work for that already. In your first example, the receiver can
make sure whatever process grabs a pull request verifies it, and in
the second example, the receiver checks the signature on her email
before saving a bundle and passing it to "git fetch".
[1] http://www.phildev.net/pgp/gpgmua.html
On Thu, Sep 15, 2011 at 7:05 AM, Jonathan Nieder [off-list ref] wrote:
Nguyen Thai Ngoc Duy wrote:
quoted
On Wed, Sep 14, 2011 at 2:45 AM, Junio C Hamano [off-list ref] wrote:
quoted
quoted
An alternative that I am considering is to let the requester say this
instead:
are available in the git repository at:
git://git.kernel.org/pub/flobar.git/ 5738c9c21e53356ab5020912116e7f82fd2d428f
[...]
quoted
Stupid question, if we agree to go with signed push, can we also sign
pull requests and verify them when we pull? I suppose most of the
time, pulling can be done automatically by extracting pull url from
the request. This would make pull/push both signed.
BTW, there's a third way (rsync is obsolete) to carry changes away in
human-unreadable way: bundles. Should we also sign the bundles too (I
guess we could just do the same as in signed push).
If I understand you correctly, then ordinary PGP email signing[1]
should work for that already. In your first example, the receiver can
make sure whatever process grabs a pull request verifies it, and in
the second example, the receiver checks the signature on her email
before saving a bundle and passing it to "git fetch".
Yes, I think we can do that already. It's just more convenient to
teach "git fetch/pull" to take pull requests and automatically verify
them. Some repositories may also want to enforce signing and we can do
that by setting config file and fetch/pull refuses if pull requests
are not signed. We can also store the sign as git notes, just like in
git-push (extra work if it has to be done manually).
From: Jeff King <hidden> Date: 2016-06-15 22:52:02
On Thu, Sep 15, 2011 at 08:42:40AM +1000, Nguyen Thai Ngoc Duy wrote:
Yes, I think we can do that already. It's just more convenient to
teach "git fetch/pull" to take pull requests and automatically verify
them. Some repositories may also want to enforce signing and we can do
that by setting config file and fetch/pull refuses if pull requests
are not signed. We can also store the sign as git notes, just like in
git-push (extra work if it has to be done manually).
Isn't there a human element in the verification? I.e., I see a pull
request, and we can computationally verify that it is signed by some
key. Now assuming GPG's web of trust works, that binds that key to an
email address and a real name. But how is that bound to the repository
you are actually fetching from (or more appropriately, that the commits
mentioned are appropriate to be pulled)?
That is a policy that the human must decide upon seeing "Oh, a pull
request from developer X; I should pull that into my local branch Y",
and which they do implicitly when they manually run the pull command
mentioned in the email.
Another way to think of it is that verifying the identity of the sender
(which GPG does) is only one step. You also need an ACL saying that the
sender is worth pulling from.
So either:
1. The human is still in the loop, in which case having git-pull
verify the sender's identity hasn't really done anything (because
probably their MUA already told them it was really from the
purported sender, and then they made the ACL decision in their head
before deciding to pull from you).
2. The human is not in the loop, and nothing is checking that ACL.
-Peff