Duy Nguyen [off-list ref] writes:
On Tue, Mar 3, 2015 at 1:12 AM, Joey Hess [off-list ref] wrote:
quoted
I support this proposal, as someone who no longer releases tarballs
of my software, when I can possibly avoid it. I have worried about
signed tags / commits only being a SHA1 break away from useless.
As to the implementation, checksumming the collection of raw objects is
certainly superior to tar. Colin had suggested sorting the objects by
checksum, but I don't think that is necessary. Just stream the commit
object, then its tree object, followed by the content of each object
listed in the tree, recursing into subtrees as necessary. That will be a
stable stream for a given commit, or tree.
It could be simplified a bit by using ls-tree -r (so you basically
have a single big tree). Then hash commit, ls-tree -r output and all
blobs pointed by ls-tree in listed order.
What problem are you trying to solve here, though, by deliberately
deviating what Git internally used to store these objects? If it is
OK to ignore the tree boundary, then you probably do not even need
trees in this secondary hash for validation in the first place.
For example, you can hash a stream:
<commit object contents> +
N * (<pathname> + NUL + <blob object contents>)
as long as the <pathname>s are sorted in a predictable order (like
in "the index order") in the output. That would be even simpler (I
am not saying it is necessarily better, and by inference neither is
your "simplification").
I was about to suggest another alternative.
Pretend as if Git internally used SHA-512 (or whatever hash you
want to use) instead of SHA-1, compute the object names that
way. Recompute the contents of a tree object is by replacing
the 20-byte SHA-1 field in it with a field with whatever
necessary length to hold the longer object names of elements in
the tree.
But then a realization hit me: what new value will be placed in the
"parent " field in the commit object? You cannot have SHA-512
variant of commit object name without recomputing the whole history.
Now, if the final objective is to replace signature of tarballs,
does it matter to cover the commit object, or is it sufficient to
cover the tree contents?
Among the ideas raised so far, I like what Joey suggested, combined
with "each should have '<type> <length>NUL' header" from Sam Vilain
the best. That is, hash the stream:
"commit <length>" NUL + <commit object contents> +
"tree <length>" NUL + <top level tree contents> +
... list the entries in the order you would find by
... some defined traversal order people can agree on.
with whatever the preferred strong hash function of the age.
On Tue, Mar 3, 2015 at 6:44 AM, Junio C Hamano [off-list ref] wrote:
Duy Nguyen [off-list ref] writes:
quoted
On Tue, Mar 3, 2015 at 1:12 AM, Joey Hess [off-list ref] wrote:
quoted
I support this proposal, as someone who no longer releases tarballs
of my software, when I can possibly avoid it. I have worried about
signed tags / commits only being a SHA1 break away from useless.
As to the implementation, checksumming the collection of raw objects is
certainly superior to tar. Colin had suggested sorting the objects by
checksum, but I don't think that is necessary. Just stream the commit
object, then its tree object, followed by the content of each object
listed in the tree, recursing into subtrees as necessary. That will be a
stable stream for a given commit, or tree.
It could be simplified a bit by using ls-tree -r (so you basically
have a single big tree). Then hash commit, ls-tree -r output and all
blobs pointed by ls-tree in listed order.
What problem are you trying to solve here, though, by deliberately
deviating what Git internally used to store these objects? If it is
OK to ignore the tree boundary, then you probably do not even need
trees in this secondary hash for validation in the first place.
For example, you can hash a stream:
<commit object contents> +
N * (<pathname> + NUL + <blob object contents>)
as long as the <pathname>s are sorted in a predictable order (like
in "the index order") in the output. That would be even simpler (I
am not saying it is necessarily better, and by inference neither is
your "simplification").
I did nearly that [1]. But this morning I realized trees carry file
permission. We should keep that in the final checksum as well.
Now, if the final objective is to replace signature of tarballs,
does it matter to cover the commit object, or is it sufficient to
cover the tree contents?
Among the ideas raised so far, I like what Joey suggested, combined
with "each should have '<type> <length>NUL' header" from Sam Vilain
the best. That is, hash the stream:
"commit <length>" NUL + <commit object contents> +
"tree <length>" NUL + <top level tree contents> +
... list the entries in the order you would find by
... some defined traversal order people can agree on.
with whatever the preferred strong hash function of the age.
A bit harder to script, but simpler to provide from cat-file, I think.
[1] http://article.gmane.org/gmane.comp.version-control.git/260211
--
Duy
On 03/03/2015 12:44 AM, Junio C Hamano wrote:
[...]
I was about to suggest another alternative.
Pretend as if Git internally used SHA-512 (or whatever hash you
want to use) instead of SHA-1, compute the object names that
way. Recompute the contents of a tree object is by replacing
the 20-byte SHA-1 field in it with a field with whatever
necessary length to hold the longer object names of elements in
the tree.
But then a realization hit me: what new value will be placed in the
"parent " field in the commit object? You cannot have SHA-512
variant of commit object name without recomputing the whole history.
Now, if the final objective is to replace signature of tarballs,
does it matter to cover the commit object, or is it sufficient to
cover the tree contents?
The original goal was to replace a tarball signature, for which the
"alternative" that you described above seems quite elegant.
If the goal were really to certify the entire history, then none of the
proposals that I have seen so far is adequate anyway, because none of
them propose to include better than the original SHA-1s of the parent
commits.
Including other metadata from the release commit does not seem useful to
me; how valuable is it to know the author and commit message of the last
commit that happened to make it into a release? It would be more useful
to know the SHA-1 of that commit, but that would presumably be included
elsewhere in the packaging data used by the distribution.
[...]
Michael
--
Michael Haggerty
mhagger@alum.mit.edu