From: Joachim Breitner <hidden> Date: 2016-06-15 22:59:53
Hi,
judging from the documentation I got the impression that I can pass any
git object has to "git note -C <hash>" and it would stored as-is. But it
seems the objects gets mangled somehow...
(I want to attach a commit object as a note, to reference the history of
a feature before the final cleanup rebase. For that I turn the reflog
into a series of commits, and the final commit is the one I want to
store there.)
$ mkdir foo
$ cd foo/
$ echo foo > a
$ git init
Initialisierte leeres Git-Repository in /tmp/foo/.git/
$ git add a
$ git commit -m 'A commit'
[master (Basis-Commit) 3d7de37] A commit
1 file changed, 1 insertion(+)
create mode 100644 a
$ echo foo2 > a
$ git commit -m 'A commit 2' -a
[master e1bfac4] A commit 2
1 file changed, 1 insertion(+), 1 deletion(-)
$ git notes --ref history add -C 3d7de37 e1bfac4
$ git ls-tree notes/history
100644 blob 5b73d5152e6207e3a2b67e57ca3a2cb94d12061e e1bfac434ebd3135a3784f6fc802f235098eebd0
I was expecting 3d7de37 to be referenced here.
Is that a bug, or is storing commits as notes not supported?
Thanks,
Joachim
--
Joachim “nomeata” Breitner
mail@joachim-breitner.de • http://www.joachim-breitner.de/
Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C
Debian Developer: nomeata@debian.org
From: Johan Herland <hidden> Date: 2016-06-15 22:59:53
On Tue, Feb 11, 2014 at 6:23 PM, Joachim Breitner
[off-list ref] wrote:
Hi,
judging from the documentation I got the impression that I can pass any
git object has to "git note -C <hash>" and it would stored as-is. But it
seems the objects gets mangled somehow...
...well... the documentation does not say "any object", it actually
explicitly says "blob object"... ;)
(I want to attach a commit object as a note, to reference the history of
a feature before the final cleanup rebase. For that I turn the reflog
into a series of commits, and the final commit is the one I want to
store there.)
$ mkdir foo
$ cd foo/
$ echo foo > a
$ git init
Initialisierte leeres Git-Repository in /tmp/foo/.git/
$ git add a
$ git commit -m 'A commit'
[master (Basis-Commit) 3d7de37] A commit
1 file changed, 1 insertion(+)
create mode 100644 a
$ echo foo2 > a
$ git commit -m 'A commit 2' -a
[master e1bfac4] A commit 2
1 file changed, 1 insertion(+), 1 deletion(-)
$ git notes --ref history add -C 3d7de37 e1bfac4
$ git ls-tree notes/history
100644 blob 5b73d5152e6207e3a2b67e57ca3a2cb94d12061e e1bfac434ebd3135a3784f6fc802f235098eebd0
I was expecting 3d7de37 to be referenced here.
Is that a bug, or is storing commits as notes not supported?
I guess it depends on your POV... The current documentation says "blob
object", and what actually happens (in builtin/notes.c) is that the
given object (3d7de37 in your example) is read into a strbuf (in
parse_reuse_arg()) and stored back into a note object (in
create_note()), without preserving the object type ("blob" type is
hardcoded). This means an incoming blob object (as documented) will be
preserved (i.e. reuse the same SHA-1), but for any other object type,
the object bits will be read, and stored back into a blob object. This
is why your commit (or any other non-blob) ends up with a different
SHA-1 when stored as a note: It is the same bytes, but in a blob
object instead of a commit object.
There is currently no way the "git notes" commands will allow you to
store the 3d7de37 commit object directly as a note. There is also
(AFAICS) no easy workaround (git fast-import could've been a
workaround if it did not already require the first N/notemodify
argument to be a blob object). The best alternative, off the top of my
head, would be to write your own program using the notes.h API to
manipulate the notes tree directly (or - suboptimally - use other
low-level Git operations to do the same).
However before you go there, let's take a step back, and look at what
the result would look like (if you were allowed to store a commit
object directly as a note):
You would have a notes ref "refs/notes/history" whose tree would
contain an entry named e1bfac434ebd3135a3784f6fc802f235098eebd0
pointing to a _commit_ (3d7de37...). Obviously, it would not make
sense to use refs/notes/history while displaying the commit log ("git
log --notes=history"), as the raw commit object would be shown in the
log. However, more fundamentally: a tree referring to a _commit_ is
usually how Git stores _submodule_ links (i.e. which revision of the
named submodule is to be used by this super-repo tree), and I'm (off
the top of my head) not at all sure that such a submodule link in a
notes tree is handled "sanely" by Git - or even that it makes sense at
all. For one, I'm not sure that Git requires (or even expects) the
commit object referenced by a tree to be present in the same object
DB. So if you share your notes, I don't know whether or not the
fetch/push machinery will include the commit object in the shared
notes... These are questions that should be answered before we decide
whether using commits directly as notes makes sense.
If we do figure out that storing commits as note objects is desirable
(and does not have too nasty side-effects), then I am not opposed to
fixing builtin/notes.c to preserve type of the object passed in -C.
Certainly, the current behavior for non-blobs (i.e. copy the object
bytes, but not the object type) is probably not useful to anyone...
That said, there may be other ways to solve your immediate problem:
Instead of storing the commit object directly as a note, you could
store the commit SHA-1 into a blob, and use that as the blob object.
That would also allow you to store multiple commits in the note for
e1bfac4 (in case you had several cleanup rebases leading to the final
commit), or you could store other kinds of metadata in the same note.
Or do you have a requirement that the reflog history (presumably
reachable from 3d7de37) need to be shared (or otherwise kept
reachable)? In that case, you might be better off using an explicit
ref to keep that history alive; e.g. you could create
refs/history/e1bfac4 to point to 3d7de37 ("git update-ref
refs/history/e1bfac4 3d7de37"), and keep everything
alive/reachable/shareable that way...
Hope this helps,
...Johan
--
Johan Herland, [off-list ref]
www.herland.net
From: Joachim Breitner <hidden> Date: 2016-06-15 22:59:53
Dear Johan,
Am Mittwoch, den 12.02.2014, 00:52 +0100 schrieb Johan Herland:
On Tue, Feb 11, 2014 at 6:23 PM, Joachim Breitner
[off-list ref] wrote:
quoted
judging from the documentation I got the impression that I can pass any
git object has to "git note -C <hash>" and it would stored as-is. But it
seems the objects gets mangled somehow...
...well... the documentation does not say "any object", it actually
explicitly says "blob object"... ;)
ok, my bad; guess I’m not fully versed with gits terminology.
You would have a notes ref "refs/notes/history" whose tree would
contain an entry named e1bfac434ebd3135a3784f6fc802f235098eebd0
pointing to a _commit_ (3d7de37...). Obviously, it would not make
sense to use refs/notes/history while displaying the commit log ("git
log --notes=history"), as the raw commit object would be shown in the
log. However, more fundamentally: a tree referring to a _commit_ is
usually how Git stores _submodule_ links (i.e. which revision of the
named submodule is to be used by this super-repo tree), and I'm (off
the top of my head) not at all sure that such a submodule link in a
notes tree is handled "sanely" by Git - or even that it makes sense at
all. For one, I'm not sure that Git requires (or even expects) the
commit object referenced by a tree to be present in the same object
DB. So if you share your notes, I don't know whether or not the
fetch/push machinery will include the commit object in the shared
notes... These are questions that should be answered before we decide
whether using commits directly as notes makes sense.
If that is the case, then my approach is indeed flawed. The main point
of the exercise is to have a tree that follows another commit (or, as a
next-best approximation, a note attached to that commit) around.
In that case, you might be better off using an explicit
ref to keep that history alive; e.g. you could create
refs/history/e1bfac4 to point to 3d7de37 ("git update-ref
refs/history/e1bfac4 3d7de37"), and keep everything
alive/reachable/shareable that way...
That’s an interesting idea; instead of relying on the notes feature
putting the hash in the ref name. But I wonder how that scales – imagine
every second feature merged into Linux¹ also having such a history ref?
I guess having a way for a tree to reference commits in a way that is
followed by git gc, i.e. separate from submodules, would allow a less
noisy implementation, and possibly create the opportunity for many other
strange uses of git :-)
Greetings,
Joachim
¹ I’m not proposing for anyone else but me to use this, at the moment,
don’t worry :-). But I am considering to use it in the context of GHC,
which isn’t a small project either.
--
Joachim “nomeata” Breitner
mail@joachim-breitner.de • http://www.joachim-breitner.de/
Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C
Debian Developer: nomeata@debian.org
From: Johan Herland <hidden> Date: 2016-06-15 22:59:53
On Wed, Feb 12, 2014 at 9:53 AM, Joachim Breitner
[off-list ref] wrote:
Am Mittwoch, den 12.02.2014, 00:52 +0100 schrieb Johan Herland:
quoted
You would have a notes ref "refs/notes/history" whose tree would
contain an entry named e1bfac434ebd3135a3784f6fc802f235098eebd0
pointing to a _commit_ (3d7de37...). Obviously, it would not make
sense to use refs/notes/history while displaying the commit log ("git
log --notes=history"), as the raw commit object would be shown in the
log. However, more fundamentally: a tree referring to a _commit_ is
usually how Git stores _submodule_ links (i.e. which revision of the
named submodule is to be used by this super-repo tree), and I'm (off
the top of my head) not at all sure that such a submodule link in a
notes tree is handled "sanely" by Git - or even that it makes sense at
all. For one, I'm not sure that Git requires (or even expects) the
commit object referenced by a tree to be present in the same object
DB. So if you share your notes, I don't know whether or not the
fetch/push machinery will include the commit object in the shared
notes... These are questions that should be answered before we decide
whether using commits directly as notes makes sense.
If that is the case, then my approach is indeed flawed. The main point
of the exercise is to have a tree that follows another commit (or, as a
next-best approximation, a note attached to that commit) around.
quoted
In that case, you might be better off using an explicit
ref to keep that history alive; e.g. you could create
refs/history/e1bfac4 to point to 3d7de37 ("git update-ref
refs/history/e1bfac4 3d7de37"), and keep everything
alive/reachable/shareable that way...
That’s an interesting idea; instead of relying on the notes feature
putting the hash in the ref name. But I wonder how that scales – imagine
every second feature merged into Linux¹ also having such a history ref?
Ah, that will probably not scale very well.
I guess having a way for a tree to reference commits in a way that is
followed by git gc, i.e. separate from submodules, would allow a less
noisy implementation, and possibly create the opportunity for many other
strange uses of git :-)
Here's another way to solve your problem, which should be fairly
transparent and performant:
Whenever you want to reference "history" of a commit (I'm using quotes
here, because we're not talking about the "regular" git sense of
history, i.e. the commit graph), you perform the following two steps:
1. Append the "historical" commit SHA-1 (3d7de37 in your example) to a
note on the "current" commit (e1bfac4). E.g.:
git notes --ref history append -m 3d7de37... e1bfac4...
2. Perform some automated merge into a "history"-tracking ref (e.g.
refs/history), to keep the "historical" commits reachable.
(You can easily wrap both steps into a script to automate things.)
Step #1 encodes the "history" of a commit in a note, but does not keep
the "history" reachable.
Step #2 keeps all "historical" commits reachable by making them part
of the history (in the git sense - without quotes) of a proper ref
(refs/history). The actual result/outcome of the merge is not
interesting. It only exists to insert the "historical" commit
(3d7de37) into the ancestry for refs/history. Since the actual merge
itself is uninteresting, you should probably use a merge strategy that
never yields conflicts, e.g. "-s ours"
You can now share the "history" by pushing/fetching the two refs
refs/notes/history and refs/history.
(In theory, you might even be able to combine the two refs, by
performing the merge directly into refs/notes/history, always taking
care to retain the notes tree contents as the result of the merge. In
other words, after you do step #1 (append the note), you manually
rewrite the just-created tip of refs/notes/history to include 3d7de37
as a second parent. This keeps 3d7de37 reachable (and it will be
shared when you share refs/notes/history), and it should not interfere
with the notes infrastructure, as they only look at the current state
of the notes tree.)
Hope this helps,
...Johan
--
Johan Herland, [off-list ref]
www.herland.net
From: Joachim Breitner <hidden> Date: 2016-06-15 22:59:53
Dear Johan,
thanks for the patch!
Am Mittwoch, den 12.02.2014, 11:26 +0100 schrieb Johan Herland:
Here's another way to solve your problem, which should be fairly
transparent and performant:
Whenever you want to reference "history" of a commit (I'm using quotes
here, because we're not talking about the "regular" git sense of
history, i.e. the commit graph), you perform the following two steps:
1. Append the "historical" commit SHA-1 (3d7de37 in your example) to a
note on the "current" commit (e1bfac4). E.g.:
git notes --ref history append -m 3d7de37... e1bfac4...
2. Perform some automated merge into a "history"-tracking ref (e.g.
refs/history), to keep the "historical" commits reachable.
(You can easily wrap both steps into a script to automate things.)
Step #1 encodes the "history" of a commit in a note, but does not keep
the "history" reachable.
Step #2 keeps all "historical" commits reachable by making them part
of the history (in the git sense - without quotes) of a proper ref
(refs/history). The actual result/outcome of the merge is not
interesting. It only exists to insert the "historical" commit
(3d7de37) into the ancestry for refs/history. Since the actual merge
itself is uninteresting, you should probably use a merge strategy that
never yields conflicts, e.g. "-s ours"
You can now share the "history" by pushing/fetching the two refs
refs/notes/history and refs/history.
(In theory, you might even be able to combine the two refs, by
performing the merge directly into refs/notes/history, always taking
care to retain the notes tree contents as the result of the merge. In
other words, after you do step #1 (append the note), you manually
rewrite the just-created tip of refs/notes/history to include 3d7de37
as a second parent. This keeps 3d7de37 reachable (and it will be
shared when you share refs/notes/history), and it should not interfere
with the notes infrastructure, as they only look at the current state
of the notes tree.)
That is quite a good approximation. What it doesn’t do is dropping
history (in the refs/history sense) of commits that disappear, but the
same problem exists with notes. Thanks!
I guess there are no plans to make the commit object format itself
extensible, are they? Extensible in the sense that I can add a custom
field to it (e.g. history:). Git would not have to know anything about
the field besides its type, i.e. that it contains refs that it has to
follow. Very much like "parent:", just without the semantics of it wrt.
"git log" and the like.
Greetings,
Joachim
--
Joachim “nomeata” Breitner
mail@joachim-breitner.de • http://www.joachim-breitner.de/
Jabber: nomeata@joachim-breitner.de • GPG-Key: 0x4743206C
Debian Developer: nomeata@debian.org