[QUESTION] Tracking HEAD changes?

Hi,

Is there any possible way to track changes to HEAD using hooks?

Being able to listen using hooks to events such as pre-head-checkout
and post-head-checkout would be the best option (from my perspective).

To my knowledge, the only possible way to do that today is by adding a
file watch over the refs directory.

Thank you!
    Yaron

On Mon, Feb 22, 2021 at 11:12:11AM +0200, Yaron Wittenstein wrote:

Is there any possible way to track changes to HEAD using hooks?

Being able to listen using hooks to events such as pre-head-checkout
and post-head-checkout would be the best option (from my perspective).

To my knowledge, the only possible way to do that today is by adding a
file watch over the refs directory.

No, I don't think there is currently a better way.

If you do go the file watch route, make sure you put one on HEAD, too
(to catch the case of checking out a different branch, as opposed to
updating it with a new commit). I use:

  gitdir=$(git rev-parse --git-dir)
  # We need delete_self to pick up changes to HEAD (since it gets renamed
  # over), and "move" to pick up changes in the refs directories.
  inotifywait -qq -t 60 -e delete_self -e move -r "$gitdir/HEAD" "$gitdir/refs"

in a script that I use for automatically running tests against the
current commit. (if you are doing something similar, I also recommend
https://github.com/mhagger/git-test for avoiding re-running against the
same commits repeatedly).

-Peff

On Mon, Feb 22, 2021 at 07:38:30PM -0500, Jeff King wrote:

On Mon, Feb 22, 2021 at 11:12:11AM +0200, Yaron Wittenstein wrote:

quoted

Is there any possible way to track changes to HEAD using hooks?

Being able to listen using hooks to events such as pre-head-checkout
and post-head-checkout would be the best option (from my perspective).

To my knowledge, the only possible way to do that today is by adding a
file watch over the refs directory.

No, I don't think there is currently a better way.

Actually, I completely forgot about Patrick's recent ref-transaction
hook. See the "reference-transaction" section of githooks(7). They do
more than you'd need, but you should be able to write a hook that just
looks for updates to HEAD, or updates to the ref that HEAD is pointing
at.

The code was introduced in 6754159767 (refs: implement reference
transaction hook, 2020-06-19), so you'll need Git v2.28.0 or later.

-Peff

That indeed seems to do the trick.
I've done a little experiment and saw that when doing git reset the
hook gets called.

However, when switching branches the hook doesn't execute :(

I don't understand if it's intentional, since when I've moved to a new
branch HEAD pointed to another commit id.
The only workaround I see here is using the post-checkout hook in addition.

Thank you very much for helping me. I really appreciate it!!
     Yaron

On Tue, Feb 23, 2021 at 10:42 PM Jeff King [off-list ref] wrote:

On Mon, Feb 22, 2021 at 07:38:30PM -0500, Jeff King wrote:

quoted

On Mon, Feb 22, 2021 at 11:12:11AM +0200, Yaron Wittenstein wrote:

quoted

Is there any possible way to track changes to HEAD using hooks?

Being able to listen using hooks to events such as pre-head-checkout
and post-head-checkout would be the best option (from my perspective).

To my knowledge, the only possible way to do that today is by adding a
file watch over the refs directory.

No, I don't think there is currently a better way.

Actually, I completely forgot about Patrick's recent ref-transaction
hook. See the "reference-transaction" section of githooks(7). They do
more than you'd need, but you should be able to write a hook that just
looks for updates to HEAD, or updates to the ref that HEAD is pointing
at.

The code was introduced in 6754159767 (refs: implement reference
transaction hook, 2020-06-19), so you'll need Git v2.28.0 or later.

-Peff

On Wed, Feb 24, 2021 at 10:21:55PM +0200, Yaron Wittenstein wrote:

That indeed seems to do the trick.
I've done a little experiment and saw that when doing git reset the
hook gets called.

However, when switching branches the hook doesn't execute :(

I don't understand if it's intentional, since when I've moved to a new
branch HEAD pointed to another commit id.
The only workaround I see here is using the post-checkout hook in addition.

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

-Peff

On Wed, Feb 24, 2021 at 04:03:14PM -0500, Jeff King wrote:

On Wed, Feb 24, 2021 at 10:21:55PM +0200, Yaron Wittenstein wrote:

quoted

That indeed seems to do the trick.
I've done a little experiment and saw that when doing git reset the
hook gets called.

However, when switching branches the hook doesn't execute :(

I don't understand if it's intentional, since when I've moved to a new
branch HEAD pointed to another commit id.
The only workaround I see here is using the post-checkout hook in addition.

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

-Peff

It was done semi-intentionally, or at least with the knowledge that
symrefs aren't covered. This is mostly because they're not covered by
the reference transaction mechanism itself.

But this again reminds me that I still have to update the documentation
of the hook to at least make it more explicit what's currently covered
and what's not.

Patrick

Thank you for the clarification!

Is there any plan to support such a case in the future?

Also, I've observed that the "post-index-change" hook is being
triggered before calling the "reference-transaction" hook (with the
"prepared" state).
It seems not intuitive to me since the index and working dirs are
being updated before approving the transaction.
(the HEAD still points to an "old" reference while the
"post-index-change" hook is executing).

Thank you,
   Yaron

On Thu, Feb 25, 2021 at 7:37 AM Patrick Steinhardt [off-list ref] wrote:

On Wed, Feb 24, 2021 at 04:03:14PM -0500, Jeff King wrote:

quoted

On Wed, Feb 24, 2021 at 10:21:55PM +0200, Yaron Wittenstein wrote:

quoted

That indeed seems to do the trick.
I've done a little experiment and saw that when doing git reset the
hook gets called.

However, when switching branches the hook doesn't execute :(

I don't understand if it's intentional, since when I've moved to a new
branch HEAD pointed to another commit id.
The only workaround I see here is using the post-checkout hook in addition.

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

-Peff

It was done semi-intentionally, or at least with the knowledge that
symrefs aren't covered. This is mostly because they're not covered by
the reference transaction mechanism itself.

But this again reminds me that I still have to update the documentation
of the hook to at least make it more explicit what's currently covered
and what's not.

Patrick

On Thu, Feb 25, 2021 at 08:01:11AM +0200, Yaron Wittenstein wrote:

Thank you for the clarification!

Is there any plan to support such a case in the future?

I personally don't have any plans to do so, but I can see usecases where
it would make sense. So I wouldn't be opposing any such efforts either.

Also, I've observed that the "post-index-change" hook is being
triggered before calling the "reference-transaction" hook (with the
"prepared" state). It seems not intuitive to me since the index and
working dirs are being updated before approving the transaction. (the
HEAD still points to an "old" reference while the "post-index-change"
hook is executing).

This is something that cannot really be helped though. The
reference-transaction hook directly hooks into the reference transaction
mechanism, which is how git updates all references. So we do not really
have any control over when the hook will get executed: it will simply
get executed whenever the reference transaction itself gets prepared
(refs get locked) and committed (written updated refs got moved into
place).

And that's by design: my objective was to catch _all_ reference updates
such that one can coordinate across multiple git nodes which all perform
the same action to assert they're moving from the same state to the same
state, regardless of whether they're doing a git-push(1), git-merge(1)
or git-update-ref(1).

So what you're observing is simply mirroring "reality": the order in
which git does its things here. There can be arbitrarily many
transactions in a given git command, and the only way this can be
changed is by changing how the command operating the transcations works.

Patrick

Thank you,
   Yaron

On Thu, Feb 25, 2021 at 7:37 AM Patrick Steinhardt [off-list ref] wrote:

quoted

On Wed, Feb 24, 2021 at 04:03:14PM -0500, Jeff King wrote:

quoted

On Wed, Feb 24, 2021 at 10:21:55PM +0200, Yaron Wittenstein wrote:

quoted

That indeed seems to do the trick.
I've done a little experiment and saw that when doing git reset the
hook gets called.

However, when switching branches the hook doesn't execute :(

I don't understand if it's intentional, since when I've moved to a new
branch HEAD pointed to another commit id.
The only workaround I see here is using the post-checkout hook in addition.

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

-Peff

It was done semi-intentionally, or at least with the knowledge that
symrefs aren't covered. This is mostly because they're not covered by
the reference transaction mechanism itself.

But this again reminds me that I still have to update the documentation
of the hook to at least make it more explicit what's currently covered
and what's not.

Patrick

On Thu, Feb 25, 2021 at 06:36:54AM +0100, Patrick Steinhardt wrote:

quoted

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

It was done semi-intentionally, or at least with the knowledge that
symrefs aren't covered. This is mostly because they're not covered by
the reference transaction mechanism itself.

Ah, right, I forgot about that. That might be something that we'd want
to fix if anybody wanted to use the transaction hooks to track the
complete state of refs.

At GitHub, our repository replication does track symbolic ref updates,
but we handle it outside of Git. Our normal ref updates use a few custom
bits of code similar to your transaction hooks in order to implement a
3-phase commit. But our symref updates don't; they just take a lock, run
git-symbolic-ref on each, and then vote on the result. This is slower
and less robust, but symbolic ref updates are relatively rare, so it
hasn't been a big deal.

-Peff

Thank you all!

I'd like to make sure I understand the way things are right now:

1. The reference-transaction hook as it's today doesn't intercept
symbolic-references changes. It means that when HEAD changes due to
branch-switching the hook won't
get called.

Are there any other cases that today the transaction won't execute? (I
couldn't think of one)

2. The mechanisms that handle the changes to index and working-dir are
isolated from the one that manages the refs updates.

A side-effect to that is the post-index-change hook running before the
reference-transaction one.
Future code changes to the way git operates could result in the
reference-transaction hook running before the post-index-change one.

Is that correct?

  Thank you!
      Yaron

On Fri, Feb 26, 2021 at 7:59 AM Jeff King [off-list ref] wrote:

On Thu, Feb 25, 2021 at 06:36:54AM +0100, Patrick Steinhardt wrote:

quoted

quoted

Hmm, I would have thought that the branch switch would trigger the hooks
because they're updating HEAD. I wonder if that is a bug (or lack of
feature :) ) in the transaction hooks, or something Patrick did
intentionally.

It was done semi-intentionally, or at least with the knowledge that
symrefs aren't covered. This is mostly because they're not covered by
the reference transaction mechanism itself.

Ah, right, I forgot about that. That might be something that we'd want
to fix if anybody wanted to use the transaction hooks to track the
complete state of refs.

At GitHub, our repository replication does track symbolic ref updates,
but we handle it outside of Git. Our normal ref updates use a few custom
bits of code similar to your transaction hooks in order to implement a
3-phase commit. But our symref updates don't; they just take a lock, run
git-symbolic-ref on each, and then vote on the result. This is slower
and less robust, but symbolic ref updates are relatively rare, so it
hasn't been a big deal.

-Peff

On Fri, Feb 26, 2021 at 10:58:37PM +0200, Yaron Wittenstein wrote:

I'd like to make sure I understand the way things are right now:

1. The reference-transaction hook as it's today doesn't intercept
symbolic-references changes. It means that when HEAD changes due to
branch-switching the hook won't
get called.

Are there any other cases that today the transaction won't execute? (I
couldn't think of one)

No, I don't think so.

2. The mechanisms that handle the changes to index and working-dir are
isolated from the one that manages the refs updates.

A side-effect to that is the post-index-change hook running before the
reference-transaction one.
Future code changes to the way git operates could result in the
reference-transaction hook running before the post-index-change one.

Is that correct?

Yes, that matches my understanding.

-Peff

Cool, thanks!

On Mon, Mar 1, 2021 at 11:03 AM Jeff King [off-list ref] wrote:

On Fri, Feb 26, 2021 at 10:58:37PM +0200, Yaron Wittenstein wrote:

quoted

I'd like to make sure I understand the way things are right now:

1. The reference-transaction hook as it's today doesn't intercept
symbolic-references changes. It means that when HEAD changes due to
branch-switching the hook won't
get called.

Are there any other cases that today the transaction won't execute? (I
couldn't think of one)

No, I don't think so.

quoted

2. The mechanisms that handle the changes to index and working-dir are
isolated from the one that manages the refs updates.

A side-effect to that is the post-index-change hook running before the
reference-transaction one.
Future code changes to the way git operates could result in the
reference-transaction hook running before the post-index-change one.

Is that correct?

Yes, that matches my understanding.

-Peff