Git log of all (modifying) commands run on a repo?

Our company recently switched to git.  Our enthusiasm for git outweighs
the problems we have encountered so far, universally those of user
befuddlement, not git problems per se.

I have destroyed several repos by accidentally pulling from or pushing
to the wrong branch, not realizing this until later, and, trying what
I thought the proper way to undo it, failed that miserably.  I have
destroyed a few other repos by pulling from a "polluted" repo of a
co-worker that had mistakenly merged things in, and have had to resort
each time to full-scale reconstructions (again, almost trivially
accomplished thanks to git).  Thankfully, our company repo is so far
pristine, and I have been able to quickly reconstruct things and
continue working.

A co-worker just called to say he apparently had done something
similar, as he tried to pull from our company repo, got a conflict,
and then discovered files and changes to files that should not have
been on the master branch.  These files and changes do not exist in
our company repo, of that we are certain.

I'm not certain how I managed to screw up my repo, and he is equally
unsure of how he managed to screw up his.

I thought that it would be very useful to have a history of git
commands that have been run against a repo, in order to hopefully
debug this sort of thing --- perhaps only those that modify the repo.

I suppose this would need to include information about which branches
were affected as well.

% cat .git/repo.log
[....]
2007-02-07 10:05:44 git pull . .....
** Pull from master branch onto project branch
** Pull from foobar branch onto boobar branch
** ...
2007-02-07 10:05:44 git pull git://source/project master:project
** Your explanation here...

Would this sort of thing be easy to do?  I presume git does not
currently have support for this, but thought I would ask.


Bill

[Cc: git@vger.kernel.org]

Bill Lear wrote:

I thought that it would be very useful to have a history of git
commands that have been run against a repo, in order to hopefully
debug this sort of thing --- perhaps only those that modify the repo.

[...]

Would this sort of thing be easy to do?  I presume git does not
currently have support for this, but thought I would ask.

If all commands were run via git wrapper (and all scripts would
use git-command version, perhaps with exception of gitweb), it
would be fairly easy to do (although this should probably be
repository config variable).

Is history of shell commands too short? Reflogs doesn't help
(concatenate all reflogs, perhaps preceding them with file name,
and sort by date) either?
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

On Wednesday, February 7, 2007 at 23:29:56 (+0100) Jakub Narebski writes:

...
Is history of shell commands too short? Reflogs doesn't help
(concatenate all reflogs, perhaps preceding them with file name,
and sort by date) either?

Well, I thought of shell history also, but the problem is, that we often
have multiple repos on a machine (we may be executing a long-running
experiment in one, for example, and not want to disturb it).

Shell history in that case doesn't help.  Also, not having the branch
information is a minus (each of us puts the current git branch in our
prompt).

I didn't know about reflogs.  What did you have in mind?  Could you
show an example, please?


Bill

On Wed, 7 Feb 2007, Bill Lear wrote:

I thought that it would be very useful to have a history of git
commands that have been run against a repo, in order to hopefully
debug this sort of thing --- perhaps only those that modify the repo.

The "reflog" is kind of that, and you can literally do

	less .git/logs/refs/heads/master

to get some kind of idea what's up.

Newer gits enable reflogs by default, older gits don't (and *really* old 
git versions don't support it at all).

Of course, we should probably just have a

	git reflog show

command for this to make it prettier.

NOTE! It's very much per-repository. When you push out your changes, your 
reflog stays private.

			Linus

Linus Torvalds [off-list ref] wrote:

Of course, we should probably just have a

	git reflog show

command for this to make it prettier.

I tried that (I submitted patches to create `git reflog show`).
Dscho's `git log -g` was the better candidate, and won.
Its already in git.git's master branch.

-- 
Shawn.

Bill Lear wrote:

On Wednesday, February 7, 2007 at 23:29:56 (+0100) Jakub Narebski writes:

quoted

...
Is history of shell commands too short? Reflogs doesn't help
(concatenate all reflogs, perhaps preceding them with file name,
and sort by date) either?

Well, I thought of shell history also, but the problem is, that we often
have multiple repos on a machine (we may be executing a long-running
experiment in one, for example, and not want to disturb it).

Shell history in that case doesn't help.  Also, not having the branch
information is a minus (each of us puts the current git branch in our
prompt).

Fact. Although in worst case, if shell history is deep enough, you could
do forensic on it... unless you use multiple shells, freely interspersing
commands in them...
 

I didn't know about reflogs.  What did you have in mind?  Could you
show an example, please?

Unfortunately till some time ago reflogs were not enabled by default.
If they are enabled, for each branch you should have 
$GIT_DIR/logs/refs/heads/<branch> file, which contains information about
operations on given branch. It has pre- and post-operation commit ids,
and timestamp of a change, together with short description. You could
concatenate reflogs, and sort them by timestamp.

Reflogs are purely local matter, they represent local view of history.

See git-update-ref(1), "Logging Updates" section, git-repo-config(1)
(or git-config(1)), description of core.logAllRefUpdates variable,
and Git User's Manual.


Examples (sha1 are shortened, and lines are broken for easier reading):

 * .git/logs/refs/heads/origin
 798123... e68989... Jakub Narebski [off-list ref] 1170836783 +0100 \
    pull origin git://git.kernel.org/pub/scm/git/git.git: fast-forward
 e68989... 6506e1... Jakub Narebski [off-list ref] 1170873620 +0100 \
   fetch origin git://git.kernel.org/pub/scm/git/git.git: fast-forward

 * git/logs/refs/heads/master
 93342d... 39daf6... Jakub Narebski [off-list ref] 1169555797 +0100 \
   commit: t/t1300-repo-config.sh: value continued on next line
 39daf6... b62502... Jakub Narebski [off-list ref] 1169809125 +0100 \
   commit (merge): Merge branch 'master' of git://git.kernel.org/pub/scm/git/git
 b62502... 7ab704... Jakub Narebski [off-list ref] 1170150241 +0100 \
   pull origin: Merge made by recursive.
 7ab704... bb77c5... Jakub Narebski [off-list ref] 1170238531 +0100 \
   pull origin: In-index merge

 * git/logs/refs/heads/gitweb/web
 ceae8a... a29878... Jakub Narebski [off-list ref] 1169973601 +0100 \
   commit: Git config file reader in Perl (WIP 5)
 a29878... 1b600e... Jakub Narebski [off-list ref] 1170034498 +0100 \
   reset --hard 1b600e659abc7e409c9d830e332d3cef010
 62c1c1... 05b936... Jakub Narebski [off-list ref] 1170034499 +0100 \
   rebase: Git config file reader in Perl (WIP 1)
-- 
Jakub Narebski
Poland

On Wednesday, February 7, 2007 at 14:49:50 (-0800) Linus Torvalds writes:

...
The "reflog" is kind of that, and you can literally do

less .git/logs/refs/heads/master

to get some kind of idea what's up.

Newer gits enable reflogs by default, older gits don't (and *really* old 
git versions don't support it at all).

Ah, I don't seem to have .git/logs at all.

We plan to upgrade to 1.5 when it's out, so I guess we'll just
have to wait.


Bill

On Wed, 7 Feb 2007, Linus Torvalds wrote:

Newer gits enable reflogs by default [...]

This, btw, is only true for non-bare ones. 

In particular, if you only _push_ to a repository, it won't have reflogs 
enabled by default, because pushing is a "bare git" action even if it 
actually has a working tree (because the push will ignore the working tree 
and enter the ".git" directory itself).

So even with new git, if you want to have reflogs enabled for shared 
repositories that people push to (as opposed to the individual *developer* 
repositories), you need to enable reflogs by hand (either in the git 
config file, or by just creating the empty log file).

		Linus

On Wed, 7 Feb 2007, Bill Lear wrote:

Ah, I don't seem to have .git/logs at all.

We plan to upgrade to 1.5 when it's out, so I guess we'll just
have to wait.

Even with the git you have, you can probably just do a

	git repo-config --global core.logallrefupdates true

and you'll have reflogs enabled for all of your repositories (well, at 
least the ones that share that home directory and thus ~/.gitconfig)

		Linus

On Wednesday, February 7, 2007 at 15:29:40 (-0800) Linus Torvalds writes:

...
Even with the git you have, you can probably just do a

git repo-config --global core.logallrefupdates true

and you'll have reflogs enabled for all of your repositories (well, at 
least the ones that share that home directory and thus ~/.gitconfig)

Ok, this is good to know, thank you.


Bill

Linus Torvalds wrote:

On Wed, 7 Feb 2007, Bill Lear wrote:

quoted

I thought that it would be very useful to have a history of git
commands that have been run against a repo, in order to hopefully
debug this sort of thing --- perhaps only those that modify the repo.

The "reflog" is kind of that, and you can literally do

        less .git/logs/refs/heads/master

to get some kind of idea what's up.

Or something like this:

find .git/logs -type f | xargs cat | sort -t'>' -k2 -n | less

-- Hannes