Git not commiting anything if file created and "git add"ed in pre-commit hook is the only file in the staging area

Hi,
Git seems to behave inconsistently when creating and "git add"ing
content (files) from within a pre-commit hook. It claims there is
"nothing to commit", if the stating area was empty before the commit
command. Otherwise, the new content becomes part of the commit. See
details on how to reproduce and a log of the individual steps below.

Regards
Peter


=== Amended Git bug report
=============================================================================================
Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

I'm quite new to using git, so please bear with me should I mix up some
terminology.

I need some new file to be created at each commit. The new file must
become part of the commit.
I thought a git pre-commit hook would be the appropriate place. While
the file is created as desired,
git does *not* include this new file in the commit, when there is
nothing else in the staging area. It
reports the new file as being untracked, and says nothing added to
commit. A git status immediately
thereafter does, however, show the new file in the staging area. If I
run another commit just now, another
new file is created as expected, but this time, git adds them both to
the commit. This can be reliably
reproduced by repeating 'git commit -m "empty"', and every second time,
the commit is done.

Further testing shows that the newly created file *is* added to the
commit, *if* at least one other change
was registered to be commited (git add some time before the git commit).

Same behaviour, if two files are created and added withing the
pre-commit hook. Both files are added to
the commit only every second time, or if some other change does exist in
the staging area before the commit
command.

The git pre-commit hook looks like this (indented for readability, only):
     #!/bin/bash
     fn="folder1\folder1_file3_$(date +%Y%m%d_%H%M%S).txt"
     echo "some text" > $fn
     git add $fn

Note: This testing was done on Windows 10, but initially, I detected
this on a server running Linux.






What did you expect to happen? (Expected behavior)

I expect the file created and added from within the git pre-commit hook
to become part of the commit
no matter whether there are or aren't any other changes in the staging
are, i.e. added beforehand.




What happened instead? (Actual behavior)

File created and 'git added' from within the pre-commit hook shell
script is *not* added to the current
commit, if there are no other changes in the staging area.

The behaviour is inconsistent. If it is allowed to do a "git add" from
within a pre-commit hook (is it allowed?),
then file added must become part of the commit irrespective of whether
or not there are other changes in the
staging area that have been there *before* the commit command was
issued. Else git should reject the "git add"
(and other commands) from within the pre-commit hook.

What's different between what you expected and what actually happened?

see above,






Anything else you want to add:

I'm adding the log from a terminal session where I reproduce the
behaviour below, limited by a line
of '=' signs. Individual commands are separated by a line of '-' signs
with a comment what the next step
will be.

========================================================================================================

--- content of directory before git init
---------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ ls -lR
.:
total 6
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file1.txt
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file2.txt
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:19 folder1/
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:16 git-hook-saved/

./folder1:
total 2
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:58 folder1_file1.txt
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:58 folder1_file2.txt

./git-hook-saved:
total 1
-rwxr-xr-x 1 Peter 197121 111 Oct 25 14:21 pre-commit*

--- git init
-----------------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing
$ git init
Initialized empty Git repository in D:/Temp/git-testing/.git/

--- git initial status
-------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master

No commits yet

Untracked files:
   (use "git add <file>..." to include in what will be committed)
         file1.txt
         file2.txt
         folder1/
         git-hook-saved/

nothing added to commit but untracked files present (use "git add" to track)

--- adding all initial data to the staging area
---------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git add "*"

--- commiting initial data
------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "initial commit"
[master (root-commit) 4daefe7] initial commit
  5 files changed, 12 insertions(+)
  create mode 100644 file1.txt
  create mode 100644 file2.txt
  create mode 100644 folder1/folder1_file1.txt
  create mode 100644 folder1/folder1_file2.txt
  create mode 100644 git-hook-saved/pre-commit

--- Show status after initial commit
-----------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
nothing to commit, working tree clean

--- Install git pre-commit hook, then do an empty commit
-------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "empty 1"
warning: LF will be replaced by CRLF in
folder1/folder1_file3_20211025_142454.txt.
The file will have its original line endings in your working directory
On branch master
Untracked files:
   (use "git add <file>..." to include in what will be committed)
         folder1/folder1_file3_20211025_142454.txt

nothing added to commit but untracked files present (use "git add" to track)

--- Show whats in the file system at this point in time
-------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ ls -lR
.:
total 6
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file1.txt
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file2.txt
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:24 folder1/
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:16 git-hook-saved/

./folder1:
total 3
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file1.txt
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file2.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:24 folder1_file3_20211025_142454.txt

./git-hook-saved:
total 1
-rwxr-xr-x 1 Peter 197121 111 Oct 25 14:21 pre-commit

--- What is the status of git? The "add" from the hook did

work!------------------------
Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
Changes to be committed:
   (use "git restore --staged <file>..." to unstage)
         new file:   folder1/folder1_file3_20211025_142454.txt

--- Immediately do another (not really empty) git commit
-------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "empty 2"
warning: LF will be replaced by CRLF in
folder1/folder1_file3_20211025_142554.txt.
The file will have its original line endings in your working directory
[master e9809aa] empty 2
  2 files changed, 2 insertions(+)
  create mode 100644 folder1/folder1_file3_20211025_142454.txt
  create mode 100644 folder1/folder1_file3_20211025_142554.txt

--- Show whats in the file system at this point in time
-------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ ls -lR
.:
total 6
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file1.txt
-rw-r--r-- 1 Peter 197121 9 Oct 25 13:55 file2.txt
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:25 folder1/
drwxr-xr-x 1 Peter 197121 0 Oct 25 14:16 git-hook-saved/

./folder1:
total 4
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file1.txt
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file2.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:24 folder1_file3_20211025_142454.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:25 folder1_file3_20211025_142554.txt

./git-hook-saved:
total 1
-rwxr-xr-x 1 Peter 197121 111 Oct 25 14:21 pre-commit

--- Once more, this time really an "empty" commit again.
------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "empty 3"
warning: LF will be replaced by CRLF in
folder1/folder1_file3_20211025_142645.txt.
The file will have its original line endings in your working directory
On branch master
Untracked files:
   (use "git add <file>..." to include in what will be committed)
         folder1/folder1_file3_20211025_142645.txt

nothing added to commit but untracked files present (use "git add" to track)

--- Show the status, same behaviour again.
---------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
Changes to be committed:
   (use "git restore --staged <file>..." to unstage)
         new file:   folder1/folder1_file3_20211025_142645.txt

--- Yet another commit immediately thereafter
---------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "empty 4"
warning: LF will be replaced by CRLF in
folder1/folder1_file3_20211025_142659.txt.
The file will have its original line endings in your working directory
[master 032a58e] empty 4
  2 files changed, 2 insertions(+)
  create mode 100644 folder1/folder1_file3_20211025_142645.txt
  create mode 100644 folder1/folder1_file3_20211025_142659.txt

--- Status?
---------------------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
nothing to commit, working tree clean

--- Content of filesystem
---------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ ls -lR
.:
total 7
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:55 file1.txt
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:55 file2.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:27 file3.txt
drwxr-xr-x 1 Peter 197121  0 Oct 25 14:28 folder1/
drwxr-xr-x 1 Peter 197121  0 Oct 25 14:16 git-hook-saved/

./folder1:
total 7
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file1.txt
-rw-r--r-- 1 Peter 197121  9 Oct 25 13:58 folder1_file2.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:24 folder1_file3_20211025_142454.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:25 folder1_file3_20211025_142554.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:26 folder1_file3_20211025_142645.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:26 folder1_file3_20211025_142659.txt
-rw-r--r-- 1 Peter 197121 10 Oct 25 14:28 folder1_file3_20211025_142815.txt

./git-hook-saved:
total 1
-rwxr-xr-x 1 Peter 197121 111 Oct 25 14:21 pre-commit

--- Creating a new file outside of hook
--------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ echo "some text" > file3.txt

--- Git status?
------------------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
Untracked files:
   (use "git add <file>..." to include in what will be committed)
         file3.txt

nothing added to commit but untracked files present (use "git add" to track)

--- Adding the new file to the staging area
------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git add *
warning: LF will be replaced by CRLF in file3.txt.
The file will have its original line endings in your working directory

--- Status again
----------------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
Changes to be committed:
   (use "git restore --staged <file>..." to unstage)
         new file:   file3.txt

--- Committing the new file (and the one created by the pre-commit hook)
--------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git commit -m "one new file present"
warning: LF will be replaced by CRLF in
folder1/folder1_file3_20211025_142815.txt.
The file will have its original line endings in your working directory
[master 4ea47ec] one new file present
  2 files changed, 2 insertions(+)
  create mode 100644 file3.txt
  create mode 100644 folder1/folder1_file3_20211025_142815.txt

--- Status?
--------------------------------------------------------------------

Peter@MySystem MINGW64 /d/Temp/git-testing (master)
$ git status
On branch master
nothing to commit, working tree clean



========================================================================================================








Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.32.0.windows.2
cpu: x86_64
built from commit: 3d45ac813c4adf97fe3733c1f763ab6617d5add5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Windows 10.0 19042
compiler info: gnuc: 10.3
libc info: no libc information available
$SHELL (typically, interactive shell): C:\Programme
Non-UAC\Git\usr\bin\bash.exe


[Enabled Hooks]
pre-commit

On 2021-10-27 at 19:03:49, Peter Hunkeler wrote:

Hi,
Git seems to behave inconsistently when creating and "git add"ing
content (files) from within a pre-commit hook. It claims there is
"nothing to commit", if the stating area was empty before the commit
command. Otherwise, the new content becomes part of the commit. See
details on how to reproduce and a log of the individual steps below.

Regards
Peter


=== Amended Git bug report
=============================================================================================
Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

I'm quite new to using git, so please bear with me should I mix up some
terminology.

I need some new file to be created at each commit. The new file must
become part of the commit.
I thought a git pre-commit hook would be the appropriate place. While
the file is created as desired,
git does *not* include this new file in the commit, when there is
nothing else in the staging area. It
reports the new file as being untracked, and says nothing added to
commit. A git status immediately
thereafter does, however, show the new file in the staging area. If I
run another commit just now, another
new file is created as expected, but this time, git adds them both to
the commit. This can be reliably
reproduced by repeating 'git commit -m "empty"', and every second time,
the commit is done.

Further testing shows that the newly created file *is* added to the
commit, *if* at least one other change
was registered to be commited (git add some time before the git commit).

Yes, this occurs because the first thing we do is invoke the pre-commit
hook.  It passes, and then we consider various reasons why we should not
commit: the hook failed, there's nothing to commit, etc.  We discover
one since we haven't re-read the index yet and last we saw there was
nothing to commit, so we abort.

There's a giant comment before we re-read the index that says this:

		/*
		 * Re-read the index as pre-commit hook could have updated it,
		 * and write it out as a tree.  We must do this before we invoke
		 * the editor and after we invoke run_status above.
		 */

The commit history does not explain why we must do this _after_ invoking
run_status, but if the comment is incorrect and we re-read it
immediately, then this problem would go away.  I lack the relevant
context to determine whether this is appropriate, but if folks think
this is advisable, then I can write up a patch this weekend.  It should
be relatively trivial.

I should point out here that it isn't intended for pre-commit hooks to
be used this way; they're intended to verify that the commit meets some
standards, not to modify it, although it is of course possible to do.

This will happen to work in your case if you use --allow-empty, which
will bypass the "nothing to commit" check.

The git pre-commit hook looks like this (indented for readability, only):
    #!/bin/bash
    fn="folder1\folder1_file3_$(date +%Y%m%d_%H%M%S).txt"
    echo "some text" > $fn
    git add $fn

In general, you want to avoid adding automatically generated files to
your repository.  That tends to bloat the repository needlessly and is a
great way to lead to frequent, hard-to-resolve merge conflicts.

You may also want to read the “How do I use hooks to prevent users from
making certain changes?” in gitfaq(7) and note that hooks can be
bypassed on developer systems, so whatever changes you make in a hook
aren't guaranteed to be applied unless you're the only one working on
the repository.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

On Wed, Oct 27, 2021 at 10:07:59PM +0000, brian m. carlson wrote:

quoted

Further testing shows that the newly created file *is* added to the
commit, *if* at least one other change
was registered to be commited (git add some time before the git commit).

Yes, this occurs because the first thing we do is invoke the pre-commit
hook.  It passes, and then we consider various reasons why we should not
commit: the hook failed, there's nothing to commit, etc.  We discover
one since we haven't re-read the index yet and last we saw there was
nothing to commit, so we abort.

There's a giant comment before we re-read the index that says this:

		/*
		 * Re-read the index as pre-commit hook could have updated it,
		 * and write it out as a tree.  We must do this before we invoke
		 * the editor and after we invoke run_status above.
		 */

The commit history does not explain why we must do this _after_ invoking
run_status, but if the comment is incorrect and we re-read it
immediately, then this problem would go away.  I lack the relevant
context to determine whether this is appropriate, but if folks think
this is advisable, then I can write up a patch this weekend.  It should
be relatively trivial.

There's some related discussion of this area in:

  https://lore.kernel.org/git/CADv3qkGq3jA8iXsjhrqfsUX=gW+KOuLyeVgDzmku1tUpsMdvtw@mail.gmail.com/

I don't find it particularly enlightening as to the history, but it does
point to other people wanting to re-read before run_status().

-Peff

Am 28.10.2021 um 00:07 schrieb brian m. carlson:

I should point out here that it isn't intended for pre-commit hooks to
be used this way; they're intended to verify that the commit meets some
standards, not to modify it, although it is of course possible to do.

I can accept that comment. However:

- wouldn't you agree that git should work consistently? It does not in
this case. If there is anything to be commited in the index, then the
"git add" from within the pre-commit hook *is* respected in this commit.
If there is *nothing* to be commited, except from what was added by the
pre-commit exit, then it is ignored *for this commit*, but it is added
and will be commited next time. This is inconsistent behaviour.

- if the decision will be *not* to allow adding from within a pre-commit
hook, then the "git add" should be rejected. And the documentation
should say so.

I'll have to understand what all the comments from you and others mean
(still a git newbee).

In general, you want to avoid adding automatically generated files to
your repository.  That tends to bloat the repository needlessly and is a
great way to lead to frequent, hard-to-resolve merge conflicts.

I'm trying to use git to keep track of changes to my website, which uses
some framework. The problem is: Some changes modify filesystem entities,
only, while other changes modify the content of some mysql tables. So I
need to pack an unload of those tables into the commit. I can do this
manually (and forget every now and then :-), or I thought the pre-commit
hook would be a good place to automate this. And, yes, it may well crete
merge conflicts.

I understand the pre-commit hook is a local thing, and I have to make
sure the same is active in all repositories. Definitely not something to
use in a widely shared project.

Thanks a lot
Peter

Pardon my ignorance, but I'm unlear as to how to proceed further with my
issue.
What is the proper process to report a bug, and get a consense whether
it is accepted or rejected?

Regards
Peter

Am 28.10.2021 um 14:08 schrieb Peter Hunkeler:

Am 28.10.2021 um 00:07 schrieb brian m. carlson:

quoted

I should point out here that it isn't intended for pre-commit hooks to
be used this way; they're intended to verify that the commit meets some
standards, not to modify it, although it is of course possible to do.

I can accept that comment. However:

- wouldn't you agree that git should work consistently? It does not in
this case. If there is anything to be commited in the index, then the
"git add" from within the pre-commit hook *is* respected in this commit.
If there is *nothing* to be commited, except from what was added by the
pre-commit exit, then it is ignored *for this commit*, but it is added
and will be commited next time. This is inconsistent behaviour.

- if the decision will be *not* to allow adding from within a pre-commit
hook, then the "git add" should be rejected. And the documentation
should say so.

I'll have to understand what all the comments from you and others mean
(still a git newbee).

quoted

In general, you want to avoid adding automatically generated files to
your repository.  That tends to bloat the repository needlessly and is a
great way to lead to frequent, hard-to-resolve merge conflicts.

I'm trying to use git to keep track of changes to my website, which uses
some framework. The problem is: Some changes modify filesystem entities,
only, while other changes modify the content of some mysql tables. So I
need to pack an unload of those tables into the commit. I can do this
manually (and forget every now and then :-), or I thought the pre-commit
hook would be a good place to automate this. And, yes, it may well crete
merge conflicts.

I understand the pre-commit hook is a local thing, and I have to make
sure the same is active in all repositories. Definitely not something to
use in a widely shared project.

Thanks a lot
Peter

Am 30.10.21 um 15:21 schrieb Peter Hunkeler:

Pardon my ignorance, but I'm unlear as to how to proceed further with my
issue.
What is the proper process to report a bug, and get a consense whether
it is accepted or rejected?

Writing a message to this mailing list is all the process that exists.
You brought forward arguments in a civil manner why you think the
current behavior is not correct. That's appreciated. However, it doesn't
automatically mean that something will be changed.

In your particular case...

Am 28.10.2021 um 14:08 schrieb Peter Hunkeler:

quoted

Am 28.10.2021 um 00:07 schrieb brian m. carlson:

quoted

I should point out here that it isn't intended for pre-commit hooks to
be used this way; they're intended to verify that the commit meets some
standards, not to modify it, although it is of course possible to do.

I can accept that comment. However:

- wouldn't you agree that git should work consistently? It does not in
this case. If there is anything to be commited in the index, then the
"git add" from within the pre-commit hook *is* respected in this commit.
If there is *nothing* to be commited, except from what was added by the
pre-commit exit, then it is ignored *for this commit*, but it is added
and will be commited next time. This is inconsistent behaviour.

- if the decision will be *not* to allow adding from within a pre-commit
hook, then the "git add" should be rejected. And the documentation
should say so.

... it is clearly stated (and brian repeated it) that the pre-commit
hook is intended to check the commit for "correctness" (whatever that
means for the project). `git add` is not automatically forbidden,
because it would be an unreasonable engineering effort to forbid things
that "do not merely check" the commit. (What if a user has to use `git
add` as part of some exotic check? For example, it is possible to `git
add` to some temporary throw-away index that is different from the one
that is about to be committed.)

Even though the documentation does not say explicitly that the commit
must not be changed, it is implicit in the stated intent (that the
commit is only checked). Depending on that some particular behavior
works for you sometimes is then your own business, and when it breaks
you get to keep both parts.

In conclusion, the pre-commit hook behaves as designed and nothing has
to be changed.

-- Hannes