File extension conflict when working with git and latex

Hi,

I struggled with a really nasty issue today (and yesterday):

I work on a semester project paper which is written in latex and
therefor version-controlled with git. We compile the document using
pdflatex, we automate the compiling with make.

pdflatex generates some temporary files, which are required if you
call it the second and third time (it is a top-down compiler, which
needs to be called several times). One of the file extensions is
".idx", which (kind of) conflicts with file extensions in .git/.

Unfortunetely, we wrote our `make clean` task recursively. I think you
can imagine what went wrong: The clean-task corrupted the repository,
as it removed .idx files from within .git/.

I lost work because of this ugly name collision.

My question now:

I know, I can fix this by fixing the clean task in my Makefile. But
maybe someone somewhere on this world doesn't know the git internals
as good as me (and, of course, my coworker). Is there _any chance
at all_ that this gets mentioned somewhere, so others don't fall into
this pit?

-- 
Mit freundlichen Grüßen,
Kind regards,
Matthias Beyer

Proudly sent with mutt.
Happily signed with gnupg.

On Fri, Mar 21, 2014 at 05:13:24PM +0100, Matthias Beyer wrote:

Unfortunetely, we wrote our `make clean` task recursively. I think you
can imagine what went wrong: The clean-task corrupted the repository,
as it removed .idx files from within .git/.

I lost work because of this ugly name collision.

Hello Matthias,

You can recreate the .idx files by running

    git index-pack .git/objects/pack/pack-<hash>.pack

for each pack file.

Regards
Simon
-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

On 21-03-2014 17:36:28, Simon Ruderich wrote:

On Fri, Mar 21, 2014 at 05:13:24PM +0100, Matthias Beyer wrote:

quoted

Unfortunetely, we wrote our `make clean` task recursively. I think you
can imagine what went wrong: The clean-task corrupted the repository,
as it removed .idx files from within .git/.

I lost work because of this ugly name collision.

Hello Matthias,

You can recreate the .idx files by running

    git index-pack .git/objects/pack/pack-<hash>.pack

for each pack file.

Hi Simon,

I think so. I executed:

    git fsck # reports N missing blobs, commits, trees and dangling stuff
    git index-pack ...
    git fsck # reports only dangling commits and blobs

I don't know if this means that the repository is fixed now?

-- 
Mit freundlichen Grüßen,
Kind regards,
Matthias Beyer

Proudly sent with mutt.
Happily signed with gnupg.

On Fri, Mar 21, 2014 at 05:46:51PM +0100, Matthias Beyer wrote:

Hi Simon,

I think so. I executed:

    git fsck # reports N missing blobs, commits, trees and dangling stuff
    git index-pack ...
    git fsck # reports only dangling commits and blobs

I don't know if this means that the repository is fixed now?

Hello Matthias,

If nothing else was deleted, then yes, the repository should be
fine now.

Regards
Simon
-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

On 2014-03-21 17.13, Matthias Beyer wrote:

Hi,

I struggled with a really nasty issue today (and yesterday):

I work on a semester project paper which is written in latex and
therefor version-controlled with git. We compile the document using
pdflatex, we automate the compiling with make.

pdflatex generates some temporary files, which are required if you
call it the second and third time (it is a top-down compiler, which
needs to be called several times). One of the file extensions is
".idx", which (kind of) conflicts with file extensions in .git/.

Unfortunetely, we wrote our `make clean` task recursively. I think you
can imagine what went wrong: The clean-task corrupted the repository,
as it removed .idx files from within .git/.

I lost work because of this ugly name collision.

My question now:

I know, I can fix this by fixing the clean task in my Makefile. But
maybe someone somewhere on this world doesn't know the git internals
as good as me (and, of course, my coworker). Is there _any chance
at all_ that this gets mentioned somewhere, so others don't fall into
this pit?

The idx files are mentioned here:
http://www.git-scm.com/book/en/Git-Internals-Packfiles

(Otherwise, some random thoughts:)
- stay out of the .git directory, whatever you do
   (I lost some data because my faulty scripts using sed to replace
   "s/this/that/" and went into the .git directory) 
- Don't delete "read-only" files
   (The .idx files are all  "r--r--r--")
  Or in other words, do not use "rf -f" in a script, unless you know exactly
  what you and the script are doing.
- Replace "rm -f" whith "mv /tmp" could be an option
- As mentioned before, "git clean" can be an option,
  I used it once when the "make clean", "make distclean", "make reallydistclean"
  did not work as expected by me,
  some auto-generated config files where still not deleted. 
- Make backups of your repo (frequently)
- push the repo to a backup repo (frequently)
- Use $GIT_DIR to move the repo out of your work tree:
  https://www.kernel.org/pub/software/scm/git/docs/git.html

- Prepare a patch that renames the *.idx files used by Git into
  *.gitidx, *.gidx, *.packidx, (and the *.pack into *.packdata ??) 

Hopefully you did not loose too much data, and if you have an idea how to improve
things, you may consider sending a patch, improvements are possible and welcome.