Ramkumar Ramachandra [off-list ref] writes:
$ ~/src/git
error: object file
.git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f is empty
error: object file
.git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f is empty
fatal: loose object 8e6a6dda24b017915449897fcc1353a9b848fd2f (stored
in .git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f) is corrupt
So fsync() and close() thought that the filesystem stored this loose
object safely, but it turns out that the data is not on disk.
artagnon|remote-cruft*+:~/src/git$ rm
.git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f
As you know it is empty, removing (as long as you do not forget what
the object name was, which may later become useful when untangling
the mess further) does not hurt very much.
artagnon|remote-cruft*+:~/src/git$ git prune
artagnon|remote-cruft*+:~/src/git$ git status
fatal: bad object HEAD
fatal: bad object HEAD
And the value in the HEAD was???
artagnon|remote-cruft*+:~/src/git$ git symbolic-ref HEAD refs/heads/master
artagnon|master*+=:~/src/git$ git status
## master
MM Documentation/git-ls-remote.txt
MM remote.c
MM t/t5505-remote.sh
MM t/t5510-fetch.sh
MM t/t5515-fetch-merge-logic.sh
MM t/t5516-fetch-push.sh
?? lib/
?? outgoing/
That status is completely bogus, by the way.
... which may suggest that your index file may have been corrupted
on the filesystem.
artagnon|master*+=:~/src/git$ git reset --hard
... and using that known-to-be-corrupt index, the working tree state
is discarded.
artagnon|master*+=:~/src/git$ git checkout remote-cruft
fatal: reference is not a tree: remote-cruft
artagnon|master=:~/src/git$ git reflog
21ff915 HEAD@{10 minutes ago}: rebase -i (finish): returning to
refs/heads/remote-cruft
What happened to the rest of my reflog?!
On the filesystem known to not record the last consistent state of
the repository, the answer to that question may be rather obvious,
no?
artagnon|master=:~/src/git$ git branch -D remote-cruft
error: Couldn't look up commit object for 'refs/heads/remote-cruft'
The command would want to report what was at the tip, so it is
understandable it may want to look up that commit before removing
the ref.
artagnon|master=:~/src/git$ rm .git/refs/heads/remote-cruft
artagnon|master=:~/src/git$ git checkout -b remote-cruft
Switched to a new branch 'remote-cruft'
Huh? What happened to my upstream?
artagnon|remote-cruft:~/src/git$ git branch -u ram/remote-cruft
warning: ignoring broken ref refs/remotes/ram/remote-cruft.
So remotes/ram/remote-cruft is also broken.
Fine, let's fetch.
Why?
"fetch" walks the ancestry graph on both ends to minimize transfers.
It's not something you would expect to work when you know refs at
your end does not even record what you do have. It _may_ appear to
work if your refs are intact but you are missing objects, as they
will not be transferred again from the good copy if you let your
repository's ref claim that you have _all_ objects behind it when
you actually don't.
What would have been a better starting point to untangle is to make
a separate clone, pretending as if this repository did not even
exist, and copy the resulting packfile into this repository. That
would at least give you a known good copies of objects that you
already have pushed out.
And the next step would have been (without doing any of the above
"remove this branch, recreate this one anew") to compare the tips
of refs in this broken repository and the clone. The same ones you
can trust, and different ones you dig further.
Was I being stupid, or is fixing corrupted repositories really this
non-trivial? Comments appreciated.
I think "Let's fetch first" was the step that took you in a wrong
direction that requires unnecessary work.
Junio C Hamano wrote:
quoted
$ ~/src/git
error: object file
.git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f is empty
error: object file
.git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f is empty
fatal: loose object 8e6a6dda24b017915449897fcc1353a9b848fd2f (stored
in .git/objects/8e/6a6dda24b017915449897fcc1353a9b848fd2f) is corrupt
So fsync() and close() thought that the filesystem stored this loose
object safely, but it turns out that the data is not on disk.
Where should I start digging if I want to fix this? Actually you just
need to tell me how to build reduced-case corruptions to test: I can
trace and figure out the rest.
quoted
artagnon|remote-cruft*+:~/src/git$ git prune
artagnon|remote-cruft*+:~/src/git$ git status
fatal: bad object HEAD
fatal: bad object HEAD
And the value in the HEAD was???
ref: refs/heads/remote-cruft. That's why I included my prompt :)
quoted
artagnon|remote-cruft*+:~/src/git$ git symbolic-ref HEAD refs/heads/master
artagnon|master*+=:~/src/git$ git status
## master
MM Documentation/git-ls-remote.txt
MM remote.c
MM t/t5505-remote.sh
MM t/t5510-fetch.sh
MM t/t5515-fetch-merge-logic.sh
MM t/t5516-fetch-push.sh
?? lib/
?? outgoing/
That status is completely bogus, by the way.
... which may suggest that your index file may have been corrupted
on the filesystem.
Yeah, my question pertains to why is the index half-corrupted. Is
there no checksum to say "index corrupted; do not display bogus
nonsense"?
quoted
artagnon|master*+=:~/src/git$ git checkout remote-cruft
fatal: reference is not a tree: remote-cruft
artagnon|master=:~/src/git$ git reflog
21ff915 HEAD@{10 minutes ago}: rebase -i (finish): returning to
refs/heads/remote-cruft
What happened to the rest of my reflog?!
On the filesystem known to not record the last consistent state of
the repository, the answer to that question may be rather obvious,
no?
I didn't understand. What does .git/logs/HEAD have to do with any of
this? Why is it truncated?
quoted
artagnon|master=:~/src/git$ git branch -D remote-cruft
error: Couldn't look up commit object for 'refs/heads/remote-cruft'
The command would want to report what was at the tip, so it is
understandable it may want to look up that commit before removing
the ref.
I would have expected it to display a warning and remove the ref
anyway. Or error out, and override with a force-flag?
quoted
Fine, let's fetch.
Why?
"fetch" walks the ancestry graph on both ends to minimize transfers.
It's not something you would expect to work when you know refs at
your end does not even record what you do have. It _may_ appear to
work if your refs are intact but you are missing objects, as they
will not be transferred again from the good copy if you let your
repository's ref claim that you have _all_ objects behind it when
you actually don't.
Right. I expected it to figure out that I have a broken history and
fetch everything (which is what happened the second time, no?).
What would have been a better starting point to untangle is to make
a separate clone, pretending as if this repository did not even
exist, and copy the resulting packfile into this repository. That
would at least give you a known good copies of objects that you
already have pushed out.
Yeah, I deliberately avoided doing that: apart from the config and
refs, I had no real unpushed work in ~/src/git anyway (I push _very_
frequently, which explains my "resolve HEAD early in current" patch).
The most important part of what I did was running unpack-objects by
hand: that fixed everything. I shouldn't have had to run that by hand
though: why isn't there an in-built way to unpack everything, remove
corruptions, and repack the good stuff?
And the next step would have been (without doing any of the above
"remove this branch, recreate this one anew") to compare the tips
of refs in this broken repository and the clone. The same ones you
can trust, and different ones you dig further.
Right. I didn't have local data in this case, so I didn't bother.
quoted
Was I being stupid, or is fixing corrupted repositories really this
non-trivial? Comments appreciated.
I think "Let's fetch first" was the step that took you in a wrong
direction that requires unnecessary work.
This was mainly a learning exercise for me: I wanted to see how good
git was at working with corrupted repositories. I did my surgery
fairly quickly, and avoided large network transfers (I have a slow
connection).