Re: epic fsck SIGSEGV! (was Recovering from epic fail (deleted .git/objects/pack))

On Wed, 2008-12-10 at 15:40 -0800, Linus Torvalds wrote:

Wow. You even got _gdb_ to segfault.

You're my hero. If it can break, you will do it.

You have no idea :) So much so that a coworker got me a "FAIL" stamp for
my birthday:
http://agentdero.cachefly.net/pictotweet.com//saved/6f217a5ababb06185d5e4ca1398e743c/PIC-012835841677481.jpg )

Anyways..

That stupid fsck commit walker walks the parents recursively. That's 
horribly bogus. So you have a recursion that goes from the top-level 
commit all the way to the root, doing

	fsck_walk_commit -> walk(parent) -> fsck_walk-commit -> ..

and you have a fairly deep commit tree. 

This repository is ~3 years old and ~7.1GB small, when we finally cut
over from Subversion we were in the 130,000 revision range. 

Anyway, with a 8M stack-size I can fsck the kernel repo without any 
problem, but while the kernel repo has something like 120k commits in it, 
it's a very "bushy" repository (lots of parallelism and merges), and the 
path from the top parent to the root is actually much shorter, at just 27k 
commits.

The stack size is 8M as you assumed, I'm curious as to how the kernel
handles a process that exceeds the ulimit(2) stacksize. I know from our
experience with this repository that when Git runs up against the
address space (ulimit -v) that an ENOMEM or something similar is
returned. Is there an E_NOSTACK? :) (figured I'd ask, given your
apparent knowledge on the subject ;))

I take it that your project has a very long and linear history, which is 
why you have a long path from your HEAD to your root.

(You can do something like

	git rev-list --first-parent HEAD | wc -l

tyler@ccnet:~/source/slide/brian_main>  git rev-list --first-parent HEAD
| wc -l
46751 
tyler@ccnet:~/source/slide/brian_main> uname -a
Linux ccnet 2.6.25.18-0.2-default #1 SMP 2008-10-21 16:30:26 +0200
x86_64 x86_64 x86_64 GNU/Linux
tyler@ccnet:~/source/slide/brian_main> git --version
git version 1.6.0.2

But we should definitely fix this braindamage in fsck. Rather than 
recursively walk the commits, we should add them to a commit list and just 
walk the list iteratively.

Given that this issue affects our internal (proprietary) repository, I
can't very well give access to it or publish a clone, but I'm willing to
help in any way I can. We maintain an internal fork of the Git tree, so
I can apply any changes you'd like to an internal 1.6.0.4 or 1.6.0.5
build. For obvious reasons I ran the fsck against an upstream maintained
(stable) build of Git.


Cheers


p.s. If you find yourself in downtown San Francisco, we'd be honored to
buy you a drink here at Slide :)
-- 
-R. Tyler Ballance
Slide, Inc.