Re: git stash takes excessively long when many untracked files present

Josh Triplett <josh <at> joshtriplett.org> writes:

[CCing folks involved in the recent "stash-refuse-to-kill" merge.]

I keep portions of my home directory in git.  I tried to "git stash"
some local changes, and it ran for several minutes with no progress.  ps
showed that it was running "git ls-files --killed", which was taking
100% CPU, and occasionally reading the disk very slowly.

I've recently got the same problem, though in this case it's my 
openembedded directory that's giving me those problems. (Having an 
untracked build-directory of quiet a few GB takes some time).

I worked around it by locally patching git-stash:
-------------------------------------------

diff --git a/git-stash.sh b/git-stash.sh
index 85c9e2c..e5a2043 100755
--- a/git-stash.sh
+++ b/git-stash.sh

@@ -263,7 +263,7 @@ save_stash () {
                exit 0
        fi
        if test -z "$untracked$force" &&
-          test -n "$(git ls-files --killed | head -n 1)"
+          test -n "$(git ls-files --killed --directory | head -n 1)"
        then
                say "$(gettext "The following untracked files would NOT be 

saved
                test -n "$GIT_QUIET" || git ls-files --killed | sed 
's/^/\t/'
-------------------------------------------

It seems to work in my extremely limited testing. Though, I'm pretty sure 
that there'll be quite a few error cases... (Especially, as I just made
a naive attempt at patching git-stash, so I could go on with a few other 
things).

Do anyone have any better idea on how to approach this?

strace shows that git ls-files --killed is doing a full recursive
enumeration of my entire home directory.  That's a Really Bad Idea:

~$ find | wc -l
3248997
~$ find -type d | wc -l
350680

Not only that, but it also appears to be attempting to stat and open
several files in every single directory; for instance:

stat(".ccache/1/3/.git", 0x7fff254bc7a0) = -1 ENOENT (No such file or 

directory)

open(".ccache/1/3/.git/HEAD", O_RDONLY) = -1 ENOENT (No such file or 

directory)

stat(".ccache/1/3/.git", 0x7fff254bc770) = -1 ENOENT (No such file or 

directory)

open(".ccache/1/3/.git/packed-refs", O_RDONLY) = -1 ENOENT (No such file 

or directory)

(Yes, in that order.)

I see a lot of room for optimization here.  Most importantly, git
ls-files --killed really doesn't need to look at any directory entry
unless something in the index would conflict with it.

I guess that this would be a good optimization. Or, are ls-files --killed 
used in other cases where the current behaviour would be requiered?

Cheers,
Anders