Re: git performance after directory copy

"Gaer, A." [off-list ref] writes:

On a copied source directory "git status" seems to be about
3 times slower than on the original directory.

It is expected that the first "git status" be slower. It will most
likely have to actually re-diff the files and update the index
stat-cache.

But I'm surprised that the next "git status" are still slow. Other
people may get a better explanation, but this very much looks like a
bug.

Are you sure you don't have any permission problem, like a read-only
.git/index?

To investigate a bit, you can play with diff.autorefreshindex. When
set to false, "git diff" will tell you about the files which are
identical, but do not have the same stat information (and hence, that
Git has to re-diff). For example:

$ git config diff.autorefreshindex false
$ git diff

# no output: bar.txt exists, but is up to date.

$ touch bar.txt
$ git diff     

diff --git a/bar.txt b/bar.txt

# No actual diff, but the file appears since its stat information is
# different.

$ git diff

diff --git a/bar.txt b/bar.txt

# Further "git diff" behave the same.

$ git status
...
$ git diff  

# git status did update the stat-cache, hence, no output from git diff
# anymore.

Another diagnosis tool would be "strace -e open git status,lstat64".
For example:

$ touch bar.txt 
$ strace -e open,lstat64 git status |& grep bar.txt
lstat64("bar.txt", {st_mode=S_IFREG|0644, st_size=30, ...}) = 0
open("bar.txt", O_RDONLY|O_LARGEFILE)   = 3
$ strace -e open,lstat64 git status |& grep bar.txt
lstat64("bar.txt", {st_mode=S_IFREG|0644, st_size=30, ...}) = 0

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Matthieu Moy venit, vidit, dixit 20.09.2010 11:56:

"Gaer, A." [off-list ref] writes:

quoted

On a copied source directory "git status" seems to be about
3 times slower than on the original directory.

It is expected that the first "git status" be slower. It will most
likely have to actually re-diff the files and update the index
stat-cache.

But I'm surprised that the next "git status" are still slow. Other
people may get a better explanation, but this very much looks like a
bug.

In the OP's case, the inode/dev info differs (after copying, or after
moving across FS boundaries). I don't think "git status" updates these.
I noticed (and reported) something like this a while ago when, after a
reboot, I would get warnings about crossing FS boundaries. Only a "rm
.git/index && git reset --hard" was able to refresh the index properly.
This case seems to be similar, although a simple "git reset --hard"
appears to suffice.

Michael

Hello all,

thanks Matthieu for your tip. "git diff" without "autorefreshindex"
shows that all files have different stat info. Calling "git update-index
--refresh" seems to be the right cure.

What I find a little bit confusing is that "git status" does not warn
about such a situation, or even "repair" it. As I said, I wanted to move
the project directories of several developers to a new partition without
too much interference and the first try didn't succeed because people
complained about "bad" "git status" performance (people get used to git
rocket-fast performance very soon ;-).

Maybe this was introduced in the 1.7 release: "git status" is not "git
commit --dry-run" anymore. "git commit --dry-run" does behave as you
expect: first call takes a little bit longer, subsequent calls are fast
again.

BTW, I tested on a system with git version 1.7.1 installed, but release
notes do not suggest any changes in that respect in 1.7.2 or 1.7.3.

Regards,
 Andreas.

Am 9/20/2010 11:56, schrieb Matthieu Moy:

But I'm surprised that the next "git status" are still slow. Other
people may get a better explanation, but this very much looks like a
bug.

Most likely, Andreas works with 1.7.1. From the release notes of 1.7.1.1:

 * "git status" stopped refreshing the index by mistake in 1.7.1.

-- Hannes

Hello Johannes,

you are right. Sorry. I totally overlooked release notes for 1.7.1.1. Thanks for all your help!

Regards,
 Andreas.
 

-----Ursprüngliche Nachricht-----
Von: Johannes Sixt [mailto:j.sixt@viscovery.net] 
Gesendet: Montag, 20. September 2010 13:48
An: Matthieu Moy
Cc: Gaer, A.; git@vger.kernel.org
Betreff: Re: git performance after directory copy

Am 9/20/2010 11:56, schrieb Matthieu Moy:

But I'm surprised that the next "git status" are still slow. Other
people may get a better explanation, but this very much looks like a
bug.

Most likely, Andreas works with 1.7.1. From the release notes of 1.7.1.1:

 * "git status" stopped refreshing the index by mistake in 1.7.1.

-- Hannes