Linus Torvalds [off-list ref] writes:
Wouldn't it be lovely if it was slightly smarter (something more akin
to the index preloading that takes number of files into account) or at
least allowed people to set the parallelism explicitly with a command
line switch?
Yeah, a reasonable starting point for auto-tuning may be to use the
same kind of parameters and heuristics (i.e. max parallel of 20
threads, assume a cost to use an extra thread is the same as running
500 greps), and then tweak them (for example, thread cost of 500 may
be reasonable for lstat() but it would be way too big for grep()).
The real issue may be that we do not have a good estimate of how
many paths are involved in the request before starting these
threads, though.
On Fri, Apr 26, 2013 at 11:47 AM, Junio C Hamano [off-list ref] wrote:
The real issue may be that we do not have a good estimate of how
many paths are involved in the request before starting these
threads, though.
Yes. Also, I'm not sure if the 15% possible improvement on my SSD case
is even worth it for something that in the end isn't necessarily the
common case. I *suspect* that it might be a much bigger deal on NFS
(IO parallelism really does end up being a big deal sometimes, and
caching tends to be less aggressive too), but on rotational media it
might be much less clear, or even a loss..
Are there people out there who use "git grep" over NFS and have been
unhappy with performance? If are willing to recompile git with a
different THREAD value in builtin/grep.c, then on a Linux client you
can try
echo 3 > /proc/sys/vm/drop_caches
to largely force cold-cache behavior for testing (I say "largely",
because it won't drop busy/dirty pages, but for "git grep" kind of
loads it should be good).
Of course, you need root for it, so..
Linus