Re: [PATCH v2 3/3] grep: disable threading in all but worktree case

On Sat, Dec 24, 2011 at 2:07 PM, Jeff King [off-list ref] wrote:

I tried to get some timings for this, but ran across some quite
surprising results. Here's a simple grep of the linux-2.6 working tree,
using a single-threaded grep:

 $ time git grep SIMPLE >/dev/null
 real    0m0.439s
 user    0m0.272s
 sys     0m0.160s

and then the same thing, via xargs, without even turning on
parallelization. This should give us a measurement of the overhead for
going through xargs at all. We'd expect it to be slower, but not too
much so:

 $ time git ls-files | xargs git grep SIMPLE -- >/dev/null
 real    0m11.989s
 user    0m11.769s
 sys     0m0.268s

Twenty-five times slower! Running 'perf' reports the culprit as pathspec
matching:

 +  63.23%    git  git                 [.] match_pathspec_depth
 +  28.60%    git  libc-2.13.so        [.] __strncmp_sse42
 +   2.22%    git  git                 [.] strncmp@plt
 +   1.67%    git  git                 [.] kwsexec

where the strncmps are called as part of match_pathspec_depth. So over
90% of the CPU time is spent on matching the pathspecs, compared to less
than 2% actually grepping.

Which really makes me wonder if our pathspec matching could stand to be
faster. True, giving a bunch of single files is the least efficient way
to use pathspecs, but that's pretty amazingly slow.

We could eliminate get_pathspec_depth() in grep_directory() when
read_directory() learns to filter path properly using (and at the cost
of) tree_entry_interesting(). The latter function has more
optimizaions built in and should be faster than the former. This is a
good test case for my read_directory() rewrite. Thanks.

get_pathspec_depth() can still use some optimizations though for
grep_cache() case.
-- 
Duy