Re: [PATCH v6] Add git-grep threads param

On Mon, Nov 16, 2015 at 8:11 AM, Victor Leschuk
[off-list ref] wrote:

"git grep" can now be configured (or told from the command line)
 how many threads to use when searching in the working tree files.

 Changes to default behavior: number of threads now doesn't depend
 on online_cpus(), e.g. if specific number is not configured
 GREP_NUM_THREADS_DEFAULT (8) threads will be used even on 1-core CPU.

It's good that this behavior change (no longer consulting
online_cpus()) is now mentioned[1] in the commit message, however,
it's also important for people to understand why this change was made,
but such explanation is missing here. It's important to justify the
change in the commit message itself since only a few people were
involved in the mailing list discussion which led to the change, and
even those people may forget the reasoning at some point.

Moreover, this change (dropping online_cpus()) is sufficiently
significant and distinct from the overall purpose of the present patch
(adding --threads and 'grep.threads') that it really deserves its own
separate patch (as hinted by [1]), which would turn this into a
2-patch series.

[1]: http://article.gmane.org/gmane.comp.version-control.git/281133/

quoted hunk

Signed-off-by: Victor Leschuk <redacted>
---

diff --git a/Documentation/config.txt b/Documentation/config.txt

@@ -1447,6 +1447,13 @@ grep.extendedRegexp::
+grep.threads::
+       Number of grep worker threads, use it to tune up performance on
+       your machines. Leave it unset (or set to 0) for default behavior,
+       which for now is using 8 threads for all systems.

You could drop "for now", which is redundant with the sentence which follows.

+       Default behavior can be changed in future versions
+       to better suit hardware and circumstances.

"may change" might read better than "can be changed".

quoted hunk

 gpg.program::
        Use this custom program instead of "gpg" found on $PATH when
        making or verifying a PGP signature. The program must support the

diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt

@@ -53,6 +54,13 @@ grep.extendedRegexp::
+grep.threads::
+       Number of grep worker threads, use it to tune up performance on
+       your machines. Leave it unset (or set to 0) for default behavior,
+       which for now is using 8 threads for all systems.
+       Default behavior can be changed in future versions
+       to better suit hardware and circumstances.

Same comments as above.

quoted hunk

@@ -227,6 +235,13 @@ OPTIONS
+--threads <num>::
+       Number of grep worker threads, use it to tune up performance on
+       your machines. Leave it unset (or set to 0) for default behavior,
+       which for now is using 8 threads for all systems.
+       Default behavior can be changed in future versions
+       to better suit hardware and circumstances.

Ditto.

Would it make more sense just to have the detailed description in one
place and then reference it from the other places rather than
repeating in all places?

quoted hunk

diff --git a/builtin/grep.c b/builtin/grep.c

@@ -262,10 +265,19 @@ static int wait_all(void)
+static int grep_threads_config(const char *var, const char *value, void *cb)
+{
+       if (!strcmp(var, "grep.threads"))
+               num_threads = git_config_int(var, value); /* Sanity check of value will be perfomed later */

Nit: This is a pretty wide line. Perhaps the comment could be moved to
its own line to stay below 80 columns?

+       return 0;
+}
+
 static int grep_cmd_config(const char *var, const char *value, void *cb)
 {
        int st = grep_config(var, value, cb);
-       if (git_color_default_config(var, value, cb) < 0)
+       if (grep_threads_config(var, value, cb) < 0)
+               st = -1;
+       else if (git_color_default_config(var, value, cb) < 0)
                st = -1;

Nit: Custom is to add new cases following existing ones, which would
give you a less noisy diff.

quoted hunk

        return st;
 }

@@ -832,14 +846,18 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
        }

 #ifndef NO_PTHREADS
-       if (list.nr || cached || online_cpus() == 1)
-               use_threads = 0;
+       if (list.nr || cached)
+               num_threads = 0; /* Can not multi-thread object lookup */
+       else if (num_threads == 0)
+               num_threads = GREP_NUM_THREADS_DEFAULT; /* User didn't specify value, or just wants default behavior */

Nit: Overly wide line.

+       else if (num_threads < 0)
+               die("Invalid number of threads specified (%d)", num_threads);

If you validate the value earlier, rather than delaying it until here,
you can give a more useful error message which would state whence the
bad value arose (config vs. command-line). This may or may not be a
worthwhile improvement.

 #else
-       use_threads = 0;
+       num_threads = 0;
 #endif

 #ifndef NO_PTHREADS
-       if (use_threads) {
+       if (num_threads) {
                if (!(opt.name_only || opt.unmatch_name_only || opt.count)
                    && (opt.pre_context || opt.post_context ||
                        opt.file_break || opt.funcbody))