[PATCH] grep: detect number of CPUs for thread spawning
From: Ævar Arnfjörð Bjarmason <hidden>
Date: 2016-06-15 22:52:23
Subsystem:
the rest · Maintainer:
Linus Torvalds
From: Eric Herman <redacted> Change the number of threads that we spawn from a hardcoded value of "8" to what online_cpus() returns. Back in v1.7.0-rc1~19^2 when threaded grep was introduced the number of threads was hardcoded at compile time to 8, but this value was only used if online_cpus() returned greater than 1. However just using 8 threads regardless of the actual number of CPUs is inefficient if we have more than 8 CPUs, and potentially wasteful if we have fewer than 8 CPUs. The array holding the threads is now being allocated at runtime instead of being allocated statically. We free the array further down the line in the wait_all() function; this is OK since the allocation and freeing is guarded by the global "use_threads" variable, which is based on the return value of online_cpus(). Reviewed-by: Sverre Rabbelier <redacted> Reviewed-by: Fernando Vezzosi <redacted> Signed-off-by: Eric Herman <redacted> Signed-off-by: Ævar Arnfjörð Bjarmason <redacted> --- builtin/grep.c | 17 +++++++++++------ 1 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 3d7329d..307a253 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c@@ -27,8 +27,8 @@ static char const * const grep_usage[] = { static int use_threads = 1; #ifndef NO_PTHREADS -#define THREADS 8 -static pthread_t threads[THREADS]; +static int nthreads; +static pthread_t *threads; static void *load_sha1(const unsigned char *sha1, unsigned long *size, const char *name);
@@ -36,7 +36,7 @@ static void *load_file(const char *filename, size_t *sz); enum work_type {WORK_SHA1, WORK_FILE}; -/* We use one producer thread and THREADS consumer +/* We use one producer thread and online_cpus() consumer * threads. The producer adds struct work_items to 'todo' and the * consumers pick work items from the same array. */
@@ -254,6 +254,8 @@ static void start_threads(struct grep_opt *opt) { int i; + threads = xmalloc(nthreads * sizeof(pthread_t)); + pthread_mutex_init(&grep_mutex, NULL); pthread_mutex_init(&read_sha1_mutex, NULL); pthread_cond_init(&cond_add, NULL);
@@ -264,7 +266,7 @@ static void start_threads(struct grep_opt *opt) strbuf_init(&todo[i].out, 0); } - for (i = 0; i < ARRAY_SIZE(threads); i++) { + for (i = 0; i < nthreads; i++) { int err; struct grep_opt *o = grep_opt_dup(opt); o->output = strbuf_out;
@@ -295,7 +297,7 @@ static int wait_all(void) pthread_cond_broadcast(&cond_add); grep_unlock(); - for (i = 0; i < ARRAY_SIZE(threads); i++) { + for (i = 0; i < nthreads; i++) { void *h; pthread_join(threads[i], &h); hit |= (int) (intptr_t) h;
@@ -307,6 +309,8 @@ static int wait_all(void) pthread_cond_destroy(&cond_write); pthread_cond_destroy(&cond_result); + free(threads); + return hit; } #else /* !NO_PTHREADS */
@@ -1001,7 +1005,8 @@ int cmd_grep(int argc, const char **argv, const char *prefix) opt.regflags |= REG_ICASE; #ifndef NO_PTHREADS - if (online_cpus() == 1 || !grep_threads_ok(&opt)) + nthreads = online_cpus(); + if (nthreads == 1 || !grep_threads_ok(&opt)) use_threads = 0; if (use_threads) {
--
1.7.7