Re: [PATCH v2 2/2] index-pack: support multithreaded delta resolving

Nguyễn Thái Ngọc Duy [off-list ref] writes:

This puts delta resolving on each base on a separate thread, one base
cache per thread. Per-thread data is grouped in struct thread_local.
When running with nr_threads == 1, no pthreads calls are made. The
system essentially runs in non-thread mode.

As discussed when we took the git-grep measurements, it may be
interesting to have a way to run 1 thread.  Can you put in such an
option?

An experiment on a Xeon 24 core machine with linux-2.6.git shows that
performance does not increase proportional to the number of cores. So
by default, we use maximum 3 cores. Some numbers with --threads from 1
to 16:

1..4
real    1m16.310s  0m48.183s  0m37.866s  0m32.834s
user    1m13.773s  1m15.537s  1m15.781s  1m16.233s
sys     0m2.480s   0m3.936s   0m4.448s   0m4.852s

5..8
real    0m33.170s  0m30.369s  0m28.406s  0m26.968s
user    1m31.474s  1m30.322s  1m29.562s  1m28.694s
sys     0m6.096s   0m6.268s   0m6.684s   0m7.172s

Interesting.  Is this a real 24-core machine or 12*2 hyperthreaded?
Does it use Turbo Boost and how far (how fast and on how many cores
simultaneously) does that go?

I'm asking because if Turbo Boost starts to wear off around 4 cores,
like these measurements suggest, then it may not be beneficial to spawn
threads on 2*2HT CPUs (found in many laptops) where Turbo Boost only
really works if you only use a single core.

Oh, and could you write a perf test for this? :-)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch