Re: Better value for chunk_size when threaded

On 12/6/07, Nicolas Pitre [off-list ref] wrote:

On Thu, 6 Dec 2007, Jon Smirl wrote:

quoted

I tried some various ideas out for chunk_size and the best strategy I
found was to simply set it to a constant. How does 20,000 work on
other CPUs?

That depends on the object size.  If you have a repo with big objects
but only 1000 of them for example, then the constant doesn't work.

How about defaulting it to 20,000 and allowing an override? It's not
fatal if we guess wrong, we just want to most common cases to work out
of the box. 20,000 is definitely better than the current window *
1000.

Ideally I'd opt for a value that tend towards around 5 seconds worth of
work per segment, or something like that.  Maybe using the actual
objects size could be another way.

quoted

I'd turn on default threaded support with this change. With threads=1
versus non-threaded there is no appreciable difference in the time.

Would need a way to determine pthreads availability from Makefile.

configure knows if pthreads is there.

quoted

Is there an API to ask how many CPUs are in the system? It would be
nice to default the number of threads equal to the number of CPUs and
only use pack.threads=X to override.

If there is one besides futzing with /proc/cpuinfo I'd like to know
about it.  Bonus points if it is portable.


Nicolas

-- 
Jon Smirl
jonsmirl@gmail.com