Re: Better value for chunk_size when threaded

On Thu, 6 Dec 2007, Jon Smirl wrote:

I tried some various ideas out for chunk_size and the best strategy I
found was to simply set it to a constant. How does 20,000 work on
other CPUs?

That depends on the object size.  If you have a repo with big objects 
but only 1000 of them for example, then the constant doesn't work.

Ideally I'd opt for a value that tend towards around 5 seconds worth of 
work per segment, or something like that.  Maybe using the actual 
objects size could be another way.

I'd turn on default threaded support with this change. With threads=1
versus non-threaded there is no appreciable difference in the time.

Would need a way to determine pthreads availability from Makefile.

Is there an API to ask how many CPUs are in the system? It would be
nice to default the number of threads equal to the number of CPUs and
only use pack.threads=X to override.

If there is one besides futzing with /proc/cpuinfo I'd like to know 
about it.  Bonus points if it is portable.


Nicolas