On 12/6/2013 12:13 PM, Piergiorgio Sartor wrote:

On Fri, Dec 06, 2013 at 03:24:18AM -0600, Stan Hoeppner wrote:

quoted

On 12/5/2013 1:24 PM, Piergiorgio Sartor wrote:

quoted

The "stripe_cache_size" was set to the max 32768.

You don't want to set this so high.  Doing this will:

1.  Usually decrease throughput
2.  Eat a huge amount of memory.  With 5 drives:

    ((32768*4096)*5)/1048576 = 640 MB RAM consumed for the stripe buffer

For 5 or fewer pieces of spinning rust a value of 2048 or less should be
sufficient.  Test 512, 1024, 2048, 4096, and 8192.  You should see your
throughput go up and then back down.  Find the sweet spot and use that
value.  If two of these yield throughput within 5% of one another, use
the lower value as it eats less RAM.

Hi Stan,

thanks for the reply, I was looking forward to it,
since you always provide useful information.

I checked two systems, one, different, with RAID-5,
the other the actual RAID-6.

In the first one, 2048 seems to be the best stripe
cache size, while more results in slower writing
speed, albeit not too much.

For the RAID-6, it seems 32768 is the best value.

There is one difference, the RAID-5 has chunk size
of 512k (default), while the RAID-6 has still the 64k.

BTW, why is that? I mean why large stripe cache
results in lower writing speed?

I don't have the answer to this question.  It has been asked before.  I
can only speculate that the larger cache table introduces overhead of
some kind.  You may want to ask Neil directly.

Note that you're using dd for testing this.  dd produces single stream
serial IO.  If you test other IO patterns, such as parallel or
asynchronous, with software such as FIO, the results may be a bit different.

-- 
Stan