Re: Very long raid5 init/rebuild times

On 2/4/2014 7:19 PM, Phillip Susi wrote:

On 02/04/2014 04:58 PM, Stan Hoeppner wrote:

quoted

Yes, I stated that in this post

http://www.spinics.net/lists/raid/msg45726.html

in the context of achieving greater throughput with an FIO job
file configured to use O_DIRECT, a job file I created, that the OP
was using for testing.  That job file is quoted further down in
this same post, and is included in my posts prior this one in the
thread.  Apparently you ignored them.  The context of my comment
above is clearly established multiple times earlier in the thread.

In my paragraph directly preceding the statement you quote above,
I stated this:

"Serial submission typically doesn't reach peak throughput...  You 
usually must submit asynchronously or in parallel to reach maximum 
throughput."

And again this is in the context of the FIO job file using
O_DIRECT, and this statement is factual.  As I repeated earlier
today, O_DIRECT is used because measuring actual throughput at the
disks is straightforward.  To increase O_DIRECT write throughput in
FIO, you typically need parallel submission or AIO.  This is well
known.

Ahh, I did not gather that O_DIRECT was already assumed.  In that
case, then I was simply restating the same thing: that you want aio
with O_DIRECT, but otherwise, buffered IO works fine too ( which is
what the OP was using with dd, which is why it sounded like you were
saying not to do that, that you must use O_DIRECT + aio because
buffered IO won't get you the performance you're looking for ).

I guess maybe I wasn't clear with my wording at that time.  Yes, IIRC he
was doing dd through buffer cache.  My point to him was that O_DIRECT dd
gives more accurate throughput numbers, but a single stream may not be
sufficient to peak the disks.  Which is why I recommended FIO and
provided a job file, as it can do multiple O_DIRECT streams.  AIO can
reduce dispatch latency thus increasing throughput, but not to the
extent that multiple streams will, as the latter can fully overlap the
single stream dispatch latency, keeping the pipeline full.

-- 
Stan