Re: 3ware + RAID5 + xfs performance

I went thru the same hardware config gymnastics (confounded by bad disks, a 
bad controller, bad hotswap cages, and some SW config issues - see:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001067.html).


The 3ware info is partially reported here:
http://maillists.uci.edu/mailman/public/uci-linux/2005-June/001134.html

and I believe I also sent a similar report to this list, but google doesn't 
show it.


Here's the results of my bonnie tests on a similar system:
8x250GB WD SD series disks on a 3ware 9500S-8, on an IWILL 2x Opteron mobo, 
4GB RAM, with XFS.

The XFS parameters are with a 64K stripe to match the RAID card; other params 
more or less vanilla.

Below are some bonnie timing results with differing filesystems (1 run with 
ext3, 3 with XFS with differnt file sizes - unwrap in an editor to compare in 
columns:

 XFS sand, 
7000M,52315,99,104402,22,32242,8,32018,60,127998,20,435.5,0,80,1724,15,+++++,
+++, 3248, 16,1718,15,+++++,+++,750, 4
ext3 sand, 7000M,40682,91, 47732,23,25432,9,38027,72,179352,27,311.1,0,80, 
416,99,+++++,+++,53250,100, 423,99,+++++,+++,560,58
 XFS sand, 
7000M,50040,98,106324,24,33046,8,31269,59,112240,17,416.0,0,80,1930,17,+++++,
+++, 3753, 17,1913,17,+++++,+++,449, 2
 XFS sand, 
15000M,51065,99,101659,24,26884,7,35344,69,141223,23,263.0,0,80,1666,14,
+++++,+++, 4565, 22,1700,16,+++++,+++,793, 4

From my reading (see URL above for resourcelist), XFS is quite bad for tiny 
files - we use it for very large files (>GB size); using XFS for this would 
generally be a bad thing.  We do NOT get tremendous performance out of it; 
but the performance is much better than with ext3 and the CPU usage is lower, 
sometimes dramtically so.  Real life experience with some benchmarks confirms 
this - we get approximately ~ the same real life thruput as we do on a IBM 
SP2 8way module with a direct attach disk.

We are now considering adding a local PVFS2 system to a small cluster for very 
fast IO under MPI




On Monday 25 July 2005 18:11, Gaspar Bakos wrote:

Dear all,

The purpose of this email is twofold:
- to share the results of the many tests I performed with
  a 3ware RAID card + RAID-5 + XFS, pushing for better file I/O,
- and to initiate some brainstorming on what parameters can be tuned for
  getting a good performance out of this hardware under 2.6.* kernels.

I started all these tests because the performance was quite poor, meaning
that the write speed was slow, the read speed was barely acceptable, and
the system load went very high (10.0) during bonnie++ tests.
My questions are marked below with "Q".

<clipped>
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html