On 12/30/2013 2:32 AM, Pieter De Wit wrote:

Hi Stan,

Thanks for the long email (I didn't know about advance formatting for
one) - please see my answers inline.

On 30/12/2013 19:56, Stan Hoeppner wrote:

quoted

On 12/29/2013 3:04 PM, Pieter De Wit wrote:

quoted

<snip>
So my question is, do I need to align the partitions for the raid
devices ?

<snip>
Are these 2TB Advanced Format drives?  If so your partitions need to
align to 4KiB boundaries, otherwise you'll have RMW within each drive
which can cut your write throughput by 30-50%.

Yes - these drives are, parted printed:

Model: ATA WDC WD20EARX-008 (scsi)
Disk /dev/sdb: 3907029168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start       End          Size         File system  Name Flags
 1      2048s       500000767s   499998720s raid
 2      500000768s  3907028991s  3407028224s raid

quoted

<snip>

So given your comments then, the start of partition 1 is correct. The
start of partition 2 is also correct (not sure if this is needed), but
the size of partition 2 is incorrect, it should be 3406823424s ?

Size is incorrect in what way?  If your RAID0 chunk is 512KiB, then
3407028224 sectors is 3327176 chunks, evenly divisible, so this
partition is fully aligned.  Whether the capacity is correct is
something only you can determine.  Partition 2 is 1.587 TiB.

quoted

You're comparing apples to oranges to grapes below, and your description
lacks any level of technical detail.  How are we supposed to analyze
this?

quoted

These are desktop grade drives, but for the RAID0 device I saw quite low
throughput (15meg/sec moving data to the NAS via gig connection). I just

"15meg/sec moving data" means what, a bulk file transfer from a local
filesystem to a remote filesystem?  What types of files?  Lots of small
ones?  Of course throughput will be low.  Is the local filesystem
fragmented?  Even slower.

It's all done with pvmove, which moves 4meg chunks.

I'm not intending to be jerk, but this is a technical mailing list.  You
need to be precise so others understand EXACTLY what you're stating.
Your choice of words above suggests you -first- used NFS or CIFS and it
was slow at 15 'meg'/sec (please use MB or MiB appropriately).  "NAS" is
Network Attached Storage.  The two protocols nearly exclusively used to
communicate with a NAS device are NFS and CIFS.

What you typed may make perfect sense to YOU, but to your audience it is
thoroughly misleading.

quoted

quoted

created a RAID1 device between /dev/sda and an iSCSI target on the NAS,
and it synced at 48meg/sec, moving data at 30meg/sec - double that of
the RAID0 device.

This is block device data movement.  There is no filesystem overhead, no
fragmentation causing excess seeks, and no NFS/CIFS overhead on either
end.  Of course it will be faster.

It was all done with pvmove :)

-Second- you explicitly state here that you then created a RAID1 between
sda and an iSCSI target and achieved 3x the throughput, suggesting that
this is different than the case above.

Again, what you typed may make perfect sense to YOU, but to your
audience it is misleading, because you didn't clearly state the
configuration the first statement describes.

So all of this was done over iSCSI, correct?

Without further data I can only make a wild ass guess as to why the
RAID0 device was slower than the single disk during this -single
operation- you described that involves a network.  You didn't post
throughput numbers for the RAID0 doing a local operation so there's
nothing to compare to.  It could be due to a dozen different things.  A few?

1.  Concurrent disk access at the host
2.  Concurrent disk access/load at the NAS box
3.  One/both of the host EARX drives is flaky causing high latency
4.  Flaky GbE HBA or switch port
etc

Show your partition table for sdc.  Even if the partitions on it are not
aligned, reads shouldn't be adversely affected by it.  Show

$ mdadm --detail

for the RAID0 array.  md itself, especially in RAID0 personality, is
simply not going to be the -cause- of low performance.  The problem lay
somewhere else.  Given the track record of Western Digital's Green
series of drives I'm leaning toward that cause.  Post output from

$ smartctl -A /dev/sdb
$ smartctl -A /dev/sdc

quoted

quoted

I would have expected the RAID0 device to easily get
up to the 60meg/sec mark ?

As the source disk of a bulk file copy over NFS/CIFS?  As a point of
reference, I have a workstation that maxes 50MB/s FTP and only 24MB/s
CIFS to/from a server.  Both hosts have far in excess of 100MB/s disk
throughput.  The 50MB/s limitation is due to the cheap Realtek mobo NIC,
and the 24MB/s is a Samba limit.  I've spent dozens of hours attempting
to tweak Samba to greater throughput but it simply isn't capable on that
machine.

Your throughput issues are with your network, not your RAID.  Learn and
use FIO to see what your RAID/disks can do.  For now a really simple
test is to time cat of a large file and pipe to /dev/null.  Divide the
file size by the elapsed time.  Or simply do a large read with dd.  This
will be much more informative than "moving data to a NAS", where your
throughput is network limited, not disk.

The system is using a server grade NIC, I will run a dd/network test
shortly after the copy is done. (I am shifting all the data back to the
NAS, incase I mucked up the partitions :) ), I do recall that this
system was able to fill a gig pipe...

Now that you've made it clear the first scenario was over iSCSI same as
the 2nd scenario, and not NFS/CIFS, I doubt the TCP stack is the
problem.  Assume the network is fine for now and concentrate on the disk
drives in the host.  That's seems the most likely cause of the problem
at this point.

BTW, you didn't state the throughput of the RAID1 device on sdb/sdc.
The RAID0 device is on the same disks, yes?  RAID0 was 15 MB/s.  What
was the RAID1?

-- 
Stan