Is a raid0 512 byte chunk size possible? Or is it just too small?

Hello,
I would like to use mdadm to set up a raid0 with a 512B chunk size.

I ask as my purpose is to mimic a raid0 config from a Lacie NAS box
that uses a 512B chunk size.

The lowest chunk value mdadm will accpet is 4. Anything less and mdadm
gives an error "invalid chunk/rounding value"

Is there any way to create a raid0 with a 512B chunk?

Thanks,
Veedar

On 8/30/2013 2:32 PM, Veedar Hokstadt wrote:

Hello,
I would like to use mdadm to set up a raid0 with a 512B chunk size.

I ask as my purpose is to mimic a raid0 config from a Lacie NAS box
that uses a 512B chunk size.

Your reasoning is flawed.  Why would you want to imitate a configuration
that is inherently flawed?

The lowest chunk value mdadm will accpet is 4. Anything less and mdadm
gives an error "invalid chunk/rounding value"

For good reason.

Is there any way to create a raid0 with a 512B chunk?

First, if you're using RAID0 it absolutely must be assumed that you
desire maximum speed, care nothing for redundancy, and you don't care if
you lose your data when a disk fails because you have a full backup of
the RAID0 filesystem.

If you want speed, using RAID0 with a 512 byte chunk isn't going to
achieve it.  On the contrary, using such a small chunk will drop a
hammer on your throughput because you're processing a much larger number
of IOs per quantity of data transferred.  This is extremely inefficient,
and throughput drops.  With RAID0 you typically want a very large chunk,
the largest your drives can ingest efficiently in a single IO.  I'll
make an educated assumption that you plan to store media files on this
array, probably DVDs/CDs, and/or use it as a DVR.  In this case you want
a large chunk, 512KB-1MB.

However, you've stated you want to duplicate a NAS device.  Consider
that GbE using Rtl 81xx devices tops out at ~70-90 MBs application level
throughput.  Two modern drives in RAID0 with a proper chunk size can
read/write at double that rate.  Given this fact, why are you bothering
with RAID0?  You won't see any of the increased performance RAID0 can
give you.  In fact a single modern drive can saturate GbE.

I assume you are using RAID0 simply as an inexpensive way to maximize
your storage capacity.  That's fine with backups or if you have the
original media.  If you don't, or don't want to go through the hassle of
recreating your RAID0 after a disk failure and replacement, and copying
all your files back to it, I suggest you use RAID1/5/6/10 instead.

-- 
Stan

Maybe he has raw drives that were in this NAS and is trying to match
the layout to recover something? I know that's probably not going to
work, certainly without a lot of other things going right, but its the
first thing that came to mind, given his reasoning and how he stated
it.

On Aug 30, 2013 3:36 PM, "Stan Hoeppner" [off-list ref] wrote:

On 8/30/2013 2:32 PM, Veedar Hokstadt wrote:

quoted

Hello,
I would like to use mdadm to set up a raid0 with a 512B chunk size.

I ask as my purpose is to mimic a raid0 config from a Lacie NAS box
that uses a 512B chunk size.

Your reasoning is flawed.  Why would you want to imitate a configuration
that is inherently flawed?

quoted

The lowest chunk value mdadm will accpet is 4. Anything less and mdadm
gives an error "invalid chunk/rounding value"

For good reason.

quoted

Is there any way to create a raid0 with a 512B chunk?

First, if you're using RAID0 it absolutely must be assumed that you
desire maximum speed, care nothing for redundancy, and you don't care if
you lose your data when a disk fails because you have a full backup of
the RAID0 filesystem.

If you want speed, using RAID0 with a 512 byte chunk isn't going to
achieve it.  On the contrary, using such a small chunk will drop a
hammer on your throughput because you're processing a much larger number
of IOs per quantity of data transferred.  This is extremely inefficient,
and throughput drops.  With RAID0 you typically want a very large chunk,
the largest your drives can ingest efficiently in a single IO.  I'll
make an educated assumption that you plan to store media files on this
array, probably DVDs/CDs, and/or use it as a DVR.  In this case you want
a large chunk, 512KB-1MB.

However, you've stated you want to duplicate a NAS device.  Consider
that GbE using Rtl 81xx devices tops out at ~70-90 MBs application level
throughput.  Two modern drives in RAID0 with a proper chunk size can
read/write at double that rate.  Given this fact, why are you bothering
with RAID0?  You won't see any of the increased performance RAID0 can
give you.  In fact a single modern drive can saturate GbE.

I assume you are using RAID0 simply as an inexpensive way to maximize
your storage capacity.  That's fine with backups or if you have the
original media.  If you don't, or don't want to go through the hassle of
recreating your RAID0 after a disk failure and replacement, and copying
all your files back to it, I suggest you use RAID1/5/6/10 instead.

--
Stan



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thank you Stan. I agree with what you say. Sorry I was not more clear
about my situation.

Marcus is correct I'm trying to perform a data recovery operation from
a failed Lacie box with two drives in a raid0 / chunk 512B
configuration.

I am totally aware of the complications involved but if I can somehow
build a raid0 / chunk 512B array with mdadm using the raw Lacie dives
then I can take it from there to recover the data.

I have used another proprietary app and I am able to see the data
using a custom chunk size of 1. So it should be doable I think.

In looking at this more I tried to make a 512B stripe with dmsetup but
it fails. It seems that "getconf PAGE_SIZE"  which is set to 4096 is
the lowest allowed  stripe/chunk value. Maybe mdadm has the same size
limitation.

Thank you for your help.  -V

On 8/30/13, Marcus Sorensen [off-list ref] wrote:

Maybe he has raw drives that were in this NAS and is trying to match
the layout to recover something? I know that's probably not going to
work, certainly without a lot of other things going right, but its the
first thing that came to mind, given his reasoning and how he stated
it.

On Aug 30, 2013 3:36 PM, "Stan Hoeppner" [off-list ref] wrote:

quoted

On 8/30/2013 2:32 PM, Veedar Hokstadt wrote:

quoted

Hello,
I would like to use mdadm to set up a raid0 with a 512B chunk size.

I ask as my purpose is to mimic a raid0 config from a Lacie NAS box
that uses a 512B chunk size.

Your reasoning is flawed.  Why would you want to imitate a configuration
that is inherently flawed?

quoted

The lowest chunk value mdadm will accpet is 4. Anything less and mdadm
gives an error "invalid chunk/rounding value"

For good reason.

quoted

Is there any way to create a raid0 with a 512B chunk?

First, if you're using RAID0 it absolutely must be assumed that you
desire maximum speed, care nothing for redundancy, and you don't care if
you lose your data when a disk fails because you have a full backup of
the RAID0 filesystem.

If you want speed, using RAID0 with a 512 byte chunk isn't going to
achieve it.  On the contrary, using such a small chunk will drop a
hammer on your throughput because you're processing a much larger number
of IOs per quantity of data transferred.  This is extremely inefficient,
and throughput drops.  With RAID0 you typically want a very large chunk,
the largest your drives can ingest efficiently in a single IO.  I'll
make an educated assumption that you plan to store media files on this
array, probably DVDs/CDs, and/or use it as a DVR.  In this case you want
a large chunk, 512KB-1MB.

However, you've stated you want to duplicate a NAS device.  Consider
that GbE using Rtl 81xx devices tops out at ~70-90 MBs application level
throughput.  Two modern drives in RAID0 with a proper chunk size can
read/write at double that rate.  Given this fact, why are you bothering
with RAID0?  You won't see any of the increased performance RAID0 can
give you.  In fact a single modern drive can saturate GbE.

I assume you are using RAID0 simply as an inexpensive way to maximize
your storage capacity.  That's fine with backups or if you have the
original media.  If you don't, or don't want to go through the hassle of
recreating your RAID0 after a disk failure and replacement, and copying
all your files back to it, I suggest you use RAID1/5/6/10 instead.

--
Stan



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 8/31/2013 12:05 AM, Veedar Hokstadt wrote:

Thank you Stan. I agree with what you say. Sorry I was not more clear
about my situation.

No apology necessary.  Upon reflection I should have realized what you
were attempting, given the oddball chunk size, and the fact there are
only 3 types of email on this list--PATCH, help with creation, help with
recovery.  As we discussed, nobody in his/her right mind would want to
create a new array with a 512B chunk, which leaves recovery.  I should
have caught that...

Marcus is correct I'm trying to perform a data recovery operation from
a failed Lacie box with two drives in a raid0 / chunk 512B
configuration.

I am totally aware of the complications involved but if I can somehow
build a raid0 / chunk 512B array with mdadm using the raw Lacie dives
then I can take it from there to recover the data.

I have used another proprietary app and I am able to see the data
using a custom chunk size of 1. So it should be doable I think.

In looking at this more I tried to make a 512B stripe with dmsetup but
it fails. It seems that "getconf PAGE_SIZE"  which is set to 4096 is
the lowest allowed  stripe/chunk value. Maybe mdadm has the same size
limitation.

The Linux kernel is designed around page size accesses.  This includes
IO.  The Linux page size is 4KB.  This is why all the Linux filesystems
have a 4KB block size, and why md has a minimum 4KB chunk size.  Most
processor ISAs support other page sizes, but these are all greater than
4KB.  To do what you desire with Linux md you'd have to find a Linux
supported processor architecture with a 512B page size and hack a large
amount of the kernel code.  I don't believe anyone makes such a CPU and
I doubt you want to tackle such a kernel programming job.

My advice to you is to call Lacie and get a replacement board, or
whatever part it is that failed, or buy a used unit identical to yours
on Ebay.  Or maybe they have some custom MS Windows software they'll
sell you that will get the data off the drives.  Given their customer
base I doubt they'd have Linux software, only Windows.  I think the Ebay
route is the most appealing and has the highest probability of success.
 When you're done with it, throw it back on Ebay and get most of your
money back.

Good luck with this.  Let us know if/when you succeed, and how you did it.

P.S.  It's surprising that the electronics died but the disks are fine.
 The vast majority of the time it's the other way round.

-- 
Stan

Thank you for your help.  -V

On 8/30/13, Marcus Sorensen [off-list ref] wrote:

quoted

Maybe he has raw drives that were in this NAS and is trying to match
the layout to recover something? I know that's probably not going to
work, certainly without a lot of other things going right, but its the
first thing that came to mind, given his reasoning and how he stated
it.

On Aug 30, 2013 3:36 PM, "Stan Hoeppner" [off-list ref] wrote:

quoted

On 8/30/2013 2:32 PM, Veedar Hokstadt wrote:

quoted

Hello,
I would like to use mdadm to set up a raid0 with a 512B chunk size.

I ask as my purpose is to mimic a raid0 config from a Lacie NAS box
that uses a 512B chunk size.

Your reasoning is flawed.  Why would you want to imitate a configuration
that is inherently flawed?

quoted

The lowest chunk value mdadm will accpet is 4. Anything less and mdadm
gives an error "invalid chunk/rounding value"

For good reason.

quoted

Is there any way to create a raid0 with a 512B chunk?

First, if you're using RAID0 it absolutely must be assumed that you
desire maximum speed, care nothing for redundancy, and you don't care if
you lose your data when a disk fails because you have a full backup of
the RAID0 filesystem.

If you want speed, using RAID0 with a 512 byte chunk isn't going to
achieve it.  On the contrary, using such a small chunk will drop a
hammer on your throughput because you're processing a much larger number
of IOs per quantity of data transferred.  This is extremely inefficient,
and throughput drops.  With RAID0 you typically want a very large chunk,
the largest your drives can ingest efficiently in a single IO.  I'll
make an educated assumption that you plan to store media files on this
array, probably DVDs/CDs, and/or use it as a DVR.  In this case you want
a large chunk, 512KB-1MB.

However, you've stated you want to duplicate a NAS device.  Consider
that GbE using Rtl 81xx devices tops out at ~70-90 MBs application level
throughput.  Two modern drives in RAID0 with a proper chunk size can
read/write at double that rate.  Given this fact, why are you bothering
with RAID0?  You won't see any of the increased performance RAID0 can
give you.  In fact a single modern drive can saturate GbE.

I assume you are using RAID0 simply as an inexpensive way to maximize
your storage capacity.  That's fine with backups or if you have the
original media.  If you don't, or don't want to go through the hassle of
recreating your RAID0 after a disk failure and replacement, and copying
all your files back to it, I suggest you use RAID1/5/6/10 instead.

--
Stan



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Sat, 31 Aug 2013 01:05:44 -0400
Veedar Hokstadt [off-list ref] wrote:

I am totally aware of the complications involved but if I can somehow
build a raid0 / chunk 512B array with mdadm using the raw Lacie dives
then I can take it from there to recover the data.

You can write a simple program to read in chunks of 512 bytes from one drive,
then from the other, piece this together and write to some third file. Here is
something quick that I modified from a tool I already had. Even though it's
doing the tight loop in PHP, it seems to do about 100 MB/sec reading from two
SATA drives and writing to a file on a RAID6 in my system.

--- merge.php ---

#!/usr/bin/php
<?php
  $f = array();
  array_shift($argv);
  foreach($argv as $arg) {
    if(($f[] = @fopen($arg, "rb"))===false) die("Unable to open file $arg.\n");
  }
  while(@$feof_cnt<2) {
    $feof_cnt=0;
    foreach($f as $file) {
      print(fread($file, 512));
      if(feof($file))$feof_cnt++;
    }
  }  
?>

---

Usage: ./merge.php /dev/sda /dev/sdb > image.img

And you will need somewhere to store this image.img that's has the free space
to fit the size of sda+sdb. Also check that you specify the drives in the
proper order (only 2 tries, really).

-- 
With respect,
Roman

Just to close this out. The making a 4TB image file of the raid0 was
not feasible for me so I ended up using a commercial raid recovery app
that could handle the 512 byte chunk.

Thanks for your help.  -V

On 8/31/13, Roman Mamedov [off-list ref] wrote: