Docs and operation of RAID10, size limitations on 0.90

Hello list,

I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
array.)

I approached it this way for two reasons:

1/ It feels "cleanest"; in the sense that it's not a special case, it's
just RAID1 with two devices, which happen to be RAID5.

2/ I have a particular usage in mind: I need to be able to split the
mirror and remove half; return it later and resync.  Therefore, I want
to know which disks comprise which halves.

However, I'm rethinking toward RAID10.  However, I can't find much
documentation about layout, etc.  I believe this was recently discussed
on the list.


Can someone with RAID10 experience enlighten me?  Can it be cleaned
split, half moved to another host, or rejoined with the first half, etc.
How about performance?  (Not a huge issue, but it is better than nested
arrays as I first considered?)


Aside, I'd like it to be bootable.  I was told grub only supports
booting from 0.90 superblocks, but that 0.90 has a 2TB limitation?
Meaning, I need /boot with 0.90 and another partition (LVM) with a 1.0
superblocks?



Thanks,
Richard

Richard Michael wrote:

Hello list,

I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
array.)

I approached it this way for two reasons:

1/ It feels "cleanest"; in the sense that it's not a special case, it's
just RAID1 with two devices, which happen to be RAID5.

2/ I have a particular usage in mind: I need to be able to split the
mirror and remove half; return it later and resync.  Therefore, I want
to know which disks comprise which halves.

However, I'm rethinking toward RAID10.  However, I can't find much
documentation about layout, etc.  I believe this was recently discussed
on the list.


Can someone with RAID10 experience enlighten me?  Can it be cleaned
split, half moved to another host, or rejoined with the first half, etc.
How about performance?  (Not a huge issue, but it is better than nested
arrays as I first considered?)
  

10,f2 read performance is very good, write performance is okay. AFAIK 
you can't do a split, the object is to spread head motion and improve 
performance.

Aside, I'd like it to be bootable.  I was told grub only supports
booting from 0.90 superblocks, but that 0.90 has a 2TB limitation?
Meaning, I need /boot with 0.90 and another partition (LVM) with a 1.0
superblocks?
  

I usually create a 100MB or so partition, raid1, on my drives and boot 
off that using 0.90 superblock. The rest of the storage can be anything 
you want.

-- 
Bill Davidsen [off-list ref]
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark 

10,f2 read performance is very good, write performance is okay. AFAIK you 
can't do a split, the object is to spread head motion and improve 
performance.

Thanks Bill.  "Split-a-bility" is a requirement unfortunately, so maybe
RAID10 is not viable.  

I usually create a 100MB or so partition, raid1, on my drives and boot off 
that using 0.90 superblock. The rest of the storage can be anything you 
want.

Ok; as do I.  I guess I'll do so again to make sure everything works.

Regards,
Richard

On Wed, Jul 30, 2008 at 12:18:06PM -0400, Bill Davidsen wrote:

Richard Michael wrote:

quoted

Hello list,

I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
array.)

I approached it this way for two reasons:

1/ It feels "cleanest"; in the sense that it's not a special case, it's
just RAID1 with two devices, which happen to be RAID5.

To me it seems muddy. Just having one big raid5 or raid10 would be cleaner and
give better performance.

quoted

2/ I have a particular usage in mind: I need to be able to split the
mirror and remove half; return it later and resync.  Therefore, I want
to know which disks comprise which halves.

I am not sure what you mean with this. You want to remove half of the
disks? I think you can do that with raid10, you just remove every other
disk. I have not tried that out, however. You could try it out before
actually emplouing it.

quoted

However, I'm rethinking toward RAID10.  However, I can't find much
documentation about layout, etc.  I believe this was recently discussed
on the list.

There is documentation on http://linux-raid.osdl.org/
and on http://std.dkuug.dk/keld/stripemirror.html (for raid10,f2), and 
on http://en.wikipedia.org/wiki/Non-standard_RAID_levels


Best regards
keld

On Wed, Jul 30, 2008 at 08:29:07PM +0200, Keld Jørn Simonsen wrote:

On Wed, Jul 30, 2008 at 12:18:06PM -0400, Bill Davidsen wrote:

quoted

Richard Michael wrote:

quoted

Hello list,

I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
array.)

I approached it this way for two reasons:

1/ It feels "cleanest"; in the sense that it's not a special case, it's
just RAID1 with two devices, which happen to be RAID5.

To me it seems muddy. Just having one big raid5 or raid10 would be cleaner and
give better performance.

I can't have one big RAID5 because I want to split it in half
regularly.  I refer to it as "cleaner", because it treats the devices
separately, which is what they are; one is RAID1 device, the other is a
RAID5 device.  Each is agnostic, and independent, with regard to the
other.  "Muddy" in my opinion is a mdadm hybrid RAID10, of which I need
special knowledge.  YMMV.
 

quoted

quoted

2/ I have a particular usage in mind: I need to be able to split
the mirror and remove half; return it later and resync.  Therefore,
I want to know which disks comprise which halves.

I am not sure what you mean with this. You want to remove half of the
disks? I think you can do that with raid10, you just remove every
other disk. I have not tried that out, however. You could try it out
before actually emplouing it.

Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
remove half the disks on a regular basis, as off-site storage, and return
them to the array for periodic resync.

Regards,
Richard
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
[off-list ref] wrote:

Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
remove half the disks on a regular basis, as off-site storage, and return
them to the array for periodic resync.

Basically what you want is a raid 51.
2x RAID5 (4 disks each) mirrored to form a RAID1.
Using bitmaps in the RAID1 would let you remove one of the RAID5, do
with it what you will, and add it back in again and only the changed
parts will get syncd.
At least in theory.

Look also into using drbd.

-- 
Jon

On Wed, Jul 30, 2008 at 04:56:38PM -0500, Jon Nelson wrote:

On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
[off-list ref] wrote:

quoted

Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
remove half the disks on a regular basis, as off-site storage, and return
them to the array for periodic resync.

Basically what you want is a raid 51.
2x RAID5 (4 disks each) mirrored to form a RAID1.

Exactly.  Have you built such a config and managed to get the system
booting from the nested RAID1 (perhaps even with LVM inside)?  This is
where I am having difficulty.  Else, I'll just create an 8-way RAID1 of
100MB or so for /boot (as usual, as I mentioned elsewhere).

Look also into using drbd.

Will do; thanks.

Richard

On Wed, Jul 30, 2008 at 5:47 PM, Richard Michael
[off-list ref] wrote:

On Wed, Jul 30, 2008 at 04:56:38PM -0500, Jon Nelson wrote:

quoted

On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
[off-list ref] wrote:

quoted

Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
remove half the disks on a regular basis, as off-site storage, and return
them to the array for periodic resync.

Basically what you want is a raid 51.
2x RAID5 (4 disks each) mirrored to form a RAID1.

Exactly.  Have you built such a config and managed to get the system
booting from the nested RAID1 (perhaps even with LVM inside)?  This is
where I am having difficulty.  Else, I'll just create an 8-way RAID1 of
100MB or so for /boot (as usual, as I mentioned elsewhere).

Built? Yes. Managed to get it to boot? No. Not bothered.
Just partition the drives with a bit to spare (a few gig) and use
that. No great loss and it saves a great deal of hassle.
I'd use 4 drives for storage in raid5 and the other 4 also in raid5 for swap.


-- 
Jon