Problems with software raid - all mine I am sure

Hello,

I have been trying to get a software raid configuration working for a 
few weeks with little success.

I currently have a 3ware 7506-4 card with 4 drives.  The raid 5 
performance of this card is poor and it was recommended to me to try 
using the card as a JBOD controller and running software raid on top of it.

I am currently running FC6 on this machine.

Here is what I have tried:

1) Backup the machine
2) Boot from knoppix DVD
3) Partition the drives using fdisk into 4 partitions.  100M for /boot ( 
bootable ), 2048M for swap, 16384M for /, and the rest for /export.  All 
partitions were tagged as 'fd'
4) Use sfdisk -d to copy the exact partitions to all 4 drives
5) Use mdadm to create md0 as raid 1 for /boot ( first two disks ), 
md1-3 as raid5 with no spares ( all four disks )
6) Restore /boot, /, export
7) mkswap /dev/md1
8) update fstab and grub.conf to reflect new partitions ( chrooted )

Here is where I get lost and confused

I tried to chroot into the old image mounted completely under 
/mnt/sysimage.  I ran grub-install on /dev/sda and /dev/sdb.  It 
complained about /dev/sdb not being a bios drive ( or similar ).  I 
figured that was ok and I could fix it later.

Upon reboot, grub was definitely in sda, but it failed to boot 
complaining it couldn't find stage1 files.

I tried to boot from Redhat rescue DVD.  It sees a 4 disks, but the raid 
comes up without sda.  I can do a dmesg and see sda, but the raid says 
it is missing.  It looks like Redhat thinks there is a /dev/dm_* devices 
as well.  However, these disks have no lvm on them.

Rebooting into Knoppix now shows the array is in degradated mode.  
/dev/sda is being thrown out..but I can't remember the exact error.  
Googling showed that I needed use mdadm to fail sda, remove it, and 
re-add it.  However, mdadm wouldn't remove it because it said it was 
busy.  I am assuming this is related to same issue as the Redhat rescue 
issue.

The software raid seems a little fragile as well.  If I have my external 
drive plugged in and all devices shift ( sda-d become sdb-e) when 
booting from Knoppix which seems to cause problems as well.  I thought 
with all the data being written to the superblocks it would handle this 
problem.

So, I am looking for a little guidance in where I am going wrong.  It 
takes about 24 hours for me to make this work and restore if it goes 
badly so I am a little hesitant to play to much without figuring out a 
correct path.

In the end, I went back to hardware raid on the 3ware card which works 
perfectly, just abysmal performance.

Any thoughts or suggestions?

Michael

On Mon, Feb 26, 2007 at 10:48:26AM -0700, Michael wrote:

Hello,

... 
<mix knoppix and fc6 >

Why don't you do everything from the text/gui installation of fc6
instead of swapping distributions?

You can do everything you have said within the fc6 installer:
http://fedora.redhat.com/docs/install-guide/fc6/en/sn-disk-druid.html
boot in raid1 on sda1/sdb1
swap,/ and /export as raid5 on sd{abcd}{234}
...

you don't have to use LVM.

my .2 cents

Tru

Tru Huynh wrote:

... 
<mix knoppix and fc6 >

Why don't you do everything from the text/gui installation of fc6
instead of swapping distributions?

You can do everything you have said within the fc6 installer:
http://fedora.redhat.com/docs/install-guide/fc6/en/sn-disk-druid.html
boot in raid1 on sda1/sdb1
swap,/ and /export as raid5 on sd{abcd}{234}
...

  

I don't want to do an install - I already have a working machine and 
don't want to loose all of my changes and configuration.  I did try this 
as well but was unable to figure out how to break out of the installer 
after the disks were partitioned/formatted so I could just do a restore.

Also, I need to do a restore once the disks are assembled into the 
raid.  If I plug my external disk in and boot it becomes sda which 
screws up the raid.  If I plug in later in rescue mode, it isn't 
autodetected and I was unable to get the right modules loaded.  That is 
why I tried knoppix.

you don't have to use LVM.

  

I am not using LVM.  However, LVM seems to latch onto sda at boot time 
and keeps it busy which causes problems.

I am just surprised this is so difficult. 

Michael