RE: Removed drive from mdadm raid 5 array after reboot

From
Mike Viau <hidden>
Date
2011-02-27 03:13:42
quoted
Date: Fri, 18 Feb 2011 21:49:16 +0100
quoted
Dnia 2011-02-18, pią o godzinie 15:08 -0500, Mike Viau pisze:
I was wondering if anyone had come across an issue where after
rebooting the system, mdadm fails to reassemble an entire raid 5 array
with all the drives. I am getting the array up with just /dev/sda and
/dev/sdb, but the array is degraded as a consequence to missing
/dev/sdd (which I assume has become the parity drive).
There is no such thing as a parity drive in raid5, everything (parity
data, too) is spread out evenly over all of the disks.
Hmm, that makes more sense hence distributed parity).
quoted
Please post the output of `mdadm --examine /dev/sd{a,b,d}1` and check
your logs to see if there are any mentions of why the array isn't
assembled on boot.
mdadm --examine /dev/sd{a,b,d}1
/dev/sda1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7d8a7c68:95a230d0:0a8f6e74:4c8f81e9
Name : XEN-HOST:0 (local to host XEN-HOST)
Creation Time : Mon Dec 20 09:48:07 2010
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953517954 (931.51 GiB 1000.20 GB)
Array Size : 3907035136 (1863.02 GiB 2000.40 GB)
Used Dev Size : 1953517568 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 25f4baf0:9a378d2c:16a87f0c:ff89b2c8

Update Time : Fri Feb 18 16:32:19 2011
Checksum : 37383bee - correct
Events : 32184

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 0
Array State : AAA ('A' == active, '.' == missing)
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7d8a7c68:95a230d0:0a8f6e74:4c8f81e9
Name : XEN-HOST:0 (local to host XEN-HOST)
Creation Time : Mon Dec 20 09:48:07 2010
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953517954 (931.51 GiB 1000.20 GB)
Array Size : 3907035136 (1863.02 GiB 2000.40 GB)
Used Dev Size : 1953517568 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : f20ab5fd:1f141cae:e0547278:d6cf063e

Update Time : Fri Feb 18 16:32:19 2011
Checksum : a70821e2 - correct
Events : 32184

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 1
Array State : AAA ('A' == active, '.' == missing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x2
Array UUID : 7d8a7c68:95a230d0:0a8f6e74:4c8f81e9
Name : XEN-HOST:0 (local to host XEN-HOST)
Creation Time : Mon Dec 20 09:48:07 2010
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953521072 (931.51 GiB 1000.20 GB)
Array Size : 3907035136 (1863.02 GiB 2000.40 GB)
Used Dev Size : 1953517568 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Recovery Offset : 610474280 sectors
State : clean
Device UUID : 33d70114:ffdc4fcc:2c8d65ba:ab50bab2

Update Time : Fri Feb 18 16:32:19 2011
Checksum : b692957e - correct
Events : 32184

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 2
Array State : AAA ('A' == active, '.' == missing)



The array actually does assemble on boot, but the /dev/sdd1 partition
is not added automatically, and I am not sure why?
quoted
Have you tried booting in single user mode (I assume this drive isn't
your boot drive?) and try to assemble the array by hand? Have you seen
any errors?

What specific command should I use to assemble the array by hand? I see that there are assemble and incremental assemble commands. I'd also assume I should increase the verbosity to see what is going on.

Does anyone think that trying to re-build this raid 5 array would be a possible way to fix the removed drive problem on boot?

I read in the man page under the section for build:

"cannot differentiate between initial creation and subsequent assembly of  an  array.  
It also  cannot  perform  any  checks that appropriate components have been requested.  Because of
this, the Build mode should only be used together with a complete understanding of what you are doing."


Therefor I am cautious to proceed without some advice first...

Thanks.


The array was initially created about a month ago, below are the mdadm
events in my syslog. A couple questions arise:

1) If mdadm initially created the array as md127, and then I declared
it as md0 in mdadm.conf as:

# definitions of existing MD arrays
ARRAY /dev/md/0 metadata=1.2 UUID=7d8a7c68:95a230d0:0a8f6e74:4c8f81e9
name=XEN-HOST:0


2) If sdd was an always connected usb enclosure connected hard drive as
apposed to /dev/sda and /dev/sdb being directly connected SATA drives.


Would either of these cause the problem such as the subject of this thread?


cat /var/log/syslog | grep mdadm
Jan 30 00:57:01 XEN-HOST /USR/SBIN/CRON[6944]: (root) CMD (if [ -x
/usr/share/mdadm/checkarray ] && [ $(date +%d) -le 7 ]; then
/usr/share/mdadm/checkarray --cron --all --idle --quiet; fi)
Jan 30 22:33:48 XEN-HOST mdadm[25684]: NewArray event detected on md
device /dev/md127
Jan 30 22:33:48 XEN-HOST mdadm[25684]: DegradedArray event detected on
md device /dev/md127
Jan 30 22:45:13 XEN-HOST mdadm[1883]: NewArray event detected on md
device /dev/md127
Jan 30 22:45:15 XEN-HOST mdadm[1883]: DegradedArray event detected on
md device /dev/md127
Jan 30 23:29:31 XEN-HOST mdadm[1930]: DegradedArray event detected on
md device /dev/md/0
Jan 30 23:33:51 XEN-HOST mdadm[1889]: DegradedArray event detected on
md device /dev/md/0
Jan 30 23:38:32 XEN-HOST mdadm[1889]: RebuildStarted event detected on
md device /dev/md/0
Jan 31 01:51:53 XEN-HOST mdadm[1889]: Rebuild20 event detected on md
device /dev/md/0
Jan 31 04:05:13 XEN-HOST mdadm[1889]: Rebuild41 event detected on md
device /dev/md/0
Jan 31 06:18:34 XEN-HOST mdadm[1889]: Rebuild62 event detected on md
device /dev/md/0
Jan 31 08:15:15 XEN-HOST mdadm[1889]: Rebuild80 event detected on md
device /dev/md/0
Jan 31 10:15:50 XEN-HOST mdadm[1889]: RebuildFinished event detected on
md device /dev/md/0
Jan 31 10:15:50 XEN-HOST mdadm[1889]: SpareActive event detected on md
device /dev/md/0
Feb 6 00:57:01 XEN-HOST /USR/SBIN/CRON[13578]: (root) CMD (if [ -x
/usr/share/mdadm/checkarray ] && [ $(date +%d) -le 7 ]; then
/usr/share/mdadm/checkarray --cron --all --idle --quiet; fi)
Feb 6 00:57:01 XEN-HOST mdadm[1889]: RebuildStarted event detected on
md device /dev/md/0
Feb 6 02:37:02 XEN-HOST mdadm[1889]: Rebuild21 event detected on md
device /dev/md/0
Feb 6 04:17:02 XEN-HOST mdadm[1889]: Rebuild42 event detected on md
device /dev/md/0
Feb 6 05:40:23 XEN-HOST mdadm[1889]: Rebuild60 event detected on md
device /dev/md/0
Feb 6 07:20:23 XEN-HOST mdadm[1889]: Rebuild81 event detected on md
device /dev/md/0
Feb 6 08:44:40 XEN-HOST mdadm[1889]: RebuildFinished event detected on
md device /dev/md/0
Feb 6 23:11:07 XEN-HOST mdadm[1887]: DegradedArray event detected on
md device /dev/md/0
Feb 8 08:51:53 XEN-HOST mdadm[1887]: RebuildStarted event detected on
md device /dev/md/0
Feb 8 11:05:13 XEN-HOST mdadm[1887]: Rebuild20 event detected on md
device /dev/md/0
Feb 8 13:18:34 XEN-HOST mdadm[1887]: Rebuild41 event detected on md
device /dev/md/0
Feb 8 15:15:14 XEN-HOST mdadm[1887]: Rebuild60 event detected on md
device /dev/md/0
Feb 8 17:28:35 XEN-HOST mdadm[1887]: Rebuild81 event detected on md
device /dev/md/0
Feb 8 19:26:32 XEN-HOST mdadm[1887]: RebuildFinished event detected on
md device /dev/md/0
Feb 8 19:26:32 XEN-HOST mdadm[1887]: SpareActive event detected on md
device /dev/md/0
Feb 13 00:57:01 XEN-HOST /USR/SBIN/CRON[12933]: (root) CMD (if [ -x
/usr/share/mdadm/checkarray ] && [ $(date +%d) -le 7 ]; then
/usr/share/mdadm/checkarray --cron --all --idle --quiet; fi)
Feb 18 12:19:31 XEN-HOST mdadm[1864]: DegradedArray event detected on
md device /dev/md/0
Feb 18 12:23:07 XEN-HOST mdadm[1613]: DegradedArray event detected on
md device /dev/md/0
Feb 18 12:26:24 XEN-HOST mdadm[1767]: DegradedArray event detected on
md device /dev/md/0
Feb 18 12:50:53 XEN-HOST mdadm[1767]: RebuildStarted event detected on
md device /dev/md/0
Feb 18 15:04:14 XEN-HOST mdadm[1767]: Rebuild21 event detected on md
device /dev/md/0
 		 	   		  --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Thread (1 message) 1 message, 1 author, last reply 2011-02-27
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help