Re: Assembly failure
flat view
From: Sebastian Riemer <hidden>
Date: 2012-07-10 16:48:00
1. Are you crazy to do so? The kernel 3.2 is the stable for Ubuntu 12.04. 2. Please provide the complete Ubuntu version number of your kernel so that we can look for the commits in the Ubuntu Git. Should be git://kernel.ubuntu.com/ppisati/ubuntu-quantal.git. There were some nasty bugs in 3.4.0 mainline - I don't know if the fixes for them are in your kernel. Cheers, Sebastian On 10.07.2012 18:33, Brian Candler wrote:
An odd one here.
Ubuntu 12.04 system, updated to 3.4.0 kernel from the mainline-ppa. Machine
has a boot disk plus 12 other disks in a RAID10 far2 array.
System was working fine, but after most recent reboot mdraid failed to
assemble.
root@dev-storage1:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : inactive sdk[9](S) sdl[10](S) sdj[8](S) sdg[5](S) sdc[1](S) sdm[11](S) sdf[4](S) sde[3](S) sdi[7](S) sdd[2](S) sdb[0](S) sdh[6](S)
35163186720 blocks super 1.2
unused devices: <none>
dmesg shows periodic "export_rdev" messages:
root@dev-storage1:~# dmesg | grep md:
[ 953.986401] md: export_rdev(sdo)
[ 953.988515] md: export_rdev(sdo)
[ 960.237392] md: export_rdev(sdp)
[ 960.241928] md: export_rdev(sdp)
[ 960.965132] md: export_rdev(sdr)
[ 960.967265] md: export_rdev(sdr)
[ 1012.573415] md: export_rdev(sdo)
[ 1012.575650] md: export_rdev(sdo)
[ 1012.829690] md: export_rdev(sdp)
[ 1012.831493] md: export_rdev(sdp)
...
[19378.332473] md: export_rdev(sds)
[19378.333764] md: export_rdev(sds)
[19417.220171] md: export_rdev(sdr)
[19417.221748] md: export_rdev(sdr)
[23739.824227] md: export_rdev(sdr)
[23739.825554] md: export_rdev(sdr)
[23740.568940] md: export_rdev(sds)
[23740.570079] md: export_rdev(sds)
metadata (see below) suggests that some drives think members 1/3/4 are
missing, but those drives think the array is fine. The "Events" counts are
different on some members though.
What's the best thing to do here - attempt to force assembly? Any ideas how
it got into this state?
The machine was rebooted a couple of times but in what should have been a
clean way, i.e. sudo reboot or sudo halt -p.
Many thanks,
Brian.
root@dev-storage1:~# for i in /dev/sd{j..u}; do echo "=== $i ==="; mdadm --examine $i; done
=== /dev/sdj ===
/dev/sdj:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e96965d5:bf8986b7:fa83b813:e27aa17f
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : 5673ac95 - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 8
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdk ===
/dev/sdk:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : f15bb47f:85ca59b1:cad42dec:f8b1b63c
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : 9020674a - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 9
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdl ===
/dev/sdl:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e2d82a4c:8409d883:cf2d9b7c:83829aad
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : 30664ccb - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 10
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdm ===
/dev/sdm:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e3603570:8e767487:63f3131b:afe358ea
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : 33446897 - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 11
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdn ===
/dev/sdn:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 549a5230:005cedf8:b37a0d7e:36648ff0
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : ce0a8f46 - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 0
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdo ===
/dev/sdo:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : active
Device UUID : b09f3869:09ce8a89:31ed7097:d3621064
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Jul 7 14:02:07 2012
Checksum : 246eb119 - correct
Events : 29355
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 2
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdp ===
/dev/sdp:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : active
Device UUID : 71494ee2:1504b35a:00a1d927:543db7c6
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Jul 7 14:00:55 2012
Checksum : 61c11eeb - correct
Events : 29352
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 3
Array State : A.AA.AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdq ===
/dev/sdq:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 17fcd5b8:97fc715f:0877d022:3770d08b
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : fd1699fc - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 7
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdr ===
/dev/sdr:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e0565412:a68cf236:9da9a141:e89f935a
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 3 04:21:50 2012
Checksum : 1ba72ec1 - correct
Events : 20228
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 1
Array State : AAAAAAAAAAAA ('A' == active, '.' == missing)
=== /dev/sds ===
/dev/sds:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : active
Device UUID : c0f3bbab:70f3e69e:1314cdca:072d5ad9
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 3 06:37:33 2012
Checksum : 24f36d4c - correct
Events : 29312
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 5
Array State : A.AA.AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdt ===
/dev/sdt:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 6b4f9595:4de46aa8:fa695fe8:59b797e7
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 10 10:46:19 2012
Checksum : 44250f1b - correct
Events : 29374
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 6
Array State : A.A..AAAAAAA ('A' == active, '.' == missing)
=== /dev/sdu ===
/dev/sdu:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 16b260fd:e49bd157:da886cd0:5394e194
Name : storage1:storage1
Creation Time : Thu Jun 7 13:51:21 2012
Raid Level : raid10
Raid Devices : 12
Avail Dev Size : 5860531120 (2794.52 GiB 3000.59 GB)
Array Size : 35163168768 (16767.11 GiB 18003.54 GB)
Used Dev Size : 5860528128 (2794.52 GiB 3000.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : active
Device UUID : 46f03854:de80bec8:b44b062c:dd265ba3
Internal Bitmap : 8 sectors from superblock
Update Time : Tue Jul 3 06:36:18 2012
Checksum : 22812848 - correct
Events : 29272
Layout : far=2
Chunk Size : 1024K
Device Role : Active device 4
Array State : A.AAAAAAAAAA ('A' == active, '.' == missing)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html