Re: Recovering an Array with inconsistent Superblocks

Hi Fabian,

I see good news to greet me this morning!

On 01/05/2014 05:45 AM, Fabian Knorr wrote:

$ cat /proc/mdstat 
Personalities : [raid6] [raid5] [raid4] 
md127 : active (auto-read-only) raid5 sde1[6] sdh1[2] sdi1[3] sdb1[4] sdg1[1] sdc1[5] sdf1[0]
      11721074688 blocks level 5, 1024k chunk, algorithm 2 [7/7] [UUUUUUU]
        resync=PENDING

Auto-read-only will switch to read-write as soon as you actually write
anything to the array.  Which will also kick off the resync.

unused devices: <none>

$ mdadm --stop /dev/md127 
mdadm: Cannot get exclusive access to /dev/md127:Perhaps a running process, mounted filesystem or active volume group?

LVM has the array open--but you don't need to stop it.  Just mount your
storage filesystem.

The damaged root filesystem undoubtedly had much info in RAM cache that
couldn't be written after the controller hiccup.

I'd like to keep the state of lvm-storage as it is and re-install my
system. Still, a few things I'm not sure about:

	- How can I get mdadm to initiate resync and get write access 
	  to my data?

See above.

	- As lvm-root seems to be severly damaged, is there any chance
	  of  errors in the lvm-storage FS that fsck -fn does not
	  detect? Could re-syncing therefore destroy data I have access
	  to now?

Possible, but not any help for it.  The file allocation structures are
Ok (that's what fsck can see), but if any contents had been just written
at the time of the crash, parts of those writes might not have reached
the dropped array members.

	- Can I make sure that the working setup is written to all 
	  superblocks, and zero the spare's superblock so that there is
	  no such confusion when trying to assemble the array in the
	  future?

Yes, you can use --zero-superblock on the spare before you --add it back
the array.  It might save you some warnings.

Long term, consider creating a new array with metadata type 1.x so you
can use a bitmap.  If you ever have an event like this, where one or
more devices are disconnected then reconnected, it'll greatly shorten
the recovery time.

You should also learn to use --re-add instead of --add for devices that
are supposed to already be part of an array.  Newer versions of mdadm
try to help you with this.

HTH,

Phil

(And thank you too, Neil, for a timely assist!)