kernel refuses devices mdadm -E accepts

I have a raid5 array for which every disk, when examined with "mdadm
-E /dev/sd[abcd]4" shows a Version: 1.2 superblock with a valid
checksum.

However, when I try:
mdadm -A /dev/md/backing /dev/sd[abcd]4
mdadm says:
mdadm: failed to add /dev/sdc4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sdb4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sda4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sdd4 to /dev/md/backing: Invalid argument
mdadm: /dev/md/backing assembled from 0 drives - need all 4 to start
it (use --run to insist)
The kernel says:
md: sdc4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sdb4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sda4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sdd4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22

I was in the middle of a reshape of this 4-disk raid5 when something
rebooted the computer. The system seems otherwise fine, and I suspect
someone in the house.

What is the correct next step? Should I try --run ? I would obviously
prefer not to lose the data on this array. I expect that the reshape
was NOT complete, so just recreating the array will probably corrupt
its contents.

Kernel version 3.17.8 and mdadm version 3.3.2.

Thanks for any help. If I lose this array, I am going to face a lot of grief...

cat /proc/mdstat

if the device is in use mdadm -E will work but you cannot create a
device because one already has it.

On Mon, Jan 19, 2015 at 5:01 PM, Wesley W. Terpstra [off-list ref] wrote:

I have a raid5 array for which every disk, when examined with "mdadm
-E /dev/sd[abcd]4" shows a Version: 1.2 superblock with a valid
checksum.

However, when I try:
mdadm -A /dev/md/backing /dev/sd[abcd]4
mdadm says:
mdadm: failed to add /dev/sdc4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sdb4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sda4 to /dev/md/backing: Invalid argument
mdadm: failed to add /dev/sdd4 to /dev/md/backing: Invalid argument
mdadm: /dev/md/backing assembled from 0 drives - need all 4 to start
it (use --run to insist)
The kernel says:
md: sdc4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sdb4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sda4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
md: sdd4 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22

I was in the middle of a reshape of this 4-disk raid5 when something
rebooted the computer. The system seems otherwise fine, and I suspect
someone in the house.

What is the correct next step? Should I try --run ? I would obviously
prefer not to lose the data on this array. I expect that the reshape
was NOT complete, so just recreating the array will probably corrupt
its contents.

Kernel version 3.17.8 and mdadm version 3.3.2.

Thanks for any help. If I lose this array, I am going to face a lot of grief...
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi Wesley,

On 01/19/2015 06:01 PM, Wesley W. Terpstra wrote:

I was in the middle of a reshape of this 4-disk raid5 when something
rebooted the computer. The system seems otherwise fine, and I suspect
someone in the house.

What is the correct next step? Should I try --run ? I would obviously
prefer not to lose the data on this array. I expect that the reshape
was NOT complete, so just recreating the array will probably corrupt
its contents.

Kernel version 3.17.8 and mdadm version 3.3.2.

There have been many bug fixes to mdadm since that kernel was retired.
You should temporarily boot a current liveCD (my favorite is
systemrescuecd) and do "mdadm -Afv /dev/mdX /dev/sdX ..."

Show us the output of that if it doesn't work (it should resume your
reshape).

When it is done reshaping/recovering, consider upgrading your kernel.

Thanks for any help. If I lose this array, I am going to face a lot of grief...

I shouldn't have to say this, but RAID is for availability, not for data
security.  You still need a backup system for any important data.

Phil

Woops!

On 01/20/2015 01:28 PM, Phil Turmel wrote:

Hi Wesley,

On 01/19/2015 06:01 PM, Wesley W. Terpstra wrote:

quoted

I was in the middle of a reshape of this 4-disk raid5 when something
rebooted the computer. The system seems otherwise fine, and I suspect
someone in the house.

What is the correct next step? Should I try --run ? I would obviously
prefer not to lose the data on this array. I expect that the reshape
was NOT complete, so just recreating the array will probably corrupt
its contents.

Kernel version 3.17.8 and mdadm version 3.3.2.

There have been many bug fixes to mdadm since that kernel was retired.
You should temporarily boot a current liveCD (my favorite is
systemrescuecd) and do "mdadm -Afv /dev/mdX /dev/sdX ..."

I misread that kernel version.  It's relatively current.  Hmmm.

I'd still try a liveCD with 3.18.x or 3.19.

Show us the output of that if it doesn't work (it should resume your
reshape).

This still holds. :-)

Phil

On Tue, Jan 20, 2015 at 7:28 PM, Phil Turmel [off-list ref] wrote:

quoted

Kernel version 3.17.8 and mdadm version 3.3.2.

There have been many bug fixes to mdadm since that kernel was retired.

Really? I will give that a shot.

Show us the output of that if it doesn't work (it should resume your
reshape).

I will try that.

It was totally stupid reshape, I was changing the "data-offset" to
align it. Really unnecessary, and really slow, so I feel like an idiot
for starting it.

quoted

Thanks for any help. If I lose this array, I am going to face a lot of grief...

I shouldn't have to say this, but RAID is for availability, not for data
security.  You still need a backup system for any important data.

Sure. The really vital (and small) stuff I have backed up elsewhere,
but the sheer volume of raw video my girlfriend records is not
economically feasible to backup beyond raid+btrfs snapshots.

An email race condition. ;-)

On Tue, Jan 20, 2015 at 7:32 PM, Phil Turmel [off-list ref] wrote:

Woops!
I'd still try a liveCD with 3.18.x or 3.19.

Right. I am considering ordering one of these new 8TB hard disks so I
can raw copy the raid partitions before I try anything drastic.
Unfortunately even 8TB won't fit 4x3TB of data. :-/ I am fairly
certain my data is still good since all disks have a clean bill of
health and other partitions on them worked with bonnie. So, I figure
my biggest risk right now, is doing something else stupid.

The system remains bootable as the uefi+boot+root filesystems were on
RAID1 x4 arrays and not the data RAID5 array. I'm hesitant to take
action until I find away to store the raw partitions, but installing
3.19 and booting that from my existing root partition sounds harmless
enough.

The system does not have a CD-ROM and has insufficient SATA connectors
to attach one. I could try booting via a USB drive, though.