Re: Self inflicted reshape catastrophe

That command wouldn't have the effect you describe (and is visible in
the --examine output - thanks).

Maybe you mean "--add" ???

I feel like I did `--assemble` but it is entirely possible I am not
remembering correctly and did an `--add`. It may be worth testing both
scenarios to see if `mdadm` could stop people like me from shooting
themselves in the foot. To be clear I am not placing any blame on
`mdadm` for my stupid mistakes.

To reconstruct the correct metadata, the easiest approach is probably to
copy the superblock from the best drive in md0 and use a binary-editor
to change the 'Device Role' field to an appropriate number for each
different device.  Maybe your kernel logs will have enough info to
confirm which device was in each role.

I checked all my logs and all I can see is the start of the reshape
with no indication of what is changing.

Jan 14 00:20:46 nas2 kernel: md: reshape of RAID array md0

One approach to copying the metadata is to use "mdadm --dump=/tmp/md0 /dev/md0"
which should create sparse files in /tmp/md0 with the metadata from each
device.
Then binary-edit those files, and rename them. Then use
   mdadm --restore=/tmp/md0 /dev/md0
to copy the metadata back.  Maybe.

Then use "mdadm --examine --super=1.2" to check that the superblock
looks OK and to find out what the "expected" checksum is.  Then edit the
superblock again to set the checksum.

Then try assembling the array with
   mdadm --assemble --freeze-reshape --readonly ....

which should minimize the damage that can be done if something isn't
right.
Then try "fsck -n" the filesystem to see if it looks OK.

Since I do not know the disk order I'll work up a script to use
overlays and craft these super blocks in each permutation with `fsck`
checks to see which one gets me the most data back.

I truly appreciate the help, thank you very much. I'll let you all
know how it goes :)