On Mon, 3 Nov 2014 10:43:29 +0100 Caspar Smit [off-list ref] wrote:

Hi Neil,

Actually BOTH your answers were correct, thank you for that.

1) Your hunge was correct as my disk contained a partition table (in
my case an msdos label) and was not added with the error in my first
mail:

mdadm: no RAID superblock on /dev/sdd.

mdadm -E /dev/sdd shows:

/dev/sdd:
   MBR Magic : aa55

So it finds 'something' but clearly unusable to mdadm.

Wiping the partition table and trying again resulted in a different
error message:

mdadm: no recognisable superblock on /dev/sdd.

Which is better but still the disk was not added to the array.

2) To make it work i also needed the domain=default in the POLICY setting.

It still gave me the:

mdadm: no recognisable superblock on /dev/sdd.

But now the disk got added to the array and started rebuilding.

Note: ONLY setting the domain=default in POLICY without clearing the
partition table results in:
mdadm: no RAID superblock on /dev/sdd. and the disk will not be added
so BOTH measures were needed.

Thanks for testing and reported.... the patch I posted before (included more
completely below) should allow "domain=default" to be enough.

Note2: I didn't need the spare-group directive so I think
domain=default is a special case were all disks and arrays are placed
in the same domain.

"spare-group" is really only for "legacy" support.  If a domain is defined
for disks, the array made up of those disks inherits the domain.

Furthermore i found out something which i think should not happen
(bug?) or maybe i am wrong:

With a working clean array:

# more /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd[3] sdc[1] sdb[0]
      203776 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

# mdadm --fail /dev/md0 /dev/sdd
mdadm: set /dev/sdd faulty in /dev/md0

# mdadm --remove /dev/md0 /dev/sdd
mdadm: hot removed /dev/sdd from /dev/md0

# mdadm --incremental /dev/sdd
mdadm: failed to add /dev/sdd to /dev/md/0: Invalid argument.

So when it actually finds a device with an MD superblock it doesn't
add it, is this expected behavior as the disk was failed (so probably
not a good idea to add it back) or is this a bug?

Presumably "action=force-spare" was still active when you tried this?

In that case it is a bug (I think).  It should clean-out the device and add
it as a spare...

I just tested with mdadm from my 'git', and  it works as expected.
When action=force-spare  I get

mdadm: /dev/loop2 attached to /dev/md0 which is already active.

When I have "action=re-add" I get:

mdadm: can only add /dev/loop2 to /dev/md0 as a spare, and force-spare is not set.
mdadm: failed to add /dev/loop2 to existing array /dev/md0: Invalid argument.

Maybe you need a newer mdadm ...

Thanks,
NeilBrown


From: NeilBrown <redacted>
Date: Wed, 5 Nov 2014 16:21:42 +1100
Subject: [PATCH] Incremental: don't be distracted by partition table when
 calling try_spare.

Currently a partition table on a device makes "mdadm -I" think
the array has a particular metadata type and so will only
add it to an array of that (partition table) type .. which doesn't
make any sense.

So tell guess_super to only look for 'array' metadata.

Reported-by: Caspar Smit <redacted>
Signed-off-by: NeilBrown <redacted>

diff --git a/Incremental.c b/Incremental.c
index c9372587f518..13b68bc0adea 100644
--- a/Incremental.c
+++ b/Incremental.c

@@ -196,13 +196,13 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
 	policy = disk_policy(&dinfo);
 	have_target = policy_check_path(&dinfo, &target_array);
 
-	if (st == NULL && (st = guess_super(dfd)) == NULL) {
+	if (st == NULL && (st = guess_super_type(dfd, guess_array)) == NULL) {
 		if (c->verbose >= 0)
 			pr_err("no recognisable superblock on %s.\n",
 			       devname);
 		rv = try_spare(devname, &dfd, policy,
 			       have_target ? &target_array : NULL,
-			       st, c->verbose);
+			       NULL, c->verbose);
 		goto out;
 	}
 	st->ignore_hw_compat = 1;