Re: Weirdness with DDF arrays (mdadm 3.3)

On 09/14/2013 05:25 PM, Francis Moreau wrote:

Hi Martin,

On Sat, Sep 14, 2013 at 4:16 PM, Martin Wilck [off-list ref] wrote:

quoted

Hi Francis, hi Neil,

quoted

Still testing MD arrays using DDF metadata and find another possible issues :)

I'm creating a new DDF array containing 2 disks. After that
/proc/mdstat looks correct:

# cat /proc/mdstat
Personalities : [raid1]
md124 : active raid1 loop0[1] loop1[0]
      84416 blocks super external:/md125/0 [2/2] [UU]

md125 : inactive loop1[1](S) loop0[0](S)
      65536 blocks super external:ddf

Now I'm stopping the array and restart it by incrementaly adding the 2 disks:
# mdadm --stop /dev/md124
# mdadm --stop /dev/md125
# mdadm -IRs /dev/loop0

This is wrong, because -IRs "wills can the mapfile for arrays that are
being incrementally assembled snd will try to start any that are not
already started".

mdadm -IRs will first add /dev/loop0, then see that there is an
incomplete array, and start it.

quoted

# mdadm -IRs /dev/loop1

Now you add /dev/loop1, but as the array is already started, it will be
added as a spare. That's what you see below.

Ah you're right, sorry for the noise, however I still doesn't
understand my last point, please see the end of the email.

quoted

However, there is room for improvement here. The array hasn't been
written to, so even if it is started, it should be possible to re-add
the second disk cleanly.

Looking into that.

thanks

quoted

Martin

quoted

# cat /proc/mdstat
Personalities : [raid1]
md124 : active (auto-read-only) raid1 loop1[2] loop0[0]
      84416 blocks super external:/md125/0 [2/1] [_U]

md125 : inactive loop1[1](S) loop0[0](S)
      65536 blocks super external:ddf

Parsing mdstat content tells me disk "loop1" have a role number equal
to 2 which is greater than 1 indicating that "loop1" is a spare disk
and the "[_U]" below indicates "loop1" is down".

Why is "loop1" down now ?

I decided to still use the md device by creating a new partition on it:
# fdisk /dev/md124
...
Calling ioctl() to re-read partition table.
Syncing disks.

Now inspecting /proc/mdstat:

# cat /proc/mdstat
Personalities : [raid1]
md124 : active raid1 loop1[2] loop0[0]
      84416 blocks super external:/md125/0 [2/2] [UU]

md125 : inactive loop1[1](S) loop0[0](S)
      65536 blocks super external:ddf

which looks even weirder: "loop1[2]" indicates that the disk is a
spare one whereas "[UU]" tells me the opposite.

Could you tell me if I'm wrong in my interpretation or what's going wrong ?

What about the loop1 in spare and [UU] indicating that loop1 is used ?

After you added loop1, the array was in read-auto state. Rebuild doesn't
start in this state.

When you created the partition table, the array went in active state and
was rebuilt. When you looked at mdstat again, the rebuild was already
finished. Therefore you got "[UU]" after that.

Wrt loop1[2], I think you interpret the [2] wrongly. It seems to be the
kernel index of the device somehow. The mdstat parsing code of mdadm
doesn't look at this number. If you look at
/sys/class/block/md124/md/dev-loop*/slot, the number should be correct -
I tried it here.

Perhaps someone else can comment on this, too.

/proc/mdstat can be pretty cryptic. If you are uncertain about anything,
look under /sys.

Martin

Thanks