Re: Linux RAID metadata format docs?

On Friday November 14, jgarzik@pobox.com wrote:

Do any docs exist, that describe the on-disk md format?

Not to my knowledge, beyond source code.

I'm talking with a vendor that would like to support booting off of 
"Linux RAID" (as they call it), from their card's BIOS, which implies
the vendor would need to add support for the md format to their BIOS.

I don't see that knowledge of the RAID format is needed by the bios.

It might be needed by the boot sector, and certainly by the next stage
(whether that be the kernel itself, or grub or whatever).

Based on my limited understanding, the bios should only need:
  1/ to be able to try subsequent drives if the first fails,
  2/ be able to tell the boot sector which drive was used
  3/ to allow the boot sector to indicate failure
 and maybe
  4/ be able to have a dynamically determined boot order
   (so that when one drive fails, it gets put to the bottom of the 
    list).

Clearly 1,2,3 don't need the bios to know anything about RAID layout.  4
could use raid layout information, and an easy interface for the
kernel to tell it when things have changed would be a lot easier and
more general.

I think the simplest approach would be:

 Bios attempts each potential boot device in some order, loading the
 boot sector and telling the boot sector that the device is "0x80".
 If the boot sector fails (e.g. bad code, such as all zeros), it tries
 the next.

 The boot sector tries to load a kernel or second-stage off "0x80" as
 normal.   The kernel or second-stage understands raid and makes sure
 the correct root is assembled.

 Some daemon monitors drives that fail, and attempts to over-write
 sector 0 of any such drive that is a boot-candidate.

This wouldn't quite do the right thing if:
  The "first" drive goes off line for some reason.
  A new kernel/second-stage is installed on remaining drive(s).
  machine is rebooted and "first" drive comes back online during the
  boot.

The "first" drive will look OK, but will not have a current kernel on
it.

Does any of this make sense?

Were you thinking of some RAID level other than RAID1?

NeilBrown

Neil Brown wrote:

On Friday November 14, jgarzik@pobox.com wrote:

quoted

Do any docs exist, that describe the on-disk md format?

Not to my knowledge, beyond source code.

Thanks.

I don't see that knowledge of the RAID format is needed by the bios.

[...]

 The boot sector tries to load a kernel or second-stage off "0x80" as
 normal.   The kernel or second-stage understands raid and makes sure
 the correct root is assembled.

 Some daemon monitors drives that fail, and attempts to over-write
 sector 0 of any such drive that is a boot-candidate.

This wouldn't quite do the right thing if:
  The "first" drive goes off line for some reason.
  A new kernel/second-stage is installed on remaining drive(s).
  machine is rebooted and "first" drive comes back online during the
  boot.

The "first" drive will look OK, but will not have a current kernel on
it.

Does any of this make sense?

Were you thinking of some RAID level other than RAID1?

All RAID levels, 1 and 5 particularly.  To me, it boils down to a 
choice:  support in the bootloader, and/or support in the BIOS.  If the 
BIOS vendor is willing to support the _Linux_ md format, it seems to me 
that's a more reliable place to handle booting and failover in RAID1 or 
5.  OTOH, while coordinating multiple boot loaders (since one must 
predict behavior of lilo/grub in all raid1/5 failure scenarios) seems 
much more difficult, but ultimate not specific to one vendor.

By tradition the BIOS/firmware is the software that loads the 
bootloader.  It feels like the right thing to me, for that to continue 
even in the face of RAID arrays.  Users of "fake RAID" (i.e. the vendors 
market it as RAID, even though it's really software RAID) are used to 
their RAID arrays being accessible as a BIOS device, as weird as that 
may be :)

	Jeff