On Fri, Sep 22, 2023 at 10:36 AM Stan Liao [off-list ref] wrote:

Xiao Ni wrote:

quoted

On Thu, Sep 21, 2023 at 11:22 AM Stan Liao [off-list ref] wrote:

quoted

Hi,
A md-raid (level 1) is created with 5 nvme drives and the metadata version

is specified as 1.2. The following command is used.

quoted

sudo mdadm --create /dev/md0 --level=1 --raid-devices=5
/dev/nvme{1,2,3,4}n1 /dev/nvme4n2 --metadata=1.2 The capacities of

nvme{1,2,3,4}n1 and nvme4n2 are 3.2TB, 1.92TB, 3.2TB, 512GB, and 512GB.

quoted

OS: 20.04.1-Ubuntu
mdadm version: v4.1 - 2018-10-01
After creation, we found that the size of the bitmap_super_t and internal

bitmap is 16KB (this is concluded by observing FF value is filled from around
byte offset 0x100 of LBA 0x10 to byte offset 0x1FF of LBA 0x1F), but the
mdp_superblock_1.bblog_offset value is 0x10. As a result, the
mdp_superblock_1 occupies LBA 0x08 ~ 0x0F; bitmap_super_t and internal
bitmap occupy LBA 0x10 ~ 0x20; bblog occupies LBA 0x18 ~ 0x20.

quoted

If bblog and bitmap do overlap, I would like to know the size value used to

calculate bitmap size and bblog_offset. The size value used to calculate
bitmap size and bblog_offset is mdp_superblock_1.size or
mdp_superblock_1.data_size? Thanks a lot.

quoted

Hi

For super1.2 the layout should be:
|    4KB    |    4KB    |    bitmap space    |    head room    |
data |

The first 4KB is empty from the beginning of the disk. The second 4KB is for
md superblock. Then is bitmap suerblock. So if you want to see the content
of bitmap_super_t, the offset should be 0x2000?

Regards
Xiao

Right. The bitmap_super_t is at offset 0x2000 (absolute LBA address 0x10).
And mdadm write bitmap_super_t and bitmap from LBA 0x10 to LBA 0x1F.
Including bblog, super1.2 layout would be:
|  4KB  |  4KB  |  bitmap space  |  bblog  |  head room  | data |
We observe the bblog_offset value, which resides in mdp_superblock_1, is 0x10.
So we think there is an overlap between bitmap and bblog.

Because the bblog_offset is based on the start of superblock. So if
you want to calculate the offset based on the start of the disk, it
needs to add 4KB on bblog_offset. Now your bblog_offset is 0x10. It
means from the start of the superblock, it's 4K(superblock) +
4K(bitmap space). So your raid only has one page, right?

You can also you mdadm -E $member to check the "Bad Block Log"

Regards
Xiao