Stan Hoeppner <stan <at> hardwarefreak.com> writes:

On 3/27/2014 10:23 AM, Bernd Schubert wrote:
...

quoted

quoted

commit 508a7f16b242d6c3353e15aab46ac8ca8dc7cd08
Author: NeilBrown <neilb <at> suse.de>
Date:   Wed Apr 4 14:00:40 2012 +1000

    super1: leave more space in front of data by default.

    The kernel is growing the ability to avoid the need for a
    backup file during reshape by being able to change the data offset.

    For this to be useful we need plenty of free space before the
    data so the data offset can be reduced.

    So for v1.1 and v1.2 metadata make the default data_offset much
    larger.  Aim for 128Meg, but keep a power of 2 and don't use more
    than 0.1% of each device.

    Don't change v1.0 as that is used when the data_offset is required to
    be zero.

This is a good match because the discrepancy on his RAID6 is pretty
close to 128MB per drive.  However, both his RAID5 and RAID6 arrays are
metadata 1.2.  So this commit alone may not fully explain the capacity
difference he's seeing between RAID5 and RAID6.  Or is this commit RAID6
specific?  I don't see that in the comments above.

Cheers,

Stan

Yes, it's not fully explain the difference. Here's another test with mdadm
v3.2.6 (which already have the 508a7f16b242d6c3353e15aab46ac8ca8dc7cd08
commit) in CentOS 6.5.

Created a RAID 5 array with 5 1GiB loop devices, the data_offset of this new
array is only 1024 sectors (0.5M).

# MAKEDEV /dev/loop

# for hdd in {1..5}; do truncate -s 1G hdd5$hdd; losetup /dev/loop5$hdd
hdd5$hdd; done

# mdadm -C /dev/md5 -l5 -n4 -x1 /dev/loop5[1-5]

# mdadm --version
mdadm - v3.2.6 - 25th October 2012

# mdadm --examine /dev/loop51
/dev/loop51:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 5fb7e243:01760f20:c773f451:09928fa0
           Name : RecordFileServer:5  (local to host RecordFileServer)
  Creation Time : Fri Mar 28 10:14:36 2014
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 2096128 (1023.67 MiB 1073.22 MB)
     Array Size : 3142656 (3.00 GiB 3.22 GB)
  Used Dev Size : 2095104 (1023.17 MiB 1072.69 MB)
    Data Offset : 1024 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : d662c632:b2aafbac:406c84a6:ae519102

    Update Time : Fri Mar 28 10:15:07 2014
       Checksum : ab92e515 - correct
         Events : 20

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)

# mdadm -D /dev/md5
/dev/md5:
        Version : 1.2
  Creation Time : Fri Mar 28 10:14:36 2014
     Raid Level : raid5
     Array Size : 3142656 (3.00 GiB 3.22 GB)
  Used Dev Size : 1047552 (1023.17 MiB 1072.69 MB)
   Raid Devices : 4
  Total Devices : 5
    Persistence : Superblock is persistent

    Update Time : Fri Mar 28 10:14:56 2014
          State : clean 
 Active Devices : 4
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 512K

           Name : RecordFileServer:5  (local to host RecordFileServer)
           UUID : 5fb7e243:01760f20:c773f451:09928fa0
         Events : 20

    Number   Major   Minor   RaidDevice State
       0       7       51        0      active sync   /dev/loop51
       1       7       52        1      active sync   /dev/loop52
       2       7       53        2      active sync   /dev/loop53
       5       7       54        3      active sync   /dev/loop54

       4       7       55        -      spare   /dev/loop55