Re: O_DIRECT to md raid 6 is slow

On Wed, Aug 15, 2012 at 4:50 AM, John Robinson
[off-list ref] wrote:

On 15/08/2012 01:49, Andy Lutomirski wrote:

quoted

If I do:
# dd if=/dev/zero of=/dev/md0p1 bs=8M

[...]

quoted

It looks like md isn't recognizing that I'm writing whole stripes when
I'm in O_DIRECT mode.

I see your md device is partitioned. Is the partition itself stripe-aligned?

Crud.

md0 : active raid6 sdg1[5] sdf1[4] sde1[3] sdd1[2] sdc1[1] sdb1[0]
      11720536064 blocks super 1.2 level 6, 512k chunk, algorithm 2
[6/6] [UUUUUU]

IIUC this means that I/O should be aligned on 2MB boundaries (512k
chunk * 4 non-parity disks).  gdisk put my partition on a 2048 sector
(i.e. 1MB) boundary.

Sadly, /sys/block/md0/md0p1/alignment_offset reports 0 (instead of 1MB).

Fixing this has no effect, though.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC