# Mystery RCWs

- From: Dallas Clement \<hidden\>
- Date: 2016-05-20 22:46:51 
- List: linux-raid
- Page: https://ratatoskr.run/linux-raid/2016/05/6632102
- Thread: https://ratatoskr.run/linux-raid/2016/05/6632102/t.md

```text
Hi All,

I've got a new little piece of NAS hardware I am working with and
trying to evaluate RAID 5, 6 performance on it.  It has an ARM based
Annapurna Alpine SoC which apparently has a built-in XOR engine to
help out with RAID 5, 6.  This little box has four spinning drives
which each have a transfer rate of about 150 MB/s.

I am only seeing a little over 200 MB/s sequential write throughput
for RAID 6 as measured with fio.  This is a bit disappointing.  I was
hoping that if writing stripe-aligned blocks I would get at least 250
MB/s.

I am measuring the peformance of writing 1M blocks to the array.  My
RAID 6 array was constructed with a 512KB chunk size.

# fio --runtime=30 --ramp_time=5 --numjobs=4 --iodepth=32
--ioengine=libaio --direct=1 --filename=/dev/md10 --bs=1M --rw=write
-name poi

I am expecting with a 512KB chunk size that I will be writing stripe
aligned blocks.

I then used blktrace to see what is going on suspecting that there
might be excessive RMWs.  I did not see any RMWs but I did see many
RCWs (about 20K).

 9,10   3       66     0.701873200 22465  Q  WS 347136 + 1024 [fio]
  9,10   3       67     0.701925540 22465  U   N [fio] 16
  9,10   3       68     0.702259400 22465  Q  WS 348160 + 1024 [fio]
  9,10   3       69     0.702324340 22465  Q  WS 349184 + 1024 [fio]
  9,10   3       70     0.702378960 22465  U   N [fio] 16
  9,10   3       71     0.702627200 22465  Q  WS 350208 + 1024 [fio]
  9,10   3       72     0.702668180 22465 UT   N [fio] 10
  9,10   0      287     0.666149800  6504  C  WS 300032 [0]
  9,10   0      288     0.666154080  6504  C  WS 299008 [0]
  9,10   0      289     0.666491900  6504  C  WS 301056 [0]
  9,10   0      290     0.666495620  6504  C  WS 302080 [0]
  9,10   0      291     0.669055940  6504  C  WS 305152 [0]
  9,10   0      292     0.669059720  6504  C  WS 306176 [0]
  9,10   0        0     0.671655160     0  m   N raid5 rcw 168960 1 1 0
  9,10   0        0     0.671671240     0  m   N raid5 rcw 169024 1 1 0
  9,10   0        0     0.671679220     0  m   N raid5 rcw 169088 1 1 0
  9,10   0        0     0.671684620     0  m   N raid5 rcw 169152 1 1 0
  9,10   0        0     0.671697260     0  m   N raid5 rcw 169216 1 1 0
  9,10   0        0     0.671702560     0  m   N raid5 rcw 169280 1 1 0
  9,10   0        0     0.671708620     0  m   N raid5 rcw 169344 1 1 0
  9,10   0        0     0.671713600     0  m   N raid5 rcw 169408 1 1 0

It seems that prior to the RCWs, this process 6504 starts doing some
writes.  It looks like this is the md driver.

# ps -elf | grep 6504
1 S root      6504     2  0  80   0 -     0 md_thr May19 ?
00:00:36 [md10_raid6]

Can anyone explain what the md driver may be doing and why it's
triggering all these RCWs?

By the way this is running on a Annapurna modified 3.10 kernel:

     3.10.20-031020-generic-sa #201311201536 SMP

Thanks,

Dallas
```