Thomas Jager wrote:

Hi list.

I run a file server on MD raid-5.
If a client reads one big file and at the same time another client 
tries to write a file, the thread writing just sits in uninterruptible 
sleep until the reader has finished. Only very small amount of writes 
get trough while the reader is still working.
I'm having some trouble pinpointing the problem.
It's not consistent either sometimes it works as expected both the 
reader and writer gets some transactions. On huge reads I've seen the 
writer blocked for 30-40 minutes without any significant writes 
happening (Maybe a few megabytes, of several gigs waiting). It happens 
with NFS, SMB and FTP, and local with dd. And seems to be connected to 
raid-5. This does not happen on block devices without raid-5. I'm also 
wondering if it can have anything to do with loop-aes? I use loop-aes 
on top of the md, but then again i have not observed this problem on 
loop-devices with disk backend. I do know that loop-aes degrades 
performance but i didn't think it would do something like this?

I've seen this problem in 2.6.16-2.6.21

All disks in the array is connected to a controller with a SiI 3114 chip.

I just noticed something else. A couple of slow readers where running on 
my raid-5 array. Then i started a copy from another local disk to the 
array. Then i got the extremely long wait. I noticed something in iostat:

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.90    0.00   48.05   31.93    0.00   16.12

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
....
sdg               0.80        25.55         0.00        128          0
sdh             154.89       632.34         0.00       3168          0
sdi               0.20        12.77         0.00         64          0
sdj               0.40        25.55         0.00        128          0
sdk               0.40        25.55         0.00        128          0
sdl               0.80        25.55         0.00        128          0
sdm               0.80        25.55         0.00        128          0
sdn               0.60        23.95         0.00        120          0
md0             199.20       796.81         0.00       3992          0

All disks are member of the same raid array (md0). One of the disks has 
a ton of transactions compared to the other disks. Read operations as 
far as i can tell. Why? May be connected with my problem?