Thanks for the response.

Nice try... But, the reason I’m using the 3.18.4 kernel is that it has the parallelization.  I've got group_thread_cnt set to 32.  I'm watching the CPU's with mpstat, and they're pretty much idle.  I'm also watching the system traces with perf.  It claims that only 11.9% of my time is spent doing the xor.

I've got my CS set at 128k.  I have noticed that if I set the CS to 32k, the TP is about 2x.  I'm pretty sure the problem is that the 1M writes I'm doing are being broken into 4K pages, and then reassembled before going to disk.

Also, this is independent of the IO Scheduler.  I've tried all 3 and got the same results.

Bob Kierski
Senior Storage Performance Engineer
Cray Inc.
380 Jackson Street
Suite 210
St. Paul, MN 55101
Tele: 651-967-9590
Fax:  651-605-9001
Cell: 651-890-7461


-----Original Message-----
From: Phil Turmel [mailto:philip@turmel.org] 
Sent: Wednesday, December 02, 2015 8:45 AM
To: Robert Kierski; Dallas Clement
Cc: linux-raid@vger.kernel.org
Subject: Re: RAID 5,6 sequential writing seems slower in newer kernels

On 12/02/2015 09:18 AM, Robert Kierski wrote:

Sorry... I should have mentioned that I'm running the 3.18.4 kernel 
with a 32 core xeon and 128G of memory.  I'm not using a FS, I'm going 
directly to the raid block device.

I'm not sure if the parallelization of raid parity has been merged yet, but I'm pretty sure it isn't in 3.18.  With one core tied up computing parity and the rest idle, that'd be 96.875% idle.

Phil