Thanks, Asai!  I don't think cfq is the ideal I/O scheduler to be
testing.  Could you run again with deadline and/or noop and see how that
changes your throughput and perf report?  Also, just for completeness,
could you tell us which kernel you ran this against?

How many processors is this system, just looking at the lock contention
which is pretty horrible.

I'd been expecting various red flags in the AHCI/libata/scsi queue code
but it seems at first glance that the block queue stuff is killing us and
the scsi/ata code is a distraction (unless of course its causing a lot of
the lock time)

No-op would be most interesting but the I/O scheduler numbers don't look
pretty.

Alan