"Asai Thambi Samymuthu Pattrayasamy (asamymuthupa) [CONTRACTOR]"
[off-list ref] writes:

quoted

2.6.39 introduced the on-stack plugging work from Jens, the intent of
which is to reduce queue lock contention.  It would be great if you
could run with that kernel and noop to see if we make up some of the
performance gap (which looks to be just north of 10%).

Asai, thanks for running these tests and providing all of this data!

-Jeff

At least with noop and deadline for ahci driver, now queue lock is not
the top offender. With the new change in plugging, seems noop and
deadline are spending more time in processing I/O similar to Micron
block driver.

With the current test results with 2.6.39.1 (new optimization for
plugging) and application queue depth of 32, 
     * Micron block driver exhibits 43% better IOPS than ahci driver
with noop
     * Micron block driver slightly better in CPU utilization.

With application queue depth of 256, Micron block driver is able to
leverage the device capability, and hence performance increases more
than 225%.

From the perf report, I would have guessed that the CPU utilization for

the ahci test case would have been lower than the Micron block driver.
Odd, I wonder what I'm missing.  Asai, did you notice if any of the CPUs
was completely pegged during testing with ahci?  You're using a NUMA
box, right?  I also wonder what the irq distribution looked like, and
whether rq_affinity is hurting performance for the ahci case.  Also,
does the Micron driver do any sort of interrupt coalescing that maybe
the ahci driver isn't doing?

Anywho, a 40% difference is pretty significant (though NUMA can have
that sort of impact).  Alan, what do you think?  I was never clear on
how exactly the ahci driver would handle a queue depth larger than 32
(if it can't, then clearly we'd need a block driver for this hardware).

Cheers,
Jeff