-----Original Message-----
From: Jason Warr [mailto:jason-/cow75dQlsI@public.gmane.org]
Sent: Friday, January 18, 2013 9:26 PM
To: Amit Kale
Cc: thornber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; device-mapper development;
kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; Mike Snitzer; LKML; linux-
bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [dm-devel] Announcement: STEC EnhanceIO SSD caching
software for Linux kernel


On 01/18/2013 03:08 AM, Amit Kale wrote:

quoted

quoted

quoted

Can you explain what you mean by that in a little more detail?

Let's say latency of a block device is 10ms for 4kB requests. With

single threaded IO, the throughput will be 4kB/10ms = 400kB/s. If the
device is capable of more throughput, a multithreaded IO will generate
more throughput. So with 2 threads the throughput will be roughly
800kB/s. We can keep increasing the number of threads resulting in an
approximately linear throughput. It'll saturate at the maximum capacity
the device has. So it could saturate at perhaps at 8MB/s. Increasing
the number of threads beyond this will not increase throughput.

quoted

This is a simplistic computation. Throughput, latency and number of

threads are related in a more complex relationship. Latency is still
important, but throughput is more important.

quoted

The way all this matters for SSD caching is, caching will typically

show a higher latency compared to the base SSD, even for a 100% hit
ratio. It may be possible to reach the maximum throughput achievable
with the base SSD using a high number of threads. Let's say an SSD
shows 450MB/s with 4 threads. A cache may show 440MB/s with 8 threads.

quoted

A practical difficulty in measuring latency is that the latency seen

by an application is a sum of the device latency plus the time spent in
request queue (and caching layer, when present). Increasing number of
threads shows latency increase, although it's only because the requests
stay in request queue for a longer duration. Latency measurement in a
multithreaded environment is very challenging. Measurement of
throughput is fairly straightforward.

quoted

quoted

quoted

As an enterprise level user I see both as important overall.
However, the biggest driving factor in wanting a cache device in
front of any sort of target in my use cases is to hide latency as
the number of threads reading and writing to the backing device go
up.  So for me the cache is basically a tier stage where your
ability to keep dirty blocks on it is determined by the specific

use case.

quoted

SSD caching will help in this case since SSD's latency remains almost

constant regardless of location of data. HDD latency for sequential and
random IO could vary by a factor of 5 or even much more.

quoted

Throughput with caching could even be 100 times the HDD throughput

when using multiple threaded non-sequential IO.

quoted

-Amit

Thank you for the explanation.  In context your reasoning makes more
sense to me.

If I am understanding you correctly when you refer to throughput your
speaking more in terms of IOPS than what most people would think of as
referencing only bit rate.

I would expect a small increase in minimum and average latency when
adding in another layer that the blocks have to traverse.  If my
minimum and average increase by 20% on most of my workloads, that is
very acceptable as long as there is a decrease in 95th and 99th
percentile maximums.  I would hope that absolute maximum would decrease
as well but that is going to be much harder to achieve.

If I can help test and benchmark all three of these solutions please
ask.  I have allot of hardware resources available to me and perhaps I
can add value from an outsiders perspective.

That'll be great. I have so far marked EIO's status as alpha. Will require a little more functionality testing before performance. Perhaps in a week or so.

-Amit

PROPRIETARY-CONFIDENTIAL INFORMATION INCLUDED



This electronic transmission, and any documents attached hereto, may contain confidential, proprietary and/or legally privileged information. The information is intended only for use by the recipient named above. If you received this electronic message in error, please notify the sender and delete the electronic message. Any disclosure, copying, distribution, or use of the contents of information received in error is strictly prohibited, and violators will be pursued legally.