On Thu, Dec 29, 2016 at 4:04 PM, Adam Goryachev
[off-list ref] wrote:
On 30/12/16 03:56, Robert LeBlanc wrote:
quoted
This is a similar workload as Ceph and you may find more information
from their mailing lists. When I was working with Ceph about a year
ago, we tested a bunch of SSDs and found that sync=1 really
differentiates drives and you really find which drives are better. In
our testing, we found that the 35xx, 36xx, and 37xx drives handled the
workloads the best. The 3x00 drives were close to EOL, so we focused
on the 3x10 drives. I don't have the data anymore, but the 3610 had
the best performance, the 3710 had the best data integrity in the case
of power failure, and the 3510 had the best price.
So it seems that my "good/best" results were based on the 3510, which was
the cheapest out of the options you tested. Any chance you could find the
raw data again? Or do you recall the relative performance difference between
these three drives?
This was done at another job and the data stayed when I left. The
performance between the three drives were pretty close, I think less
than 10%, but I can't remember exactly.
quoted
The 3510 had about
~.1 drive writes per day, the 3610 had ~1 DWPD and the 3710 had ~3
DWPD.
We seem to be around 0.03 DWPD, so I don't think any of these drives would
be a problem for us. Lifetime seems much longer than the useful life, given
capacity/etc.
We had really good wear on the 35xx drives, I think they are
understated, but I don't have the data to back that up.
quoted
Due to the fault tolerance of Ceph, we felt comfortable with the
3610s.
Equally, we have fault tolerance (RAID5) as well as DRBD onto the other node
which also has RAID5. I also monitor the drive lifetime, I'm not sure what
value I would consider urgent replacement, but probably around 20% remaining
life....
You may never even get there at 0.03 DWPD.
quoted
In our testing, we exceeded the performance numbers listed for
the drives on their data sheets when running up to 8 jobs even with
sync=1 which no other manufacture did. For Ceph, we could put multiple
OSDs on a disk and take advantage of this performance gain. You may be
able to do something similar by partitioning your RAID 5 and putting
multiple DRBDs on it.
We do this already... we use a single RAID5 which is split with LVM2 (20
LV's), and each LV is then a DRBD device (so 20 DRBD's). This was one of the
optimisations linbit advised us to do way back at the beginning.
The problem I'm having is that a single DRBD will reach saturation because
the underlying devices are saturated. So I'm trying to improve the
underlying device performance, and expect to be able to "move" the
bottleneck to DRBD or hopefully, the ethernet of the iSCSI interface.
Regards,
Adam
----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1