Von: linux-raid-owner@vger.kernel.org [linux-raid-owner@vger.kernel.org]" im Auftrag von "Matt Garman [matthew.garman@gmail.com]
Gesendet: Freitag, 31. Juli 2015 17:23
An: Mdadm
Betreff: Thoughts on big SSD arrays?
Every few years I reprise this topic on this mailing list[1], [2].
Basically I'm just brainstorming what is possible on the DIY front
versus purchased solutions from a traditional "big iron" storage
vendor. Our particular use case is "ultra-high parallel sequential
read throughput". Our workload is effectively WORM: we do a small
daily incremental write, and then the rest of the time it's constant
re-reading of the data. Literally 99:1 read:write
I continue to be inspired by the "Dirt Cheap Data Warehouse (DCDW)"
[3]. SSD are getting bigger and prices are dropping rapidly (2 TB
SSDs available now for $800). With our WORM-like workload, I believe
we can safely get away with consumer drives, as durability shouldn't
be an issue.
So at this point I'm just putting out a feeler---has anyone out there
actually built a massive SSD array, using either Linux software raid
or hardware raid (technically off-topic for this list, though I hope
the discussion is interesting enough to let it slide). If so, how big
of an array (i.e. drives/capacity)? What was the target versus actual
performance? Any particularly challenging issues that came up?
Hi Matt,
maybe not 100% matching... We wanted to start a 22 HDD md raid 6 last
year for one of our storage servers. Doing some tests we experienced
slow random write I/Os and got discouraged. So we headed over to
hardware raid.
To get a clearer picture we did further analysis. The bootleneck arised
from two sources. md issued only 4K I/Os and it always did a reconstruct
write. Thus massive write amplification. Several patches in 4.1 mitigate
the situation. In between I like a lot of things in md raid 4/5/6. Especially
the parity calucation massively benefits from high power CPUs.
Back to the reason for my explanation: The biggest "area under construction"
in md raid 4/5/6 is the internal stripe cache handling. Allocating, flushing
and freeing stripes is based on solid but not tuned algorithms (like LRU).
IIRC it does not even have a scheduling or queueing. That could become
your bottleneck. Maybe massive CPU power easily compensates the gap.
That said. Your read mostly setup might avoid a lot of headaches but you
need to push it to the limits yourself. I do not see benchmarks in the net
that closely fit your case. So I advise to enable ramdisks in the Linux
kernel and build a raid 6 on top of /dev/ramX. A synthetic test according
to your needs should give an idea of scalability and overhead of md raid.
Good luck.
Markus