Re: Triple parity and beyond

On 20/11/13 19:09, John Williams wrote:

On Wed, Nov 20, 2013 at 2:31 AM, David Brown [off-list ref] wrote:

quoted

That's certainly a reasonable way to look at it.  We should not limit
the possibilities for high-end systems because of the limitations of
low-end systems that are unlikely to use 3+ parity anyway.  I've also
looked up a list of the processors that support SSE3 and PSHUFB - a lot
of modern "low-end" x86 cpus support it.  And of course it is possible
to implement general G(2^8) multiplication without PSHUFB, using a
lookup table - it is important that this can all work with any CPU, even
if it is slow.

Unfortunately, it is SSSE3 that is required for PSHUFB. The SSE3 set
with only two-esses does not suffice. I made that same mistake when I
first heard about Andrea's 6-parity work. SSSE3 vs. SSE3, confusing
notation!

SSSE3 is significantly less widely supported than SSE3. Particularly
on AMD, only the very latest CPUs seem to support SSSE3. Intel support
for SSSE3 goes back much further than AMD support.

Maybe it is not such a big problem, since it may be possible to
support two "roads". Both roads would include the current md RAID-5
and RAID-6. But one road, which those lacking CPUs supporting SSSE3
might choose, would continue on to the non-SSSE3 triple-parity 2^-1
technique, and then dead-end. The other road would continue with the
Cauchy matrix technique through 3-parity all the way to 6-parity.

It might even be feasible to allow someone stuck at the end of the
non-SSSE3 road to convert to the Cauchy road. You would have to go
through all the 2^-1 triple-parity and convert it to Cauchy
triple-parity. But then you would be safely on the Cauchy road.

I would not like to see two alternative triple-parity solutions - I
think that would lead to confusion, and a non-Cauchy triple parity would
not be extendible without a rebuild (I've talked before about the idea
of temporarily adding an extra parity drive with an asymmetric layout.
I really like the idea, so I keep pushing for it!).

I think it is better to accept that 3+ parity will be slow on processors
that don't support PSHUFB.  We should try to find the best alternative
SIMD for other realistic processors (such as on AMD chips without
PSHUFB, ARM's with NEON, PPC with Altivec, etc.) - but a simple table
lookup will always work as a fallback.  Other than that I think it is
fair to say that if you want /fast/ 3+ parity, you need a reasonably
modern non-budget-class cpu.