This is terrific.  fsck found tons and tons of errors and fixed them
all.  Then I ran rsync -avHcn [array] [backup] and found 5 or so files
out of 8tb which had some slight corruption, which can easily be
restored from the backup but I was curious to do a dry run first to
use vbindiff and see what the corruption looked like at the byte
level.  Interesting!

I did notice that before rsync found one of the differences (corrupt
files) it started spitting out those same "failed
command: READ FPDMA QUEUED status: { DRDY ERR } error: { UNC }" errors
as before but this time it did not fail the drive.  I take this to
mean there is still some physical problems with the drive, but with
the new timeout settings it is not unnecessarily failing the drive out
of the array.  So if I overwrite the corrupted files with the backups,
(or write any new data to the array really), will it avoid those
problem areas on the platter?

I just want to say a big thanks as this has been causing me
indescribable stress and monetary cost for months since the beginning
of february and it looks like I am back in business.  I think I will
write some perl scripts to help monitor some of these things.