Re: end to end error recovery musings

quoted

quoted

quoted

quoted

"Alan" == Alan  [off-list ref] writes:

quoted

These features make the most sense in terms of WRITE.  Disks
already have plenty of CRC on the data so if a READ fails on a
regular drive we already know about it.

Alan> Don't bet on it. 

This is why I mentioned that I want to expose the protection data to
the host.  As written, DIF only protects the path between initiator
and target.

See below...


Alan> If you want to do this seriously you need an end to end (media
Alan> to host ram) checksum. We do see bizarre and quite evil things
Alan> happen to people occasionally because they rely on bus level
Alan> protection - both faulty network cards and faulty disk or
Alan> controller RAM can cause very bad things to happen in a critical
Alan> environment and are very very hard to detect and test for.

Not sure you're up-to-date on the T10 data integrity feature.
Essentially it's an extension of the 520 byte sectors common in disk
arrays.  For each 512 byte sector (or 4KB ditto) you get 8 bytes of
protection data.  There's a 2 byte CRC (GUARD tag), a 2 byte
user-defined tag (APP) and a 4-byte reference tag (REF).  Depending on
how the drive is formatted, the REF tag usually needs to match the
lower 32-bits of the target sector #.

For each sector coming in the disk firmware verifies that the CRC and
the reference tags are in accordance with the contents of the sector
and the CDB start sector + offset.  If they don't match the drive will
reject the request.

If an HBA is capable of exposing the protection tuples to the host we
can precalculate the checksum and the LBA when submitting a WRITE.  My
current proposal involves passing them down in two separate buffers to
minimize the risk of in-memory corruption (Besides, it would suck if
you had to interleave data and protection data.  The scatterlists
would become long and twisted).

And that's when the READ case becomes interesting.  Because then the
fs can verify that the checksum of the in-buffer matches of the GUARD
tag.  In that case we'll know there's been no corruption in the
middle.

And of course this also opens up using the APP field to tag sector
contents.

-- 
Martin K. Petersen	Oracle Linux Engineering

Not sure you're up-to-date on the T10 data integrity feature.
Essentially it's an extension of the 520 byte sectors common in disk

I saw the basics but not the detail. Thanks for the explanation it was
most helpful and promises to fix a few things for some controllers.. but
here's a minor bit of passing bad news - quite a few older ATA controllers
can't issue DMA transfers that are not a multiple of 512 bytes without
crapping themselves (eg READ_LONG). Guess we may need to add
ap->i_do_not_suck or similar 8)

On the bright side I believe the Intel ICH is the only one with this
problem (and a workaround) which is SATA capable 8)

Alan

On Tuesday, February 27, 2007 12:07 PM, Martin K. Petersen wrote: 

Not sure you're up-to-date on the T10 data integrity feature.
Essentially it's an extension of the 520 byte sectors common in disk
arrays.  For each 512 byte sector (or 4KB ditto) you get 8 bytes of
protection data.  There's a 2 byte CRC (GUARD tag), a 2 byte
user-defined tag (APP) and a 4-byte reference tag (REF).  Depending on
how the drive is formatted, the REF tag usually needs to match the
lower 32-bits of the target sector #.

I from the scsi lld perspective, all we need 32 byte cdbs, and a
mechinism to pass the tags down from above.  It appears our driver to
firmware insterface is only providing the reference and application
tags. It seems the guard tag is not present, so I guess mpt fusion
controller firmware is setting it(I will have to check with others).   I
assume that for transfers greater than a sector, that the controller
firmware updates the tags for all the other sectors within the boundary.
I'm sure the flags probably tell whether EEDP is enabled or not.   I
will have to check if there are some manufacturing pages that say
whether the controller is capable of EEDP(as not all our controllers
support it).  


Here are the EEDP associated fields we provide in our scsi passthru, as
well as target assist.


u32 SecondaryReferenceTag
u16 SecondaryApplicationTag
u16 EEDPFlags
u16 ApplicationTagTranslationMask
u32 EEDPBlockSize