[PATCH 1/3] Support for PCI Express reset type

This is the first of three patches that implement a bit field that PCI Express device drivers can use to indicate they need a fundamental reset during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot reset" during recovery of a PCI Express device.  We've found a case where the device needs a "fundamental reset" to recover properly.  The current PCI error recovery and EEH frameworks do not support this distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to pci_dev that indicates whether the device requires a fundamental reset during recovery.

These patches supersede the previously submitted patch that implemented a fundamental reset bit field. 

Please review and let me know of any concerns.

Signed-off-by: Mike Mason <redacted>
Signed-off-by: Richard Lary <redacted>
 

Mike Mason [off-list ref] writes:

These patches supersede the previously submitted patch that
implemented a fundamental reset bit field. 

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC
specific hack. And where is the driver that is using it?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

On Fri, 31 Jul 2009, Andi Kleen wrote:

Mike Mason [off-list ref] writes:

quoted

These patches supersede the previously submitted patch that
implemented a fundamental reset bit field. 

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC
specific hack.

Are there any non-PPC platforms which support EEH slot-reset?

And where is the driver that is using it?

That would be the qla2xxx (FC/FCoE storage) and qlge (10gb NIC
) drivers.  Changes to the drivers could look something like:

Index: b/drivers/net/qlge/qlge_main.c
===================================================================

--- a/drivers/net/qlge/qlge_main.c
+++ b/drivers/net/qlge/qlge_main.c

@@ -4289,6 +4289,9 @@ static int __devinit qlge_probe(struct p
 		return err;
 	}
 
+	/* Set EEH reset type to fundamental for this device */
+	pdev->needs_freset = 1;
+
 	qdev = netdev_priv(ndev);
 	SET_NETDEV_DEV(ndev, &pdev->dev);
 	ndev->features = (0

Index: b/drivers/scsi/qla2xxx/qla_os.c
===================================================================

--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c

@@ -1773,6 +1773,10 @@ qla2x00_probe_one(struct pci_dev *pdev, 
 	/* Set ISP-type information. */
 	qla2x00_set_isp_flags(ha);
 
+	/* Set EEH reset type to fundamental if required by hba  */
+	if (IS_QLA24XX(ha) || IS_QLA25XX(ha) || IS_QLA81XX(ha))
+		pdev->needs_freset = 1;
+
 	/* Configure PCI I/O space */
 	ret = qla2x00_iospace_config(ha);
 	if (ret)

once the infrastructure changes were hashed out.

-- av

Hi Andi,

2009/7/31 Andi Kleen [off-list ref]:

Mike Mason [off-list ref] writes:

quoted

These patches supersede the previously submitted patch that
implemented a fundamental reset bit field.

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC
specific hack.

I've found the PCIE chipsepc somewhat daunting, but was under the
impression that much if not most of what was needed was specified
there.

See, for example:
    Documentation/PCI/pcieaer-howto.txt

which states:
|||   The PCI Express Advanced Error Reporting Driver Guide HOWTO
|||                T. Long Nguyen  [off-list ref]
|||                Yanmin Zhang    [off-list ref]
|||                                07/29/2006
[..]
||| The PCI Express AER driver provides the infrastructure to support PCI
||| Express Advanced Error Reporting capability. The PCI Express AER
||| driver provides three basic functions:
|||
||| -       Gathers the comprehensive error information if errors occurred.
||| -       Reports error to the users.
||| -       Performs error recovery actions.

I presume the last bullet point  means that the AER code works and
actually does more or less the same thing as the PPC EEH code,
but in a more architecture-independent way, as it only assumes
that PCI AER is there (and is correctly implemented in the CPI chipset)
The AER code uses the same core infrastructure as the EEH code,
at the time, I did exchange emails w/ the above authors discussing
this stuff.

As to whether the x86 server vendors are actually selling something
with AER in it, and whether any of them are actually testing this stuff
is unclear.

FWIW IBM has pretty much no incentive to lobby other server vendors
to get on the ball ...as this is viewed as one of those things that lets
IBM charge premium prices for PPC hardware.

--linas

2009/7/30 Mike Mason [off-list ref]:

This is the first of three patches that implement a bit field that PCI
Express device drivers can use to indicate they need a fundamental reset
during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot rese=

t"

during recovery of a PCI Express device. =C2=A0We've found a case where t=

he

device needs a "fundamental reset" to recover properly. =C2=A0The current=

 PCI

error recovery and EEH frameworks do not support this distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to pci_dev
that indicates whether the device requires a fundamental reset during
recovery.

These patches supersede the previously submitted patch that implemented a
fundamental reset bit field.
Please review and let me know of any concerns.

Signed-off-by: Mike Mason <redacted>
Signed-off-by: Richard Lary <redacted>

Signed-off-by: Linas Vepstas <linasvepstas@gmail.com>

On Thu, 30 Jul 2009 15:33:21 -0700
Mike Mason [off-list ref] wrote:

This is the first of three patches that implement a bit field that
PCI Express device drivers can use to indicate they need a
fundamental reset during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot
reset" during recovery of a PCI Express device.  We've found a case
where the device needs a "fundamental reset" to recover properly.
The current PCI error recovery and EEH frameworks do not support this
distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to
pci_dev that indicates whether the device requires a fundamental
reset during recovery.

These patches supersede the previously submitted patch that
implemented a fundamental reset bit field. 

Please review and let me know of any concerns.

Signed-off-by: Mike Mason <redacted>
Signed-off-by: Richard Lary <redacted>

Ok, applied this series to my linux-next branch, it looks pretty
reasonable to me.

For future patches, please cc me, and include the subsystem in the
subject, along with a specific description of the patch, e.g. "PCI: add
PCIe fundamental reset interface", "PCI: document PCIe fundamental
reset", or for arch specific patches, "PCI/powerpc: implement support
for PCIe fundamental reset".

Thanks,
-- 
Jesse Barnes, Intel Open Source Technology Center