Re: [PATCH] PCI: Add no-D3 quirk for Mellanox ConnectX-[45]

On 09/01/2019 18:25, Benjamin Herrenschmidt wrote:

On Wed, 2019-01-09 at 17:32 +1100, Alexey Kardashevskiy wrote:

quoted

I have just moved the "Mellanox Technologies MT27700 Family
[ConnectX-4]" from garrison to firestone machine and there it does not
produce an EEH, with the same kernel and skiboot (both upstream + my
debug). Hm. I cannot really blame the card but I cannot see what could
cause the difference in skiboot either. I even tried disabling NPU so
garrison would look like firestone, still EEH'ing.

The systems have a different chip though, firestone is P8 and garrison
is P8', which a slightly different PHB revision. Worth checking if we
have anything significantly different in our inits and poke at the HW
guys.

Nope, we do not have anything different for these machines. Asking HW
guys never worked for me :-/

I think the easiest is just doing what we did for PHB4 and ignoring
these D3 requests on garrisons.

BTW. Are the cards behind a switch in either case ?

No, directly connected to the root on both:

garrison:

0000:00:00.0 PCI bridge: IBM Device 03dc (rev ff)
0000:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4] (rev ff)
0000:01:00.1 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4] (rev ff)

firestone (phb #0 is taken by nvidia gpu):

0001:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3)
0001:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
0001:01:00.1 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]


-- 
Alexey