From: Ian Munsie <hidden> Date: 2016-07-13 21:17:56
This series adds support for the Mellanox CX4 network adapter operating in cxl
mode to the cxl driver and the PowerNV PHB code. The Mellanox developers will
submit a separate patch series that makes use of this in the mlx5 driver.
The CX4 card can operate in either pci mode, or cxl mode. In cxl mode, memory
accesses from the card go through the XSL (Translation Service Layer,
essentially a stripped down version of the Power Service Layer), allowing it to
transparently access unpinned memory with the cxl driver handling faulting in
pages as necessary, etc. Most of the support for the XSL is already upstream,
though this series does include a bug fix to enable bus mastering for this
(patch 3).
Patch 2 in this series provides an API which the mlx5 driver can query to check
if it is in a cxl capable slot. The card will come up in pci mode, and the mlx5
driver can choose to switch it to cxl mode, wherein it will reappear with an
additional physical function representing the XSL that the cxl driver will bind
to. Patches 13-15 add support for switching the card's mode, including using
the PCI hotplug support to re-enumerate the device tree and re-probind the
card.
Unlike previous users of the cxl kernel API where we used a virtual PHB and
exposed PCI devices under it, the Mellanox CX4 uses a peer model where cxl
binds to one of the physical functions of the card and the mlx5_core driver
binds to the other networking physical functions. Patch 6 skips creating a vPHB
for AFUs without any AFU configuration records (including devices using the
peer model) and opts out of EEH handling. Patches 7 and 8 add support for using
the cxl kernel API with the real PHB to enable this peer model. Patches 4 and
5 are prepatory patches exposing some APIs that the PHB will need to call.
While in cxl mode, interrupts from the CX4 are a little unusual - they are
neither pci interrupts, nor cxl interrutps, but rather a hybrid of the two. The
interrupts are passed from the networking hardware to the XSL using a custom
format in the MSIX table, and from there are treated as cxl interrupts. These
are configured mostly transparently using the standard msix APIs - the PHB
handles allocating and configuring the cxl interrupts, associating them with
the default context, and the mlx5 driver handles filling out the MSIX table
with their custom format (not included in this series). See patch 11.
Additionally, the CX4 has a hard limitation of the number of interrupts that
can be associated with a given context, so to overcome this patches 9 and 10
expose an API to allow the mlx5 driver to inform us of the limit, and the
interrupt allocation code in patch 11 will allocate additional contexts to
associate these with.
Patch 1 is a prepatory cleanup patch to reorganise cxl code in arch/powerpc
into a separate file.
Patch 12 is a workaround for a hardware limitation in the CX4 where a context
with PE=0 cannot be used.
The entire series is bisectable.
Changes since v2:
Addressed feedback from Andrew Donnellan:
- Fixed typos in several comments
- Moved _cxl_pci_associate_default_context and
_cxl_pci_disable_device from vphb.c to a new file phb.c since
they are used by both the vPHB and peer models. (Patch 5)
- Changed two exported symbols to EXPORT_SYMBOL_GPL (Patch 7)
- Undid change to remove static from pnv_pci_release_device and
pci_controller_ops and declare them in the header, both of
which were left over from an earlier cut. (Patch 7)
Changes since v1:
- New patch 6 to skip creating a vPHB if there are no AFU configuration
records, and opt out of EEH handling (partially split from patch 8).
- Updated comments in various patches (1, 2, 7, 10, 15) with feedback
from Andrew Donnellan and Frederic Barrat
- Handle error case if cxl_next_msi_hwirq returns 0 signifying
that an AFU IRQ is not mapped to a hardware interrupt (Patch 11)
- Dropped extraneous "select HOTPLUG_PCI_POWERNV_BASE" in Kconfig,
which was accidentally left in from an earlier non-public
revision. Thanks to Gavin Shan for pointing it out (Patch 13)
- Added new error label for error paths calling pci_dev_put() -
suggested by Ian Munsie (Patch 15)
- Added newline at end of Kconfig (Patch 15)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:01
From: Ian Munsie <redacted>
The support for using the Mellanox CX4 in cxl mode will require
additions to the PHB code. In preparation for this, move the existing
cxl code out of pci-ioda.c into a separate pci-cxl.c file to keep things
more organised.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
---
V1 -> V2:
Changed copyright message in new file to 2014-2016, since most
of the code originated in other files written in 2014, and will
be adding new code shortly.
---
arch/powerpc/platforms/powernv/Makefile | 1 +
arch/powerpc/platforms/powernv/pci-cxl.c | 163 ++++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/pci-ioda.c | 159 +----------------------------
arch/powerpc/platforms/powernv/pci.h | 6 ++
4 files changed, 173 insertions(+), 156 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/pci-cxl.c
@@ -0,0 +1,163 @@+/*+*Copyright2014-2016IBMCorp.+*+*Thisprogramisfreesoftware;youcanredistributeitand/or+*modifyitunderthetermsoftheGNUGeneralPublicLicense+*aspublishedbytheFreeSoftwareFoundation;eitherversion+*2oftheLicense,or(atyouroption)anylaterversion.+*/++#include<asm/pnv-pci.h>+#include<asm/opal.h>++#include"pci.h"++structdevice_node*pnv_pci_get_phb_node(structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);++returnof_node_get(hose->dn);+}+EXPORT_SYMBOL(pnv_pci_get_phb_node);++intpnv_phb_to_cxl_mode(structpci_dev*dev,uint64_tmode)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+structpnv_ioda_pe*pe;+intrc;++pe=pnv_ioda_get_pe(dev);+if(!pe)+return-ENODEV;++pe_info(pe,"Switching PHB to CXL\n");++rc=opal_pci_set_phb_cxl_mode(phb->opal_id,mode,pe->pe_number);+if(rc==OPAL_UNSUPPORTED)+dev_err(&dev->dev,"Required cxl mode not supported by firmware - update skiboot\n");+elseif(rc)+dev_err(&dev->dev,"opal_pci_set_phb_cxl_mode failed: %i\n",rc);++returnrc;+}+EXPORT_SYMBOL(pnv_phb_to_cxl_mode);++/* Find PHB for cxl dev and allocate MSI hwirqs?+*ReturnstheabsolutehardwareIRQnumber+*/+intpnv_cxl_alloc_hwirqs(structpci_dev*dev,intnum)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+inthwirq=msi_bitmap_alloc_hwirqs(&phb->msi_bmp,num);++if(hwirq<0){+dev_warn(&dev->dev,"Failed to find a free MSI\n");+return-ENOSPC;+}++returnphb->msi_base+hwirq;+}+EXPORT_SYMBOL(pnv_cxl_alloc_hwirqs);++voidpnv_cxl_release_hwirqs(structpci_dev*dev,inthwirq,intnum)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;++msi_bitmap_free_hwirqs(&phb->msi_bmp,hwirq-phb->msi_base,num);+}+EXPORT_SYMBOL(pnv_cxl_release_hwirqs);++voidpnv_cxl_release_hwirq_ranges(structcxl_irq_ranges*irqs,+structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+inti,hwirq;++for(i=1;i<CXL_IRQ_RANGES;i++){+if(!irqs->range[i])+continue;+pr_devel("cxl release irq range 0x%x: offset: 0x%lx limit: %ld\n",+i,irqs->offset[i],+irqs->range[i]);+hwirq=irqs->offset[i]-phb->msi_base;+msi_bitmap_free_hwirqs(&phb->msi_bmp,hwirq,+irqs->range[i]);+}+}+EXPORT_SYMBOL(pnv_cxl_release_hwirq_ranges);++intpnv_cxl_alloc_hwirq_ranges(structcxl_irq_ranges*irqs,+structpci_dev*dev,intnum)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+inti,hwirq,try;++memset(irqs,0,sizeof(structcxl_irq_ranges));++/* 0 is reserved for the multiplexed PSL DSI interrupt */+for(i=1;i<CXL_IRQ_RANGES&#i++){+try=num;+while(try){+hwirq=msi_bitmap_alloc_hwirqs(&phb->msi_bmp,try);+if(hwirq>=0)+break;+try/=2;+}+if(!try)+gotofail;++irqs->offset[i]=phb->msi_base+hwirq;+irqs->range[i]=try;+pr_devel("cxl alloc irq range 0x%x: offset: 0x%lx limit: %li\n",+i,irqs->offset[i],irqs->range[i]);+num-=try;+}+if(num)+gotofail;++return0;+fail:+pnv_cxl_release_hwirq_ranges(irqs,dev);+return-ENOSPC;+}+EXPORT_SYMBOL(pnv_cxl_alloc_hwirq_ranges);++intpnv_cxl_get_irq_count(structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;++returnphb->msi_bmp.irq_count;+}+EXPORT_SYMBOL(pnv_cxl_get_irq_count);++intpnv_cxl_ioda_msi_setup(structpci_dev*dev,unsignedinthwirq,+unsignedintvirq)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+unsignedintxive_num=hwirq-phb->msi_base;+structpnv_ioda_pe*pe;+intrc;++if(!(pe=pnv_ioda_get_pe(dev)))+return-ENODEV;++/* Assign XIVE to PE */+rc=opal_pci_set_xive_pe(phb->opal_id,pe->pe_number,xive_num);+if(rc){+pe_warn(pe,"%s: OPAL error %d setting msi_base 0x%x "+"hwirq 0x%x XIVE 0x%x PE\n",+pci_name(dev),rc,phb->msi_base,hwirq,xive_num);+return-EIO;+}+pnv_set_msi_irq_chip(phb,virq);++return0;+}+EXPORT_SYMBOL(pnv_cxl_ioda_msi_setup);
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:01
From: Ian Munsie <redacted>
This extends the check that the adapter is in a CAPI capable slot so
that it may be called by external users in the kernel API. This will be
used by the upcoming Mellanox CX4 support, which needs to know ahead of
time if the card can be switched to cxl mode so that it can leave it in
PCI mode if it is not.
This API takes a parameter to check if CAPP DMA mode is supported, which
it currently only allows on P8NVL systems, since that mode currently has
issues accessing memory < 4GB on P8, and we cannot realistically avoid
that.
This API does not currently check if a CAPP unit is available (i.e. not
already assigned to another PHB) on P8. Doing so would be racy since it
is assigned on a first come first serve basis, and so long as CAPP DMA
mode is not supported on P8 we don't need this, since the only
anticipated user of this API requires CAPP DMA mode.
Cc: Philippe Bergheaud <redacted>
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
---
V1->V2:
- Fixed typos in comments spotted by Andrew
---
drivers/misc/cxl/pci.c | 37 +++++++++++++++++++++++++++++++++++++
include/misc/cxl.h | 15 +++++++++++++++
2 files changed, 52 insertions(+)
@@ -24,6 +24,21 @@*genericPCIAPI.ThisAPIisagnostictotheactualAFU.*/+#define CXL_SLOT_FLAG_DMA 0x1++/*+*Checksifthegivencardisinacxlcapableslot.PassCXL_SLOT_FLAG_DMAif+*thecardrequiresCAPPDMAmodetoalsocheckifthesystemsupportsit.+*Thisisintendedtobeusedbybi-modaldevicestodetermineiftheycanuse+*cxlmodeoriftheyshouldcontinuerunninginPCImode.+*+*Notethatthisonlychecksiftheslotiscxlcapable-itdoesnot+*currentlycheckiftheCAPPiscurrentlyavailableforchipswhereitcanbe+*assignedtodifferentPHBsonafirstcomefirstservebasis(i.e.P8)+*/+boolcxl_slot_is_supported(structpci_dev*dev,intflags);++/* Get the AFU associated with a pci_dev */structcxl_afu*cxl_pci_to_afu(structpci_dev*dev);
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:03
From: Ian Munsie <redacted>
The Mellanox CX4 uses a model where the AFU is one physical function of
the device, and is used by other peer physical functions of the same
device. This will require those other devices to grab a reference on the
AFU when they are initialised to make sure that it does not go away
during their lifetime.
Move the AFU refcount functions to base.c so they can be called from
the PHB code.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
---
drivers/misc/cxl/base.c | 13 +++++++++++++
drivers/misc/cxl/cxl.h | 12 ------------
include/misc/cxl-base.h | 4 ++++
3 files changed, 17 insertions(+), 12 deletions(-)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:03
From: Ian Munsie <redacted>
Devices that use CAPP DMA mode (such as the Mellanox CX4) require bus
master to be enabled in order for the CAPI traffic to flow. This should
be harmless to enable for other cxl devices, so unconditionally enable
it in the adapter init flow.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
---
drivers/misc/cxl/pci.c | 3 +++
1 file changed, 3 insertions(+)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:06
From: Ian Munsie <redacted>
The cxl kernel API has a concept of a default context associated with
each PCI device under the virtual PHB. The Mellanox CX4 will also use
the cxl kernel API, but it does not use a virtual PHB - rather, the AFU
appears as a physical function as a peer to the networking functions.
In order to allow the kernel API to work with those networking
functions, we will need to associate a default context with them as
well. To this end, refactor the corresponding code to do this in vphb.c
and export it so that it can be called from the PHB code.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
V2->V3:
Addressed feedback from Andrew Donnellan:
- Fixed typo in comment
- Moved _cxl_pci_associate_default_context and
_cxl_pci_disable_device from vphb.c to a new file phb.c since
they are used by both the vPHB and peer models.
---
drivers/misc/cxl/Makefile | 2 +-
drivers/misc/cxl/base.c | 35 +++++++++++++++++++++++++++++++++++
drivers/misc/cxl/cxl.h | 6 ++++++
drivers/misc/cxl/main.c | 2 ++
drivers/misc/cxl/phb.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
drivers/misc/cxl/vphb.c | 30 +++---------------------------
include/misc/cxl-base.h | 6 ++++++
7 files changed, 97 insertions(+), 28 deletions(-)
create mode 100644 drivers/misc/cxl/phb.c
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:08
From: Ian Munsie <redacted>
The vPHB model of the cxl kernel API is a hierarchy where the AFU is
represented by the vPHB, and it's AFU configuration records are exposed
as functions under that vPHB. If there are no AFU configuration records
we will create a vPHB with nothing under it, which is a waste of
resources and will opt us into EEH handling despite not having anything
special to handle.
This also does not make sense for cards using the peer model of the cxl
kernel API, where the other functions of the device are exposed via
additional peer physical functions rather than AFU configuration
records. This model will also not work with the existing EEH handling in
the cxl driver, as that is designed around the vPHB model.
Skip creating the vPHB for AFUs without any AFU configuration records,
and opt out of EEH handling for them.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
drivers/misc/cxl/pci.c | 3 +++
drivers/misc/cxl/vphb.c | 11 +++++++++++
2 files changed, 14 insertions(+)
@@ -1572,6 +1572,9 @@ static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev,*/for(i=0;i<adapter->slices;i++){afu=adapter->afu[i];+/* Only participate in EEH if we are on a virtual PHB */+if(afu->phb==NULL)+returnPCI_ERS_RESULT_NONE;cxl_vphb_error_detected(afu,state);}returnPCI_ERS_RESULT_DISCONNECT;
@@ -188,6 +188,17 @@ int cxl_pci_vphb_add(struct cxl_afu *afu)structdevice_node*vphb_dn;structdevice*parent;+/*+*IftherearenoAFUconfigurationrecordswewon'thaveanythingto+*exposeunderthevPHB,soskipcreatingone,returningsuccesssince+*thisisstillavalidcase.ThiswillalsooptusoutofEEH+*handlingsincewewon'thaveanythingspecialtodoifthereareno+*kerneldriversattachedtothevPHB,andEEHhandlingisnotyet+*supportedinthepeermodel.+*/+if(!afu->crs_num)+return0;+/* The parent device is the adapter. Reuse the device node of*theadapter.*Wedon'tseemtocarewhatdevicenodeisusedforthevPHB,
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:08
From: Ian Munsie <redacted>
This adds support for the peer model of the cxl kernel api to the
PowerNV PHB, in which physical function 0 represents the cxl function on
the card (an XSL in the case of the CX4), which other physical functions
will use for memory access and interrupt services. It is referred to as
the peer model as these functions are peers of one another, as opposed
to the Virtual PHB model which forms a hierarchy.
This patch exports APIs to enable the peer mode, check if a PCI device
is attached to a PHB in this mode, and to set and get the peer AFU for
this mode.
The cxl driver will enable this mode for supported cards by calling
pnv_cxl_enable_phb_kernel_api(). This will set a flag in the PHB to note
that this mode is enabled, and switch out it's controller_ops for the
cxl version.
The cxl version of the controller_ops struct implements it's own
versions of the enable_device_hook and release_device to handle
refcounting on the peer AFU and to allocate a default context for the
device.
Once enabled, the cxl kernel API may not be disabled on a PHB. Currently
there is no safe way to disable cxl mode short of a reboot, so until
that changes there is no reason to support the disable path.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
V1->V2:
- Add an explanation of the peer model to the commit message,
and a comment above the pnv_cxl_enable_device_hook function.
V2->V3
Addressed comments from Andrew Donnellan:
- Fix typo in comment
- Changed two exported symbols to EXPORT_SYMBOL_GPL
- Undid change to remove static from pnv_pci_release_device and
pci_controller_ops and declare them in the header, both of
which were left over from an earlier cut.
---
arch/powerpc/include/asm/pnv-pci.h | 7 ++
arch/powerpc/platforms/powernv/pci-cxl.c | 120 ++++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/pci-ioda.c | 18 ++++-
arch/powerpc/platforms/powernv/pci.h | 14 ++++
4 files changed, 158 insertions(+), 1 deletion(-)
@@ -38,6 +38,13 @@ int pnv_cxl_alloc_hwirq_ranges(struct cxl_irq_ranges *irqs,structpci_dev*dev,intnum);voidpnv_cxl_release_hwirq_ranges(structcxl_irq_ranges*irqs,structpci_dev*dev);++/* Support for the cxl kernel api on the real PHB (instead of vPHB) */+intpnv_cxl_enable_phb_kernel_api(structpci_controller*hose,boolenable);+boolpnv_pci_on_cxl_phb(structpci_dev*dev);+structcxl_afu*pnv_cxl_phb_to_afu(structpci_controller*hose);+voidpnv_cxl_phb_set_peer_afu(structpci_dev*dev,structcxl_afu*afu);+#endif#endif
@@ -161,3 +164,120 @@ int pnv_cxl_ioda_msi_setup(struct pci_dev *dev, unsigned int hwirq,return0;}EXPORT_SYMBOL(pnv_cxl_ioda_msi_setup);++/*+*Setsflagsandswitchesthecontrolleropstoenablethecxlkernelapi.+*OriginallythecxlkernelAPIoperatedonavirtualPHB,butcertaincards+*suchastheMellanoxCX4useapeermodelinsteadandforthesecardsthe+*cxlkernelapiwilloperateontherealPHB.+*/+intpnv_cxl_enable_phb_kernel_api(structpci_controller*hose,boolenable)+{+structpnv_phb*phb=hose->private_data;+structmodule*cxl_module;++if(!enable){+/*+*OncecxlmodeisenabledonthePHB,thereiscurrentlyno+*knownsafemethodtodisableitagain,andtryingrisksa+*checkstop.Ifwecanfindawaytosafelydisablecxlmode+*inthefuturewecanrevisitthis,butfornowtheonlysane+*thingtodoistorefusetodisablecxlmode:+*/+return-EPERM;+}++/*+*HoldareferencetothecxlmodulesinceseveralPHBoperationsnow+*dependonit,anditwouldbeinsanetoallowittoberemovedso+*longasweareinthismode(andsincewecan'tsafelydisablethis+*modeonceenabled...).+*/+mutex_lock(&module_mutex);+cxl_module=find_module("cxl");+if(cxl_module)+__module_get(cxl_module);+mutex_unlock(&module_mutex);+if(!cxl_module)+return-ENODEV;++phb->flags|=PNV_PHB_FLAG_CXL;+hose->controller_ops=pnv_cxl_cx4_ioda_controller_ops;++return0;+}+EXPORT_SYMBOL_GPL(pnv_cxl_enable_phb_kernel_api);++boolpnv_pci_on_cxl_phb(structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;++return!!(phb->flags&PNV_PHB_FLAG_CXL);+}+EXPORT_SYMBOL_GPL(pnv_pci_on_cxl_phb);++structcxl_afu*pnv_cxl_phb_to_afu(structpci_controller*hose)+{+structpnv_phb*phb=hose->private_data;++return(structcxl_afu*)phb->cxl_afu;+}+EXPORT_SYMBOL_GPL(pnv_cxl_phb_to_afu);++voidpnv_cxl_phb_set_peer_afu(structpci_dev*dev,structcxl_afu*afu)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;++phb->cxl_afu=afu;+}+EXPORT_SYMBOL_GPL(pnv_cxl_phb_set_peer_afu);++/*+*Inthepeercxlmodel,theXSL/PSLisphysicalfunction0,andwillbeused+*byotherfunctionsonthedeviceformemoryaccessandinterrupts.Whenthe+*otherfunctionsareenabledweexplicitlytakeareferenceonthecxl+*functionsincetheywilluseit,andallocateadefaultcontextassociated+*withthatfunctionjustlikethevPHBmodelofthecxlkernelAPI.+*/+boolpnv_cxl_enable_device_hook(structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+structcxl_afu*afu=phb->cxl_afu;++if(!pnv_pci_enable_device_hook(dev))+returnfalse;+++/* No special handling for the cxl function, which is always PF 0 */+if(PCI_FUNC(dev->devfn)==0)+returntrue;++if(!afu){+dev_WARN(&dev->dev,"Attempted to enable function > 0 on CXL PHB without a peer AFU\n");+returnfalse;+}++dev_info(&dev->dev,"Enabling function on CXL enabled PHB with peer AFU\n");++/* Make sure the peer AFU can't go away while this device is active */+cxl_afu_get(afu);++returncxl_pci_associate_default_context(dev,afu);+}++voidpnv_cxl_disable_device(structpci_dev*dev)+{+structpci_controller*hose=pci_bus_to_host(dev->bus);+structpnv_phb*phb=hose->private_data;+structcxl_afu*afu=phb->cxl_afu;++/* No special handling for cxl function: */+if(PCI_FUNC(dev->devfn)==0)+return;++cxl_pci_disable_device(dev);+cxl_afu_put(afu);+}
@@ -218,6 +222,7 @@ extern int pnv_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type);externvoidpnv_teardown_msi_irqs(structpci_dev*pdev);externstructpnv_ioda_pe*pnv_ioda_get_pe(structpci_dev*dev);externvoidpnv_set_msi_irq_chip(structpnv_phb*phb,unsignedintvirq);+externboolpnv_pci_enable_device_hook(structpci_dev*dev);externvoidpe_level_printk(conststructpnv_ioda_pe*pe,constchar*level,constchar*fmt,...);
@@ -238,4 +243,13 @@ extern long pnv_npu_unset_window(struct pnv_ioda_pe *npe, int num);externvoidpnv_npu_take_ownership(structpnv_ioda_pe*npe);externvoidpnv_npu_release_ownership(structpnv_ioda_pe*npe);++/* cxl functions */+externboolpnv_cxl_enable_device_hook(structpci_dev*dev);+externvoidpnv_cxl_disable_device(structpci_dev*dev);+++/* phb ops (cxl switches these when enabling the kernel api on the phb) */+externconststructpci_controller_opspnv_cxl_cx4_ioda_controller_ops;+#endif /* __POWERNV_PCI_H */
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:10
From: Ian Munsie <redacted>
This hooks up support for using the kernel API with a real PHB. After
the AFU initialisation has completed it calls into the PHB code to pass
it the AFU that will be used by other peer physical functions on the
adapter.
The cxl_pci_to_afu API is extended to work with peer PCI devices,
retrieving the peer AFU from the PHB. This API may also now return an
error if it is called on a PCI device that is not associated with either
a cxl vPHB or a peer PCI device to an AFU, and this error is propagated
down.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
V1->V2:
- Removed change to skip participating in EEH without a vPHB -
moved out into an earlier patch.
---
drivers/misc/cxl/api.c | 5 +++++
drivers/misc/cxl/pci.c | 3 +++
drivers/misc/cxl/vphb.c | 16 ++++++++++++++--
3 files changed, 22 insertions(+), 2 deletions(-)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:11
From: Ian Munsie <redacted>
These APIs will be used by the Mellanox CX4 support. While they function
standalone to configure existing behaviour, their primary purpose is to
allow the Mellanox driver to inform the cxl driver of a hardware
limitation, which will be used in a future patch.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
drivers/misc/cxl/api.c | 27 +++++++++++++++++++++++++++
include/misc/cxl.h | 10 ++++++++++
2 files changed, 37 insertions(+)
@@ -447,3 +447,30 @@ ssize_t cxl_read_adapter_vpd(struct pci_dev *dev, void *buf, size_t count)returncxl_ops->read_adapter_vpd(afu->adapter,buf,count);}EXPORT_SYMBOL_GPL(cxl_read_adapter_vpd);++intcxl_set_max_irqs_per_process(structpci_dev*dev,intirqs)+{+structcxl_afu*afu=cxl_pci_to_afu(dev);+if(IS_ERR(afu))+return-ENODEV;++if(irqs>afu->adapter->user_irqs)+return-EINVAL;++/* Limit user_irqs to prevent the user increasing this via sysfs */+afu->adapter->user_irqs=irqs;+afu->irqs_max=irqs;++return0;+}+EXPORT_SYMBOL_GPL(cxl_set_max_irqs_per_process);++intcxl_get_max_irqs_per_process(structpci_dev*dev)+{+structcxl_afu*afu=cxl_pci_to_afu(dev);+if(IS_ERR(afu))+return-ENODEV;++returnafu->irqs_max;+}+EXPORT_SYMBOL_GPL(cxl_get_max_irqs_per_process);
@@ -166,6 +166,16 @@ void cxl_psa_unmap(void __iomem *addr);/* Get the process element for this context */intcxl_process_element(structcxl_context*ctx);+/*+*Limitthenumberofinterruptsthatasinglecontextcanallocatevia+*cxl_start_work.Ifusingtheapiwitharealphb,thismaybeusedto+*requestthatadditionaldefaultcontextsbecreatedwhenallocating+*interruptsviapci_enable_msix_range.Thesewillbesettothesamerunning+*stateasthedefaultcontext,andifthatisrunningitwillreusethe+*parameterspreviouslypassedtocxl_start_contextforthedefaultcontext.+*/+intcxl_set_max_irqs_per_process(structpci_dev*dev,intirqs);+intcxl_get_max_irqs_per_process(structpci_dev*dev);/**Thesecallsallowdriverstocreatetheirownfiledescriptorsandmakethem
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:13
From: Ian Munsie <redacted>
The Mellanox CX4 has a hardware limitation where only 4 bits of the
AFU interrupt number can be passed to the XSL when sending an interrupt,
limiting it to only 15 interrupts per context (AFU interrupt number 0 is
invalid).
In order to overcome this, we will allocate additional contexts linked
to the default context as extra address space for the extra interrupts -
this will be implemented in the next patch.
This patch adds the preliminary support to allow this, by way of adding
a linked list in the context structure that we use to keep track of the
contexts dedicated to interrupts, and an API to simultaneously iterate
over the related context structures, AFU interrupt numbers and hardware
interrupt numbers. The point of using a single API to iterate these is
to hide some of the details of the iteration from external code, and to
reduce the number of APIs that need to be exported via base.c to allow
built in code to call.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
---
V1->V2:
- Fixed typo spotted by Fred
---
drivers/misc/cxl/api.c | 15 +++++++++++++++
drivers/misc/cxl/base.c | 17 +++++++++++++++++
drivers/misc/cxl/context.c | 1 +
drivers/misc/cxl/cxl.h | 10 ++++++++++
drivers/misc/cxl/main.c | 1 +
include/misc/cxl.h | 9 +++++++++
6 files changed, 53 insertions(+)
@@ -178,6 +178,15 @@ int cxl_set_max_irqs_per_process(struct pci_dev *dev, int irqs);intcxl_get_max_irqs_per_process(structpci_dev*dev);/*+*Usetosimultaneouslyiterateoverhardwareinterruptnumbers,contextsand+*afuinterruptnumbersallocatedforthedeviceviapci_enable_msix_rangeand+*isausefulconveniencefunctionwhenworkingwithhardwarethathas+*limitationsonthenumberofinterruptsperprocess.*ctxand*afu_irq+*shouldbeNULLand0tostarttheiteration.+*/+intcxl_next_msi_hwirq(structpci_dev*pdev,structcxl_context**ctx,int*afu_irq);++/**Thesecallsallowdriverstocreatetheirownfiledescriptorsandmakethem*identicaltothecxlfiledescriptoruserAPI.Anexampleusecase:*
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:15
From: Ian Munsie <redacted>
The CX4 card cannot cope with a context with PE=0 due to a hardware
limitation, resulting in:
[ 34.166577] command failed, status limits exceeded(0x8), syndrome 0x5a7939
[ 34.166580] mlx5_core 0000:01:00.1: Failed allocating uar, aborting
Since the kernel API allocates a default context very early during
device init that will almost certainly get Process Element ID 0 there is
no easy way for us to extend the API to allow the Mellanox to inform us
of this limitation ahead of time.
Instead, work around the issue by extending the XSL structure to include
a minimum PE to allocate. Although the bug is not in the XSL, it is the
easiest place to work around this limitation given that the CX4 is
currently the only card that uses an XSL.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
---
drivers/misc/cxl/context.c | 3 ++-
drivers/misc/cxl/cxl.h | 1 +
drivers/misc/cxl/pci.c | 1 +
3 files changed, 4 insertions(+), 1 deletion(-)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:16
From: Ian Munsie <redacted>
The Mellanox CX4 in cxl mode uses a hybrid interrupt model, where
interrupts are routed from the networking hardware to the XSL using the
MSIX table, and from there will be transformed back into an MSIX
interrupt using the cxl style interrupts (i.e. using IVTE entries and
ranges to map a PE and AFU interrupt number to an MSIX address).
We want to hide the implementation details of cxl interrupts as much as
possible. To this end, we use a special version of the MSI setup &
teardown routines in the PHB while in cxl mode to allocate the cxl
interrupts and configure the IVTE entries in the process element.
This function does not configure the MSIX table - the CX4 card uses a
custom format in that table and it would not be appropriate to fill that
out in generic code. The rest of the functionality is similar to the
"Full MSI-X mode" described in the CAIA, and this could be easily
extended to support other adapters that use that mode in the future.
The interrupts will be associated with the default context. If the
maximum number of interrupts per context has been limited (e.g. by the
mlx5 driver), it will automatically allocate additional kernel contexts
to associate extra interrupts as required. These contexts will be
started using the same WED that was used to start the default context.
Signed-off-by: Ian Munsie <redacted>
---
V1->V2:
- Handle error case if cxl_next_msi_hwirq returns 0 signifying
that an AFU IRQ is not mapped to a hardware interrupt.
---
arch/powerpc/platforms/powernv/pci-cxl.c | 84 +++++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/pci-ioda.c | 4 ++
arch/powerpc/platforms/powernv/pci.h | 2 +
drivers/misc/cxl/api.c | 71 ++++++++++++++++++++++++++
drivers/misc/cxl/base.c | 31 ++++++++++++
drivers/misc/cxl/cxl.h | 4 ++
drivers/misc/cxl/main.c | 2 +
include/misc/cxl-base.h | 4 ++
8 files changed, 202 insertions(+)
@@ -247,6 +247,8 @@ extern void pnv_npu_release_ownership(struct pnv_ioda_pe *npe);/* cxl functions */externboolpnv_cxl_enable_device_hook(structpci_dev*dev);externvoidpnv_cxl_disable_device(structpci_dev*dev);+externintpnv_cxl_cx4_setup_msi_irqs(structpci_dev*pdev,intnvec,inttype);+externvoidpnv_cxl_cx4_teardown_msi_irqs(structpci_dev*pdev);/* phb ops (cxl switches these when enabling the kernel api on the phb) */
@@ -489,3 +490,73 @@ int cxl_get_max_irqs_per_process(struct pci_dev *dev)returnafu->irqs_max;}EXPORT_SYMBOL_GPL(cxl_get_max_irqs_per_process);++/*+*ThisisaspecialinterruptallocationroutinecalledfromthePHB'sMSI+*setupfunction.Whencapiinterruptsareallocatedinthismannertheymust+*stillbeassociatedwitharunningcontext,butsincetheMSIAPIshaveno+*waytospecifythisweusethedefaultcontextassociatedwiththedevice.+*+*TheMellanoxCX4hasahardwarelimitationthatrestrictsthemaximumAFU+*interruptnumber,soinordertoovercomethistheirdriverinformsusof+*therestrictionbysettingthemaximuminterruptspercontext,andwe+*allocateadditionalcontextsasnecessarysothatwecankeeptheAFU+*interruptnumberwithinthesupportedrange.+*/+int_cxl_cx4_setup_msi_irqs(structpci_dev*pdev,intnvec,inttype)+{+structcxl_context*ctx,*new_ctx,*default_ctx;+intremaining;+intrc;++ctx=default_ctx=cxl_get_context(pdev);+if(WARN_ON(!default_ctx))+return-ENODEV;++remaining=nvec;+while(remaining>0){+rc=cxl_allocate_afu_irqs(ctx,min(remaining,ctx->afu->irqs_max));+if(rc){+pr_warn("%s: Failed to find enough free MSIs\n",pci_name(pdev));+returnrc;+}+remaining-=ctx->afu->irqs_max;++if(ctx!=default_ctx&&default_ctx->status==STARTED){+WARN_ON(cxl_start_context(ctx,+be64_to_cpu(default_ctx->elem->common.wed),+NULL));+}++if(remaining>0){+new_ctx=cxl_dev_context_init(pdev);+if(!new_ctx){+pr_warn("%s: Failed to allocate enough contexts for MSIs\n",pci_name(pdev));+return-ENOSPC;+}+list_add(&new_ctx->extra_irq_contexts,&ctx->extra_irq_contexts);+ctx=new_ctx;+}+}++return0;+}+/* Exported via cxl_base */++void_cxl_cx4_teardown_msi_irqs(structpci_dev*pdev)+{+structcxl_context*ctx,*pos,*tmp;++ctx=cxl_get_context(pdev);+if(WARN_ON(!ctx))+return;++cxl_free_afu_irqs(ctx);+list_for_each_entry_safe(pos,tmp,&ctx->extra_irq_contexts,extra_irq_contexts){+cxl_stop_context(pos);+cxl_free_afu_irqs(pos);+list_del(&pos->extra_irq_contexts);+cxl_release_context(pos);+}+}+/* Exported via cxl_base */
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:17
From: Andrew Donnellan <redacted>
The cxl driver will use infrastructure from pnv_php to handle device tree
updates when switching bi-modal CAPI cards into CAPI mode.
To enable this, export pnv_php_find_slot() and
pnv_php_set_slot_power_state(), and add corresponding declarations, as well
as the definition of struct pnv_php_slot, to asm/pnv-pci.h.
Cc: Gavin Shan <redacted>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Acked-by: Gavin Shan <redacted>
---
V1->V2:
- Dropped extraneous "select HOTPLUG_PCI_POWERNV_BASE" in Kconfig,
which was accidentally left in from an earlier non-public
revision. Thanks to Gavin Shan for pointing it out
---
arch/powerpc/include/asm/pnv-pci.h | 28 ++++++++++++++++++++++++++++
drivers/pci/hotplug/pnv_php.c | 32 +++++---------------------------
2 files changed, 33 insertions(+), 27 deletions(-)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:19
From: Andrew Donnellan <redacted>
When calling pnv_php_set_slot_power_state() with state ==
OPAL_PCI_SLOT_OFFLINE, remove devices from the device tree as if we're
dealing with OPAL_PCI_SLOT_POWER_OFF.
Cc: Gavin Shan <redacted>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Acked-by: Gavin Shan <redacted>
---
drivers/pci/hotplug/pnv_php.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Ian Munsie <hidden> Date: 2016-07-13 21:18:21
From: Andrew Donnellan <redacted>
Add a new API, cxl_check_and_switch_mode() to allow for switching of
bi-modal CAPI cards, such as the Mellanox CX-4 network card.
When a driver requests to switch a card to CAPI mode, use PCI hotplug
infrastructure to remove all PCI devices underneath the slot. We then write
an updated mode control register to the CAPI VSEC, hot reset the card, and
reprobe the card.
As the card may present a different set of PCI devices after the mode
switch, use the infrastructure provided by the pnv_php driver and the OPAL
PCI slot management facilities to ensure that:
* the old devices are removed from both the OPAL and Linux device trees
* the new devices are probed by OPAL and added to the OPAL device tree
* the new devices are added to the Linux device tree and probed through
the regular PCI device probe path
As such, introduce a new option, CONFIG_CXL_BIMODAL, with a dependency on
the pnv_php driver.
Refactor existing code that touches the mode control register in the
regular single mode case into a new function, setup_cxl_protocol_area().
Co-authored-by: Ian Munsie [off-list ref]
Cc: Gavin Shan <redacted>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Gavin Shan <redacted>
---
V1->V2:
- Added comments around pci_dev_put() - suggested by Frederic
Barrat
- Added new error label for error paths calling pci_dev_put() -
suggested by Ian Munsie
- Added newline at end of Kconfig
- Removed extraneous comment in setup_cxl_protocol_area()
---
drivers/misc/cxl/Kconfig | 8 ++
drivers/misc/cxl/pci.c | 236 +++++++++++++++++++++++++++++++++++++++++++----
include/misc/cxl.h | 25 +++++
3 files changed, 251 insertions(+), 18 deletions(-)
@@ -614,36 +616,234 @@ static int setup_cxl_bars(struct pci_dev *dev)return0;}-/* pciex node: ibm,opal-m64-window = <0x3d058 0x0 0x3d058 0x0 0x8 0x0>; */-staticintswitch_card_to_cxl(structpci_dev*dev)-{+#ifdef CONFIG_CXL_BIMODAL++structcxl_switch_work{+structpci_dev*dev;+structwork_structwork;intvsec;+intmode;+};++staticvoidswitch_card_to_cxl(structwork_struct*work)+{+structcxl_switch_work*switch_work=+container_of(work,structcxl_switch_work,work);+structpci_dev*dev=switch_work->dev;+structpci_bus*bus=dev->bus;+structpci_controller*hose=pci_bus_to_host(bus);+structpci_dev*bridge;+structpnv_php_slot*php_slot;+unsignedintdevfn;u8val;intrc;-dev_info(&dev->dev,"switch card to CXL\n");+dev_info(&bus->dev,"cxl: Preparing for mode switch...\n");+bridge=list_first_entry_or_null(&hose->bus->devices,structpci_dev,+bus_list);+if(!bridge){+dev_WARN(&bus->dev,"cxl: Couldn't find root port!\n");+gotoerr_dev_put;+}-if(!(vsec=find_cxl_vsec(dev))){-dev_err(&dev->dev,"ABORTING: CXL VSEC not found!\n");+php_slot=pnv_php_find_slot(pci_device_to_OF_node(bridge));+if(!php_slot){+dev_err(&bus->dev,"cxl: Failed to find slot hotplug "+"information. You may need to upgrade "+"skiboot. Aborting.\n");+gotoerr_dev_put;+}++rc=CXL_READ_VSEC_MODE_CONTROL(dev,switch_work->vsec,&val);+if(rc){+dev_err(&bus->dev,"cxl: Failed to read CAPI mode control: %i\n",rc);+gotoerr_dev_put;+}+devfn=dev->devfn;++/* Release the reference obtained in cxl_check_and_switch_mode() */+pci_dev_put(dev);++dev_dbg(&bus->dev,"cxl: Removing PCI devices from kernel\n");+pci_lock_rescan_remove();+pci_hp_remove_devices(bridge->subordinate);+pci_unlock_rescan_remove();++/* Switch the CXL protocol on the card */+if(switch_work->mode==CXL_BIMODE_CXL){+dev_info(&bus->dev,"cxl: Switching card to CXL mode\n");+val&=~CXL_VSEC_PROTOCOL_MASK;+val|=CXL_VSEC_PROTOCOL_256TB|CXL_VSEC_PROTOCOL_ENABLE;+rc=pnv_cxl_enable_phb_kernel_api(hose,true);+if(rc){+dev_err(&bus->dev,"cxl: Failed to enable kernel API"+" on real PHB, aborting\n");+gotoerr_free_work;+}+}else{+dev_WARN(&bus->dev,"cxl: Switching card to PCI mode not supported!\n");+gotoerr_free_work;+}++rc=CXL_WRITE_VSEC_MODE_CONTROL_BUS(bus,devfn,switch_work->vsec,val);+if(rc){+dev_err(&bus->dev,"cxl: Failed to configure CXL protocol: %i\n",rc);+gotoerr_free_work;+}++/*+*TheCAIAspec(v1.1,Section10.6Bi-modalDeviceSupport)states+*wemustwait100msafterthismodeswitchbeforetouchingPCIeconfig+*space.+*/+msleep(100);++/*+*Hotresettocausethecardtocomebackincxlmode.A+*OPAL_RESET_PCI_LINKwouldbesufficient,butcurrentlylackssupport+*inskiboot,soweuseahotresetinstead.+*+*Wecallpci_set_pcie_reset_state()onthebridge,asaCAPIcardis+*guaranteedtositdirectlyundertherootport,andsettingthereset+*stateonadevicedirectlyundertherootportisequivalenttodoing+*itontherootportiself.+*/+dev_info(&bus->dev,"cxl: Configuration write complete, resetting card\n");+pci_set_pcie_reset_state(bridge,pcie_hot_reset);+pci_set_pcie_reset_state(bridge,pcie_deassert_reset);++dev_dbg(&bus->dev,"cxl: Offlining slot\n");+rc=pnv_php_set_slot_power_state(&php_slot->slot,OPAL_PCI_SLOT_OFFLINE);+if(rc){+dev_err(&bus->dev,"cxl: OPAL offlining call failed: %i\n",rc);+gotoerr_free_work;+}++dev_dbg(&bus->dev,"cxl: Onlining and probing slot\n");+rc=pnv_php_set_slot_power_state(&php_slot->slot,OPAL_PCI_SLOT_ONLINE);+if(rc){+dev_err(&bus->dev,"cxl: OPAL onlining call failed: %i\n",rc);+gotoerr_free_work;+}++pci_lock_rescan_remove();+pci_hp_add_devices(bridge->subordinate);+pci_unlock_rescan_remove();++dev_info(&bus->dev,"cxl: CAPI mode switch completed\n");+kfree(switch_work);+return;++err_dev_put:+/* Release the reference obtained in cxl_check_and_switch_mode() */+pci_dev_put(dev);+err_free_work:+kfree(switch_work);+}++intcxl_check_and_switch_mode(structpci_dev*dev,intmode,intvsec)+{+structcxl_switch_work*work;+u8val;+intrc;++if(!cpu_has_feature(CPU_FTR_HVMODE))return-ENODEV;++if(!vsec){+vsec=find_cxl_vsec(dev);+if(!vsec){+dev_info(&dev->dev,"CXL VSEC not found\n");+return-ENODEV;+}}-if((rc=CXL_READ_VSEC_MODE_CONTROL(dev,vsec,&val))){-dev_err(&dev->dev,"failed to read current mode control: %i",rc);+rc=CXL_READ_VSEC_MODE_CONTROL(dev,vsec,&val);+if(rc){+dev_err(&dev->dev,"Failed to read current mode control: %i",rc);returnrc;}-val&=~CXL_VSEC_PROTOCOL_MASK;-val|=CXL_VSEC_PROTOCOL_256TB|CXL_VSEC_PROTOCOL_ENABLE;-if((rc=CXL_WRITE_VSEC_MODE_CONTROL(dev,vsec,val))){-dev_err(&dev->dev,"failed to enable CXL protocol: %i",rc);-returnrc;++if(mode==CXL_BIMODE_PCI){+if(!(val&CXL_VSEC_PROTOCOL_ENABLE)){+dev_info(&dev->dev,"Card is already in PCI mode\n");+return0;+}+/*+*TODO:Beforeit'ssafetoswitchthecardbacktoPCImode+*weneedtodisabletheCAPPandmakesureanycachelinesthe+*cardholdshavebeenflushedout.Needsskibootsupport.+*/+dev_WARN(&dev->dev,"CXL mode switch to PCI unsupported!\n");+return-EIO;}++if(val&CXL_VSEC_PROTOCOL_ENABLE){+dev_info(&dev->dev,"Card is already in CXL mode\n");+return0;+}++dev_info(&dev->dev,"Card is in PCI mode, scheduling kernel thread "+"to switch to CXL mode\n");++work=kmalloc(sizeof(structcxl_switch_work),GFP_KERNEL);+if(!work)+return-ENOMEM;++pci_dev_get(dev);+work->dev=dev;+work->vsec=vsec;+work->mode=mode;+INIT_WORK(&work->work,switch_card_to_cxl);++schedule_work(&work->work);+/*-*TheCAIAspec(v0.1211.6Bi-modalDeviceSupport)states-*wemustwait100msafterthismodeswitchbeforetouching-*PCIeconfigspace.+*Wereturnafailurenowtoabortthedriverinit.Oncethe+*linkhasbeencycledandthecardisincxlmodewewill+*comeback(possiblyusingthegenericcxldriver),but+*returnsuccessasthecardshouldthenbeincxlmode.+*+*TODO:WhatifthecardcomesbackinPCImodeevenafter+*theswitch?Don'twanttospinendlessly.*/-msleep(100);+return-EBUSY;+}+EXPORT_SYMBOL_GPL(cxl_check_and_switch_mode);++#endif /* CONFIG_CXL_BIMODAL */++staticintsetup_cxl_protocol_area(structpci_dev*dev)+{+u8val;+intrc;+intvsec=find_cxl_vsec(dev);++if(!vsec){+dev_info(&dev->dev,"CXL VSEC not found\n");+return-ENODEV;+}++rc=CXL_READ_VSEC_MODE_CONTROL(dev,vsec,&val);+if(rc){+dev_err(&dev->dev,"Failed to read current mode control: %i\n",rc);+returnrc;+}++if(!(val&CXL_VSEC_PROTOCOL_ENABLE)){+dev_err(&dev->dev,"Card not in CAPI mode!\n");+return-EIO;+}++if((val&CXL_VSEC_PROTOCOL_MASK)!=CXL_VSEC_PROTOCOL_256TB){+val&=~CXL_VSEC_PROTOCOL_MASK;+val|=CXL_VSEC_PROTOCOL_256TB;+rc=CXL_WRITE_VSEC_MODE_CONTROL(dev,vsec,val);+if(rc){+dev_err(&dev->dev,"Failed to set CXL protocol area: %i\n",rc);+returnrc;+}+}return0;}
@@ -39,6 +39,31 @@boolcxl_slot_is_supported(structpci_dev*dev,intflags);+#define CXL_BIMODE_CXL 1+#define CXL_BIMODE_PCI 2++/*+*Checkthemodethatthegivenbi-modalCXLadapteriscurrentlyinand+*changeitifnecessary.ThisdoesnotapplytoAFUdrivers.+*+*Ifthemodematchestherequestedmodethisfunctionwillreturn0-ifthe+*driverwasexpectingthegenericCXLdrivertohaveboundtotheadapterand+*itgetsthisreturnvalueitshouldfailtheprobefunctiontogivetheCXL+*driverachancetoprobeit.+*+*Ifthemodedoesnotmatchitwillstartabackgroundtasktounplugthe+*devicefromLinuxandswitchitsmode,andwillreturn-EBUSY.Atthis+*pointthecallingdrivershouldmakesureithasreleasedthedeviceand+*failitsprobefunction.+*+*TheoffsetoftheCXLVSECcanbeprovidedtothisfunction.If0ispassed,+*thisfunctionwillsearchforaCXLVSECwithID0x1280andreturn-ENODEV+*ifitisnotfound.+*/+#ifdef CONFIG_CXL_BIMODAL+intcxl_check_and_switch_mode(structpci_dev*dev,intmode,intvsec);+#endif+/* Get the AFU associated with a pci_dev */structcxl_afu*cxl_pci_to_afu(structpci_dev*dev);
From: Andrew Donnellan <hidden> Date: 2016-07-14 05:35:02
On 14/07/16 07:17, Ian Munsie wrote:
From: Ian Munsie <redacted>
The Mellanox CX4 in cxl mode uses a hybrid interrupt model, where
interrupts are routed from the networking hardware to the XSL using the
MSIX table, and from there will be transformed back into an MSIX
interrupt using the cxl style interrupts (i.e. using IVTE entries and
ranges to map a PE and AFU interrupt number to an MSIX address).
We want to hide the implementation details of cxl interrupts as much as
possible. To this end, we use a special version of the MSI setup &
teardown routines in the PHB while in cxl mode to allocate the cxl
interrupts and configure the IVTE entries in the process element.
This function does not configure the MSIX table - the CX4 card uses a
custom format in that table and it would not be appropriate to fill that
out in generic code. The rest of the functionality is similar to the
"Full MSI-X mode" described in the CAIA, and this could be easily
extended to support other adapters that use that mode in the future.
The interrupts will be associated with the default context. If the
maximum number of interrupts per context has been limited (e.g. by the
mlx5 driver), it will automatically allocate additional kernel contexts
to associate extra interrupts as required. These contexts will be
started using the same WED that was used to start the default context.
Signed-off-by: Ian Munsie <redacted>
Some minor nitpicks below, which shouldn't block acceptance.
Reviewed-by: Andrew Donnellan <redacted>
@@ -489,3 +490,73 @@ int cxl_get_max_irqs_per_process(struct pci_dev *dev)returnafu->irqs_max;}EXPORT_SYMBOL_GPL(cxl_get_max_irqs_per_process);++/*+*ThisisaspecialinterruptallocationroutinecalledfromthePHB'sMSI+*setupfunction.Whencapiinterruptsareallocatedinthismannertheymust+*stillbeassociatedwitharunningcontext,butsincetheMSIAPIshaveno+*waytospecifythisweusethedefaultcontextassociatedwiththedevice.+*+*TheMellanoxCX4hasahardwarelimitationthatrestrictsthemaximumAFU+*interruptnumber,soinordertoovercomethistheirdriverinformsusof+*therestrictionbysettingthemaximuminterruptspercontext,andwe+*allocateadditionalcontextsasnecessarysothatwecankeeptheAFU+*interruptnumberwithinthesupportedrange.+*/+int_cxl_cx4_setup_msi_irqs(structpci_dev*pdev,intnvec,inttype)+{+structcxl_context*ctx,*new_ctx,*default_ctx;+intremaining;+intrc;++ctx=default_ctx=cxl_get_context(pdev);+if(WARN_ON(!default_ctx))+return-ENODEV;
I have a very slight preference for:
if (!default_ctx) {
dev_WARN(&pdev->dev, "couldn't get default context");
return -ENODEV;
}
(I see this in your arch/powerpc code too, but that's obviously copied
from the regular powernv irq code. Also, why is there no dev_WARN_ON()
function?)
+
+ remaining = nvec;
+ while (remaining > 0) {
+ rc = cxl_allocate_afu_irqs(ctx, min(remaining, ctx->afu->irqs_max));
+ if (rc) {
+ pr_warn("%s: Failed to find enough free MSIs\n", pci_name(pdev));
dev_warn(&pdev->dev, "failed to find enough free MSIs\n"); is more
common in the cxl code.
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:08
On Wed, 2016-13-07 at 21:17:00 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The support for using the Mellanox CX4 in cxl mode will require
additions to the PHB code. In preparation for this, move the existing
cxl code out of pci-ioda.c into a separate pci-cxl.c file to keep things
more organised.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:10
On Wed, 2016-13-07 at 21:17:01 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
This extends the check that the adapter is in a CAPI capable slot so
that it may be called by external users in the kernel API. This will be
used by the upcoming Mellanox CX4 support, which needs to know ahead of
time if the card can be switched to cxl mode so that it can leave it in
PCI mode if it is not.
This API takes a parameter to check if CAPP DMA mode is supported, which
it currently only allows on P8NVL systems, since that mode currently has
issues accessing memory < 4GB on P8, and we cannot realistically avoid
that.
This API does not currently check if a CAPP unit is available (i.e. not
already assigned to another PHB) on P8. Doing so would be racy since it
is assigned on a first come first serve basis, and so long as CAPP DMA
mode is not supported on P8 we don't need this, since the only
anticipated user of this API requires CAPP DMA mode.
Cc: Philippe Bergheaud <redacted>
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:11
On Wed, 2016-13-07 at 21:17:02 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
Devices that use CAPP DMA mode (such as the Mellanox CX4) require bus
master to be enabled in order for the CAPI traffic to flow. This should
be harmless to enable for other cxl devices, so unconditionally enable
it in the adapter init flow.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:12
On Wed, 2016-13-07 at 21:17:03 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The Mellanox CX4 uses a model where the AFU is one physical function of
the device, and is used by other peer physical functions of the same
device. This will require those other devices to grab a reference on the
AFU when they are initialised to make sure that it does not go away
during their lifetime.
Move the AFU refcount functions to base.c so they can be called from
the PHB code.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:13
On Wed, 2016-13-07 at 21:17:04 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The cxl kernel API has a concept of a default context associated with
each PCI device under the virtual PHB. The Mellanox CX4 will also use
the cxl kernel API, but it does not use a virtual PHB - rather, the AFU
appears as a physical function as a peer to the networking functions.
In order to allow the kernel API to work with those networking
functions, we will need to associate a default context with them as
well. To this end, refactor the corresponding code to do this in vphb.c
and export it so that it can be called from the PHB code.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:14
On Wed, 2016-13-07 at 21:17:05 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The vPHB model of the cxl kernel API is a hierarchy where the AFU is
represented by the vPHB, and it's AFU configuration records are exposed
as functions under that vPHB. If there are no AFU configuration records
we will create a vPHB with nothing under it, which is a waste of
resources and will opt us into EEH handling despite not having anything
special to handle.
This also does not make sense for cards using the peer model of the cxl
kernel API, where the other functions of the device are exposed via
additional peer physical functions rather than AFU configuration
records. This model will also not work with the existing EEH handling in
the cxl driver, as that is designed around the vPHB model.
Skip creating the vPHB for AFUs without any AFU configuration records,
and opt out of EEH handling for them.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:15
On Wed, 2016-13-07 at 21:17:06 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
This adds support for the peer model of the cxl kernel api to the
PowerNV PHB, in which physical function 0 represents the cxl function on
the card (an XSL in the case of the CX4), which other physical functions
will use for memory access and interrupt services. It is referred to as
the peer model as these functions are peers of one another, as opposed
to the Virtual PHB model which forms a hierarchy.
This patch exports APIs to enable the peer mode, check if a PCI device
is attached to a PHB in this mode, and to set and get the peer AFU for
this mode.
The cxl driver will enable this mode for supported cards by calling
pnv_cxl_enable_phb_kernel_api(). This will set a flag in the PHB to note
that this mode is enabled, and switch out it's controller_ops for the
cxl version.
The cxl version of the controller_ops struct implements it's own
versions of the enable_device_hook and release_device to handle
refcounting on the peer AFU and to allocate a default context for the
device.
Once enabled, the cxl kernel API may not be disabled on a PHB. Currently
there is no safe way to disable cxl mode short of a reboot, so until
that changes there is no reason to support the disable path.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:17
On Wed, 2016-13-07 at 21:17:07 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
This hooks up support for using the kernel API with a real PHB. After
the AFU initialisation has completed it calls into the PHB code to pass
it the AFU that will be used by other peer physical functions on the
adapter.
The cxl_pci_to_afu API is extended to work with peer PCI devices,
retrieving the peer AFU from the PHB. This API may also now return an
error if it is called on a PCI device that is not associated with either
a cxl vPHB or a peer PCI device to an AFU, and this error is propagated
down.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:19
On Wed, 2016-13-07 at 21:17:08 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
These APIs will be used by the Mellanox CX4 support. While they function
standalone to configure existing behaviour, their primary purpose is to
allow the Mellanox driver to inform the cxl driver of a hardware
limitation, which will be used in a future patch.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:24
On Wed, 2016-13-07 at 21:17:09 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The Mellanox CX4 has a hardware limitation where only 4 bits of the
AFU interrupt number can be passed to the XSL when sending an interrupt,
limiting it to only 15 interrupts per context (AFU interrupt number 0 is
invalid).
In order to overcome this, we will allocate additional contexts linked
to the default context as extra address space for the extra interrupts -
this will be implemented in the next patch.
This patch adds the preliminary support to allow this, by way of adding
a linked list in the context structure that we use to keep track of the
contexts dedicated to interrupts, and an API to simultaneously iterate
over the related context structures, AFU interrupt numbers and hardware
interrupt numbers. The point of using a single API to iterate these is
to hide some of the details of the iteration from external code, and to
reduce the number of APIs that need to be exported via base.c to allow
built in code to call.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Frederic Barrat <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:25
On Wed, 2016-13-07 at 21:17:10 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The Mellanox CX4 in cxl mode uses a hybrid interrupt model, where
interrupts are routed from the networking hardware to the XSL using the
MSIX table, and from there will be transformed back into an MSIX
interrupt using the cxl style interrupts (i.e. using IVTE entries and
ranges to map a PE and AFU interrupt number to an MSIX address).
We want to hide the implementation details of cxl interrupts as much as
possible. To this end, we use a special version of the MSI setup &
teardown routines in the PHB while in cxl mode to allocate the cxl
interrupts and configure the IVTE entries in the process element.
This function does not configure the MSIX table - the CX4 card uses a
custom format in that table and it would not be appropriate to fill that
out in generic code. The rest of the functionality is similar to the
"Full MSI-X mode" described in the CAIA, and this could be easily
extended to support other adapters that use that mode in the future.
The interrupts will be associated with the default context. If the
maximum number of interrupts per context has been limited (e.g. by the
mlx5 driver), it will automatically allocate additional kernel contexts
to associate extra interrupts as required. These contexts will be
started using the same WED that was used to start the default context.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:26
On Wed, 2016-13-07 at 21:17:11 UTC, Ian Munsie wrote:
From: Ian Munsie <redacted>
The CX4 card cannot cope with a context with PE=0 due to a hardware
limitation, resulting in:
[ 34.166577] command failed, status limits exceeded(0x8), syndrome 0x5a7939
[ 34.166580] mlx5_core 0000:01:00.1: Failed allocating uar, aborting
Since the kernel API allocates a default context very early during
device init that will almost certainly get Process Element ID 0 there is
no easy way for us to extend the API to allow the Mellanox to inform us
of this limitation ahead of time.
Instead, work around the issue by extending the XSL structure to include
a minimum PE to allocate. Although the bug is not in the XSL, it is the
easiest place to work around this limitation given that the CX4 is
currently the only card that uses an XSL.
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Andrew Donnellan <redacted>
Reviewed-by: Frederic Barrat <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:27
On Wed, 2016-13-07 at 21:17:12 UTC, Ian Munsie wrote:
From: Andrew Donnellan <redacted>
The cxl driver will use infrastructure from pnv_php to handle device tree
updates when switching bi-modal CAPI cards into CAPI mode.
To enable this, export pnv_php_find_slot() and
pnv_php_set_slot_power_state(), and add corresponding declarations, as well
as the definition of struct pnv_php_slot, to asm/pnv-pci.h.
Cc: Gavin Shan <redacted>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Acked-by: Gavin Shan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:28
On Wed, 2016-13-07 at 21:17:13 UTC, Ian Munsie wrote:
From: Andrew Donnellan <redacted>
When calling pnv_php_set_slot_power_state() with state ==
OPAL_PCI_SLOT_OFFLINE, remove devices from the device tree as if we're
dealing with OPAL_PCI_SLOT_POWER_OFF.
Cc: Gavin Shan <redacted>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Acked-by: Gavin Shan <redacted>
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2016-07-15 10:53:29
On Wed, 2016-13-07 at 21:17:14 UTC, Ian Munsie wrote:
From: Andrew Donnellan <redacted>
Add a new API, cxl_check_and_switch_mode() to allow for switching of
bi-modal CAPI cards, such as the Mellanox CX-4 network card.
...
Co-authored-by: Ian Munsie [off-list ref]
Cc: Gavin Shan <redacted>
Signed-off-by: Andrew Donnellan <redacted>
Signed-off-by: Ian Munsie <redacted>
Reviewed-by: Gavin Shan <redacted>
From: Andrew Donnellan <hidden> Date: 2016-07-28 01:49:07
On 14/07/16 07:17, Ian Munsie wrote:
mutex_lock(&afu->contexts_lock);
idr_preload(GFP_KERNEL);
- i = idr_alloc(&ctx->afu->contexts_idr, ctx, 0,
+ i = idr_alloc(&ctx->afu->contexts_idr, ctx,
+ ctx->afu->adapter->native->sl_ops->min_pe,
As it turns out, dereferencing ctx->afu->adapter->native doesn't exactly
work on PowerVM...
Working on a fix.
Andrew
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited