This series of patches rebases on powerpc/next branch, plus below additional
patches:
https://patchwork.ozlabs.org/patch/534804/ (PATCH[1/1] Andrew's EEH fix)
https://patchwork.ozlabs.org/patch/534154/ (PATCH[7/7] Richard's SRIOV Rework)
commit 3b0e21e Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/scottwood/linux into next
The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skiboot@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.
The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 27] are related to this part.
From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. Conversely,
the device-tree node should be removed from the system when the PCI adapter is going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[28 - 43] are
doing the related work.
The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[44 - 49].
The last one, PATCH[50], is the standalone PCI hotplug driver for PowerPC PowerNV
platform.
Changelog
=========
v7:
* Reworked revision to some extent.
* Rebased to powerpc/next repository.
* Reorder/split/merge/drop according - Alexey.
* Defined macros and use array to track IO/M32/M64/DMA32 segments - Alexey.
* Merged 3 files to one for the hotplug driver - Alexey.
* As part of OPAL API, defined macros for PCI slot power state, hotplug
message type. Defined macros for PCI slot power confirmed state in
hotplug driver.
* Misc comments from Alexey.
* Reworked unflatten_dt_node() to avoid recursive function calls.
* Use EXPORT_SYMBOL_GPL() and document function's input/output - Rob/Frank.
v6:
* Patch reorder, split, squash - Alexey.
* Minor coding style - Alexey.
* Better function names for pcibios_{add,remove}_pci_devices - Bjorn
* Replace pr_warn() with dev_warn() in PowerNV hotplug driver - Bjorn
* Concurrent depth as parameter passed to __unflatten_dt_node() - Grant / Alexey
* Replace overlay with of_changeset - Grant
v5:
* Rebased to 4.1.rc6 and some unmerged patches as below:
Alexey's DDW patchset (v11);
Gavin's EEH error injection support (in mpe's next branch);
Richard's EEH cleanup patches (in mpe's next branch);
Richard's EEH support for VF (v7);
Gavin's misc EEH fixes for 4.2;
* The revision bases on skiboot corresponding patches (v7):
https://patchwork.ozlabs.org/patch/480437/
* Utilize OF overlay to update device-tree with help of newly introduced
OPAL API opal_get_overlay_dt().
* Split patches for easy review according to aik's comments.
* Fix coding style from checkpatchc.pl as pointed by aik.
* Code cleanup and misc fixup according to aik's input.
v4:
* Rebased to 4.1.RC1
* Added API to unflatten FDT blob to device node sub-tree, which is attached
the indicated parent device node. The original mechanism based on formatted
string stream has been dropped.
* The PATCH[v3 09/21] ("powerpc/eeh: Delay probing EEH device during hotplug")
was picked up sent to linux-ppc@ separately for review as Richard's "VF EEH
Support" depends on that.
v3:
* Rebased to 4.1.RC0
* PowerNV PCI infrasturcture is total refactored in order to support PCI
hotplug. The PowerNV hotplug driver is also reworked a lot because of
the changes in skiboot in order to support PCI hotplug.
Gavin Shan (50):
PCI: Add pcibios_setup_bridge()
powerpc/pci: Override pcibios_setup_bridge()
powerpc/pci: Cleanup on struct pci_controller_ops
powerpc/powernv: Cleanup on pnv_pci_ioda_controller_ops
powerpc/powernv: Drop pnv_ioda_setup_dev_PE()
powerpc/powernv: Drop phb->bdfn_to_pe()
powerpc/powernv: Reorder fields in struct pnv_phb
powerpc/powernv: Rename PE# fields in struct pnv_phb
powerpc/powernv: Fix initial IO and M32 segmap
powerpc/powernv: Simplify pnv_ioda_setup_pe_seg()
powerpc/powernv: IO and M32 mapping based on PCI device resources
powerpc/powernv: Track M64 segment consumption
powerpc/powernv: Rename M64 related functions
powerpc/powernv: M64 support on P7IOC
powerpc/powernv: Rename pnv_pci_ioda_setup_dma_pe()
powerpc/powernv: Define PNV_IODA1_DMA32_SEGSIZE
powerpc/powernv: Avoid calculating DMA32 segments on PHB3
powerpc/powernv: Remove DMA32 PE list
powerpc/powernv: Track DMA32 segment consumption
powerpc/powernv: Improve DMA32 segment calculation
powerpc/powernv: Increase PE# capacity
powerpc/powernv: Introduce pnv_ioda_init_pe()
powerpc/powernv: Use PE instead of number during setup and release
powerpc/powernv: Allocate PE# in reverse order
powerpc/powernv: Reserve PE for root bus
powerpc/powernv: Create PEs at PCI hot plugging time
powerpc/powernv: Dynamically release PEs
powerpc/pci: Rename pcibios_{add,remove}_pci_devices()
powerpc/pci: Rename pcibios_find_pci_bus()
powerpc/pci: Move pci_find_bus_by_node() around
powerpc/pci: Export pci_add_device_node_info()
powerpc/pci: Introduce pci_remove_device_node_info()
powerpc/pci: Export pci_traverse_device_nodes()
powerpc/pci: Delay populating pdn
powerpc/pci: Don't scan empty slot
powerpc/pci: Update bridge windows on PCI plug
powerpc/powernv: Simplify pnv_eeh_reset()
powerpc/powernv: Exclude root bus in pnv_pci_reset_secondary_bus()
powerpc/powernv: Fundamental reset in pnv_pci_reset_secondary_bus()
powerpc/powernv: Support PCI slot ID
powerpc/powernv: Use firmware PCI slot reset infrastructure
powerpc/powernv: Functions to get/set PCI slot status
powerpc/powernv: Select OF_DYNAMIC
drivers/of: Split unflatten_dt_node()
drivers/of: Avoid recursively calling unflatten_dt_node()
drivers/of: Rename unflatten_dt_node()
drivers/of: Specify parent node in of_fdt_unflatten_tree()
drivers/of: Return allocated memory from of_fdt_unflatten_tree()
drivers/of: Export OF changeset functions
PCI/hotplug: PowerPC PowerNV PCI hotplug driver
MAINTAINERS | 6 +
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/opal-api.h | 17 +-
arch/powerpc/include/asm/opal.h | 8 +-
arch/powerpc/include/asm/pci-bridge.h | 25 +-
arch/powerpc/include/asm/pnv-pci.h | 7 +
arch/powerpc/include/asm/ppc-pci.h | 8 +-
arch/powerpc/kernel/eeh_dev.c | 19 +-
arch/powerpc/kernel/eeh_driver.c | 12 +-
arch/powerpc/kernel/pci-common.c | 16 +-
arch/powerpc/kernel/pci-hotplug.c | 47 +-
arch/powerpc/kernel/pci_dn.c | 85 +-
arch/powerpc/platforms/maple/pci.c | 34 +-
arch/powerpc/platforms/pasemi/pci.c | 3 -
arch/powerpc/platforms/powermac/pci.c | 38 +-
arch/powerpc/platforms/powernv/Kconfig | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 173 ++--
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 +
arch/powerpc/platforms/powernv/pci-ioda.c | 1251 +++++++++++++++---------
arch/powerpc/platforms/powernv/pci.c | 92 +-
arch/powerpc/platforms/powernv/pci.h | 62 +-
arch/powerpc/platforms/pseries/msi.c | 4 +-
arch/powerpc/platforms/pseries/pci_dlpar.c | 32 -
arch/powerpc/platforms/pseries/setup.c | 8 +-
drivers/of/dynamic.c | 65 +-
drivers/of/fdt.c | 378 ++++---
drivers/of/of_private.h | 2 +
drivers/of/overlay.c | 8 +-
drivers/of/unittest.c | 6 +-
drivers/pci/hotplug/Kconfig | 12 +
drivers/pci/hotplug/Makefile | 3 +
drivers/pci/hotplug/pnv_php.c | 866 ++++++++++++++++
drivers/pci/hotplug/rpadlpar_core.c | 8 +-
drivers/pci/hotplug/rpaphp_core.c | 4 +-
drivers/pci/hotplug/rpaphp_pci.c | 4 +-
drivers/pci/setup-bus.c | 5 +
include/linux/of_fdt.h | 5 +-
include/linux/pci.h | 1 +
38 files changed, 2389 insertions(+), 932 deletions(-)
create mode 100644 drivers/pci/hotplug/pnv_php.c
--
2.1.0
Currently, PowerPC PowerNV platform utilizes ppc_md.pcibios_fixup(),
which is called for once after PCI probing and resource assignment
are completed, to allocate platform required resources for PCI devices:
PE#, IO and MMIO mapping, DMA address translation (TCE) table etc.
Obviously, it's not hotplug friendly.
This adds weak function pcibios_setup_bridge(), which is called by
pci_setup_bridge(). PowerPC PowerNV platform will reuse the function
to assign above platform required resources to newly added PCI devices,
in order to support PCI hotplug in subsequent patches.
Signed-off-by: Gavin Shan <redacted>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
---
drivers/pci/setup-bus.c | 5 +++++
include/linux/pci.h | 1 +
2 files changed, 6 insertions(+)
This overrides pcibios_setup_bridge() that is called to update PCI
bridge windows when PCI resource assignment is completed, to assign
PE and setup various (resource) mapping for the PE in subsequent
patches.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/pci-bridge.h | 2 ++
arch/powerpc/kernel/pci-common.c | 8 ++++++++
2 files changed, 10 insertions(+)
This cleans up on pnv_pci_ioda_controller_ops struct to use tab
instead of space indent of statement to avoid complains from
scripts/checkpatch.pl. No logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
Nobody uses this function and this just drops it.
Signed-off-by: Gavin Shan <redacted>
Reviewed-by: Alexey Kardashevskiy <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 71 -------------------------------
1 file changed, 71 deletions(-)
@@ -918,77 +918,6 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)}#endif /* CONFIG_PCI_IOV */-#if 0-staticstructpnv_ioda_pe*pnv_ioda_setup_dev_PE(structpci_dev*dev)-{-structpci_controller*hose=pci_bus_to_host(dev->bus);-structpnv_phb*phb=hose->private_data;-structpci_dn*pdn=pci_get_pdn(dev);-structpnv_ioda_pe*pe;-intpe_num;--if(!pdn){-pr_err("%s: Device tree node not associated properly\n",-pci_name(dev));-returnNULL;-}-if(pdn->pe_number!=IODA_INVALID_PE)-returnNULL;--/* PE#0 has been pre-set */-if(dev->bus->number==0)-pe_num=0;-else-pe_num=pnv_ioda_alloc_pe(phb);-if(pe_num==IODA_INVALID_PE){-pr_warning("%s: Not enough PE# available, disabling device\n",-pci_name(dev));-returnNULL;-}--/* NOTE: We get only one ref to the pci_dev for the pdn, not for the-*pointerinthePEdatastructure,bothshouldbedestroyedatthe-*sametime.However,thisneedstobelookedatmorecloselyagain-*onceweactuallystartremovingthings(Hotplug,SR-IOV,...)-*-*AtsomepointwewanttoremovethePDNcompletelyanyways-*/-pe=&phb->ioda.pe_array[pe_num];-pci_dev_get(dev);-pdn->pcidev=dev;-pdn->pe_number=pe_num;-pe->pdev=dev;-pe->pbus=NULL;-pe->tce32_seg=-1;-pe->mve_number=-1;-pe->rid=dev->bus->number<<8|pdn->devfn;--pe_info(pe,"Associated device to PE\n");--if(pnv_ioda_configure_pe(phb,pe)){-/* XXX What do we do here ? */-if(pe_num)-pnv_ioda_free_pe(phb,pe_num);-pdn->pe_number=IODA_INVALID_PE;-pe->pdev=NULL;-pci_dev_put(dev);-returnNULL;-}--/* Assign a DMA weight to the device */-pe->dma_weight=pnv_ioda_dma_weight(dev);-if(pe->dma_weight!=0){-phb->ioda.dma_weight+=pe->dma_weight;-phb->ioda.dma_pe_count++;-}--/* Link the PE */-pnv_ioda_link_pe_by_weight(phb,pe);--returnpe;-}-#endif /* Useful for SRIOV case */-staticvoidpnv_ioda_setup_same_PE(structpci_bus*bus,structpnv_ioda_pe*pe){structpci_dev*dev;
This moves those fields in struct pnv_phb that are related to PE
allocation around. No logical change.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci.h | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
This renames the fields related to PE number in "struct pnv_phb"
for better reflecting of their usages as Alexey suggested. No
logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 2 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 56 ++++++++++++++--------------
arch/powerpc/platforms/powernv/pci.c | 2 +-
arch/powerpc/platforms/powernv/pci.h | 4 +-
4 files changed, 32 insertions(+), 32 deletions(-)
@@ -134,7 +134,7 @@ static inline bool pnv_pci_is_mem_pref_64(unsigned long flags)staticvoidpnv_ioda_reserve_pe(structpnv_phb*phb,intpe_no){-if(!(pe_no>=0&&pe_no<phb->ioda.total_pe)){+if(!(pe_no>=0&&pe_no<phb->ioda.total_pe_num)){pr_warn("%s: Invalid PE %d on PHB#%x\n",__func__,pe_no,phb->hose->global_number);return;
@@ -154,8 +154,8 @@ static int pnv_ioda_alloc_pe(struct pnv_phb *phb)do{pe=find_next_zero_bit(phb->ioda.pe_alloc,-phb->ioda.total_pe,0);-if(pe>=phb->ioda.total_pe)+phb->ioda.total_pe_num,0);+if(pe>=phb->ioda.total_pe_num)returnIODA_INVALID_PE;}while(test_and_set_bit(pe,phb->ioda.pe_alloc));
@@ -209,13 +209,13 @@ static int pnv_ioda2_init_m64(struct pnv_phb *phb)*expectedtobe0orlastoneofPEcapabicity.*/r=&phb->hose->mem_resources[1];-if(phb->ioda.reserved_pe==0)+if(phb->ioda.reserved_pe_idx==0)r->start+=phb->ioda.m64_segsize;-elseif(phb->ioda.reserved_pe==(phb->ioda.total_pe-1))+elseif(phb->ioda.reserved_pe_idx==(phb->ioda.total_pe_num-1))r->end-=phb->ioda.m64_segsize;elsepr_warn(" Cannot strip M64 segment for reserved PE#%d\n",-phb->ioda.reserved_pe);+phb->ioda.reserved_pe_idx);return0;
@@ -284,7 +284,7 @@ static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all)returnIODA_INVALID_PE;/* Allocate bitmap */-size=_ALIGN_UP(phb->ioda.total_pe/8,sizeof(unsignedlong));+size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));pe_alloc=kzalloc(size,GFP_KERNEL);if(!pe_alloc){pr_warn("%s: Out of memory !\n",
@@ -465,7 +465,7 @@ static int pnv_ioda_get_pe_state(struct pnv_phb *phb, int pe_no)s64rc;/* Sanity check on PE number */-if(pe_no<0||pe_no>=phb->ioda.total_pe)+if(pe_no<0||pe_no>=phb->ioda.total_pe_num)returnOPAL_EEH_STOPPED_PERM_UNAVAIL;/*
@@ -1394,9 +1394,9 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)}else{mutex_lock(&phb->ioda.pe_alloc_mutex);*pdn->pe_num_map=bitmap_find_next_zero_area(-phb->ioda.pe_alloc,phb->ioda.total_pe,+phb->ioda.pe_alloc,phb->ioda.total_pe_num,0,num_vfs,0);-if(*pdn->pe_num_map>=phb->ioda.total_pe){+if(*pdn->pe_num_map>=phb->ioda.total_pe_num){mutex_unlock(&phb->ioda.pe_alloc_mutex);dev_info(&pdev->dev,"Failed to enable VF%d\n",num_vfs);kfree(pdn->pe_num_map);
@@ -3067,13 +3067,13 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,pr_err(" Failed to map registers !\n");/* Initialize more IODA stuff */-phb->ioda.total_pe=1;+phb->ioda.total_pe_num=1;prop32=of_get_property(np,"ibm,opal-num-pes",NULL);if(prop32)-phb->ioda.total_pe=be32_to_cpup(prop32);+phb->ioda.total_pe_num=be32_to_cpup(prop32);prop32=of_get_property(np,"ibm,opal-reserved-pe",NULL);if(prop32)-phb->ioda.reserved_pe=be32_to_cpup(prop32);+phb->ioda.reserved_pe_idx=be32_to_cpup(prop32);/* Parse 64-bit MMIO range */pnv_ioda_parse_m64_window(phb);
@@ -3082,29 +3082,29 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,/* FW Has already off top 64k of M32 space (MSI space) */phb->ioda.m32_size+=0x10000;-phb->ioda.m32_segsize=phb->ioda.m32_size/phb->ioda.total_pe;+phb->ioda.m32_segsize=phb->ioda.m32_size/phb->ioda.total_pe_num;phb->ioda.m32_pci_base=hose->mem_resources[0].start-hose->mem_offset[0];phb->ioda.io_size=hose->pci_io_size;-phb->ioda.io_segsize=phb->ioda.io_size/phb->ioda.total_pe;+phb->ioda.io_segsize=phb->ioda.io_size/phb->ioda.total_pe_num;phb->ioda.io_pci_base=0;/* XXX calculate this ? *//* Allocate aux data & arrays. We don't have IO ports on PHB3 */-size=_ALIGN_UP(phb->ioda.total_pe/8,sizeof(unsignedlong));+size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));m32map_off=size;-size+=phb->ioda.total_pe*sizeof(phb->ioda.m32_segmap[0]);+size+=phb->ioda.total_pe_num*sizeof(phb->ioda.m32_segmap[0]);if(phb->type==PNV_PHB_IODA1){iomap_off=size;-size+=phb->ioda.total_pe*sizeof(phb->ioda.io_segmap[0]);+size+=phb->ioda.total_pe_num*sizeof(phb->ioda.io_segmap[0]);}pemap_off=size;-size+=phb->ioda.total_pe*sizeof(structpnv_ioda_pe);+size+=phb->ioda.total_pe_num*sizeof(structpnv_ioda_pe);aux=memblock_virt_alloc(size,0);phb->ioda.pe_alloc=aux;phb->ioda.m32_segmap=aux+m32map_off;if(phb->type==PNV_PHB_IODA1)phb->ioda.io_segmap=aux+iomap_off;phb->ioda.pe_array=aux+pemap_off;-set_bit(phb->ioda.reserved_pe,phb->ioda.pe_alloc);+set_bit(phb->ioda.reserved_pe_idx,phb->ioda.pe_alloc);INIT_LIST_HEAD(&phb->ioda.pe_dma_list);INIT_LIST_HEAD(&phb->ioda.pe_list);
There are two arrays for IO and M32 segment maps on every PHB.
The index of the arrays are segment number and the value stored
in the corresponding element is PE number, indicating the segment
is assigned to the PE. Initially, all elements in those two arrays
are zeroes, meaning all segments are assigned to PE#0. It's wrong.
This fixes the initial values in the elements of those two arrays
to IODA_INVALID_PE, meaning all segments aren't assigned to any
PE. In order to use IODA_INVALID_PE (-1) to represent invalid PE
number, the types of those two arrays are changed from "unsigned int"
to "int".
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 9 +++++++--
arch/powerpc/platforms/powernv/pci.h | 4 ++--
2 files changed, 9 insertions(+), 4 deletions(-)
As we track M32 segment consumption, this introduces an array to
the PHB to track the mapping between M64 segment and PE number.
The information is going to be used to find M64 segment from the
PE number during PCI unplugging time in subsequent patches.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 10 ++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++-
2 files changed, 10 insertions(+), 3 deletions(-)
@@ -3103,6 +3104,8 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,/* Allocate aux data & arrays. We don't have IO ports on PHB3 */size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));+m64map_off=size;+size+=phb->ioda.total_pe_num*sizeof(phb->ioda.m64_segmap[0]);m32map_off=size;size+=phb->ioda.total_pe_num*sizeof(phb->ioda.m32_segmap[0]);if(phb->type==PNV_PHB_IODA1){
This renames those functions picking PE number based on consumed
M64 segments, mapping M64 segments to PEs as those functions are
going to be shared by IODA1/IODA2 in next patch. No logical changes
introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
@@ -293,7 +293,7 @@ static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all)}/* Figure out reserved PE numbers by the PE */-pnv_ioda2_reserve_m64_pe(bus,pe_alloc,all);+pnv_ioda_reserve_m64_pe(bus,pe_alloc,all);/**thecurrentbusmightnotownM64windowandthat'sall
@@ -374,8 +374,8 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)/* Use last M64 BAR to cover M64 window */phb->ioda.m64_bar_idx=15;phb->init_m64=pnv_ioda2_init_m64;-phb->reserve_m64_pe=pnv_ioda2_reserve_m64_pe;-phb->pick_m64_pe=pnv_ioda2_pick_m64_pe;+phb->reserve_m64_pe=pnv_ioda_reserve_m64_pe;+phb->pick_m64_pe=pnv_ioda_pick_m64_pe;}staticvoidpnv_ioda_freeze_pe(structpnv_phb*phb,intpe_no)
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 86 +++++++++++++++++++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++
2 files changed, 86 insertions(+), 3 deletions(-)
@@ -325,6 +383,26 @@ static int pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)pe->master=master_pe;list_add_tail(&pe->list,&master_pe->slaves);}++/*+*P7IOCsupportsM64DT,whichhelpsmappingM64segment+*tooneparticularPE#.However,PHB3hasfixedmapping+*betweenM64segmentandPE#.Inordertohavesamelogic+*forP7IOCandPHB3,weenforcefixedmappingbetweenM64+*segmentandPE#onP7IOC.+*/+if(phb->type==PNV_PHB_IODA1){+int64_trc;++rc=opal_pci_map_pe_mmio_window(phb->opal_id,+pe->pe_number,OPAL_M64_WINDOW_TYPE,+pe->pe_number/PNV_IODA1_M64_SEGS,+pe->pe_number%PNV_IODA1_M64_SEGS);+if(rc!=OPAL_SUCCESS)+pr_warn("%s: Error %lld mapping M64 for PHB#%d-PE#%d\n",+__func__,rc,phb->hose->global_number,+pe->pe_number);+}}kfree(pe_alloc);
@@ -339,8 +417,7 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)constu32*r;u64pci_addr;-/* FIXME: Support M64 for P7IOC */-if(phb->type!=PNV_PHB_IODA2){+if(phb->type!=PNV_PHB_IODA1&&phb->type!=PNV_PHB_IODA2){pr_info(" Not support M64 window\n");return;}
@@ -373,7 +450,10 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)/* Use last M64 BAR to cover M64 window */phb->ioda.m64_bar_idx=15;-phb->init_m64=pnv_ioda2_init_m64;+if(phb->type==PNV_PHB_IODA1)+phb->init_m64=pnv_ioda1_init_m64;+else+phb->init_m64=pnv_ioda2_init_m64;phb->reserve_m64_pe=pnv_ioda_reserve_m64_pe;phb->pick_m64_pe=pnv_ioda_pick_m64_pe;}
In pnv_ioda_setup_dma(), it's unnecessary to calculate the DMA32
segments for PEs on PHB3 as the whole available DMA32 space can
be assigned to one specific PE on PHB3.
This splits pnv_ioda_setup_dma() to pnv_pci_ioda1_setup_dma() and
pnv_pci_ioda2_setup_dma() in order to avoid calculating DMA32
segments for PEs on PHB3. No logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 41 ++++++++++++++++++-------------
1 file changed, 24 insertions(+), 17 deletions(-)
PEs are put into PHB DMA32 list (phb->ioda.pe_dma_list) according
to their DMA32 weight. The PEs on the list are iterated to setup
their TCE32 tables at system booting time. The list is used for
once and there is no good reason for it to survive.
This moves the logic calculating DMA32 weight of PHB and PE to
pnv_pci_ioda1_setup_dma() to drop PHB's DMA32 list.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 150 ++++++++++++++----------------
arch/powerpc/platforms/powernv/pci.h | 19 ----
2 files changed, 68 insertions(+), 101 deletions(-)
@@ -891,44 +891,6 @@ out:return0;}-staticvoidpnv_ioda_link_pe_by_weight(structpnv_phb*phb,-structpnv_ioda_pe*pe)-{-structpnv_ioda_pe*lpe;--list_for_each_entry(lpe,&phb->ioda.pe_dma_list,dma_link){-if(lpe->dma_weight<pe->dma_weight){-list_add_tail(&pe->dma_link,&lpe->dma_link);-return;-}-}-list_add_tail(&pe->dma_link,&phb->ioda.pe_dma_list);-}--staticunsignedintpnv_ioda_dma_weight(structpci_dev*dev)-{-/* This is quite simplistic. The "base" weight of a device-*is10.0meansnoDMAistobeaccountedforit.-*/--/* If it's a bridge, no DMA */-if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)-return0;--/* Reduce the weight of slow USB controllers */-if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||-dev->class==PCI_CLASS_SERIAL_USB_OHCI||-dev->class==PCI_CLASS_SERIAL_USB_EHCI)-return3;--/* Increase the weight of RAID (includes Obsidian) */-if((dev->class>>8)==PCI_CLASS_STORAGE_RAID)-return15;--/* Default */-return10;-}-#ifdef CONFIG_PCI_IOVstaticintpnv_pci_vf_resource_shift(structpci_dev*dev,intoffset){
@@ -1046,10 +1007,8 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)pe->flags|=(all?PNV_IODA_PE_BUS_ALL:PNV_IODA_PE_BUS);pe->pbus=bus;pe->pdev=NULL;-pe->tce32_seg=-1;pe->mve_number=-1;pe->rid=bus->busn_res.start<<8;-pe->dma_weight=0;if(all)pe_info(pe,"Secondary bus %d..%d associated with PE#%d\n",
@@ -1071,17 +1030,6 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)/* Put PE to the list */list_add_tail(&pe->list,&phb->ioda.pe_list);--/* Account for one DMA PE if at least one DMA capable device exist-*belowthebridge-*/-if(pe->dma_weight!=0){-phb->ioda.dma_weight+=pe->dma_weight;-phb->ioda.dma_pe_count++;-}--/* Link the PE */-pnv_ioda_link_pe_by_weight(phb,pe);}staticvoidpnv_ioda_setup_PEs(structpci_bus*bus)
@@ -1842,6 +1789,47 @@ static struct iommu_table_ops pnv_ioda2_iommu_ops = {.free=pnv_ioda2_table_free,};+staticintpnv_pci_ioda_dev_dma_weight(structpci_dev*dev,void*data)+{+unsignedint*weight=(unsignedint*)data;++/* This is quite simplistic. The "base" weight of a device+*is10.0meansnoDMAistobeaccountedforit.+*/++if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)+return0;++if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||+dev->class==PCI_CLASS_SERIAL_USB_OHCI||+dev->class==PCI_CLASS_SERIAL_USB_EHCI)+*weight+=3;+elseif((dev->class>>8)==PCI_CLASS_STORAGE_RAID)+*weight+=15;+else+*weight+=10;++return0;+}++staticunsignedintpnv_pci_ioda_pe_dma_weight(structpnv_ioda_pe*pe)+{+unsignedintweight=0;++if((pe->flags&PNV_IODA_PE_DEV)&&pe->pdev){+pnv_pci_ioda_dev_dma_weight(pe->pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS)&&pe->pbus){+structpci_dev*pdev;++list_for_each_entry(pdev,&pe->pbus->devices,bus_list)+pnv_pci_ioda_dev_dma_weight(pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS_ALL)&&pe->pbus){+pci_walk_bus(pe->pbus,pnv_pci_ioda_dev_dma_weight,&weight);+}++returnweight;+}+staticvoidpnv_pci_ioda1_setup_dma_pe(structpnv_phb*phb,structpnv_ioda_pe*pe,unsignedintbase,
@@ -1858,17 +1846,12 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,/* XXX FIXME: Provide 64-bit DMA facilities & non-4K TCE tables etc.. *//* XXX FIXME: Allocate multi-level tables on PHB3 */-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-tbl=pnv_pci_table_alloc(phb->hose->node);iommu_register_group(&pe->table_group,phb->hose->global_number,pe->pe_number);pnv_pci_link_table_and_group(phb->hose->node,0,tbl,&pe->table_group);/* Grab a 32-bit TCE table */-pe->tce32_seg=base;pe_info(pe," Setting up 32-bit TCE table at %08x..%08x\n",base*PNV_IODA1_DMA32_SEGSIZE,(base+segs)*PNV_IODA1_DMA32_SEGSIZE-1);
@@ -1932,8 +1915,6 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,return;fail:/* XXX Failure: Try to fallback to 64-bit only ? */-if(pe->tce32_seg>=0)-pe->tce32_seg=-1;if(tce_mem)__free_pages(tce_mem,get_order(tce32_segsz*segs));if(tbl){
@@ -2344,10 +2325,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,{int64_trc;-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-/* TVE #1 is selected by PCI address bit 59 */pe->tce_bypass_base=1ull<<59;
@@ -2355,7 +2332,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,pe->pe_number);/* The PE will reserve all possible 32-bits space */-pe->tce32_seg=0;pe_info(pe,"Setting up 32-bit TCE table at 0..%08x\n",phb->ioda.m32_pci_base);
@@ -2386,24 +2359,34 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,staticvoidpnv_pci_ioda1_setup_dma(structpnv_phb*phb){structpci_controller*hose=phb->hose;-unsignedintresidual,remaining,segs,tw,base;+unsignedintweight,total_weight,dma_pe_count;+unsignedintresidual,remaining,segs,base;structpnv_ioda_pe*pe;+total_weight=0;+dma_pe_count=0;+list_for_each_entry(pe,&phb->ioda.pe_list,list){+weight=pnv_pci_ioda_pe_dma_weight(pe);+if(weight>0)+dma_pe_count++;++total_weight+=weight;+}+/* If we have more PE# than segments available, hand out one*perPEuntilwerunoutandlettherestfail.Ifnot,*thenweassignatleastonesegmentperPE,plusmorebased*ontheamountofdevicesunderthatPE*/-if(phb->ioda.dma_pe_count>phb->ioda.tce32_count)+if(dma_pe_count>phb->ioda.tce32_count)residual=0;else-residual=phb->ioda.tce32_count--phb->ioda.dma_pe_count;+residual=phb->ioda.tce32_count-dma_pe_count;pr_info("PCI: Domain %04x has %ld available 32-bit DMA segments\n",hose->global_number,phb->ioda.tce32_count);pr_info("PCI: %d PE# for a total weight of %d\n",-phb->ioda.dma_pe_count,phb->ioda.dma_weight);+dma_pe_count,total_weight);pnv_pci_ioda_setup_opal_tce_kill(phb);
@@ -49,14 +49,7 @@ struct pnv_ioda_pe {/* PE number */unsignedintpe_number;-/* "Weight" assigned to the PE for the sake of DMA resource-*allocations-*/-unsignedintdma_weight;-/* "Base" iommu table, ie, 4K TCEs, 32-bit DMA */-inttce32_seg;-inttce32_segcount;structiommu_table_grouptable_group;/* 64-bit TCE bypass region */
@@ -74,7 +67,6 @@ struct pnv_ioda_pe {structlist_headslaves;/* Link in list of PE#s */-structlist_headdma_link;structlist_headlist;};
@@ -175,17 +167,6 @@ struct pnv_phb {/* 32-bit TCE tables allocation */unsignedlongtce32_count;-/* Total "weight" for the sake of DMA resources-*allocation-*/-unsignedintdma_weight;-unsignedintdma_pe_count;--/* Sorted list of used PE's, sorted at-*bootforresourceallocationpurposes-*/-structlist_headpe_dma_list;-/* TCE cache invalidate registers (physical and*remapped)*/
In current implementation, the DMA32 segments required by one specific
PE isn't calculated with the information hold in the PE independently.
It conflicts with the PCI hotplug design: PE centralized, meaning the
PE's DMA32 segments should be calculated from the information hold in
the PE independently.
This moves the logic calculating PE's consumed DMA32 segments from
pnv_pci_ioda1_setup_dma() to pnv_pci_ioda1_setup_dma_pe() so that PE's
DMA32 segments are calculated/allocated from the information hold in
the PE (DMA32 weight). Also the logic is improved: we try to allocate
as much DMA32 segments as we can. It's acceptable that number of DMA32
segments less than the expected number are allocated.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 119 ++++++++++++++----------------
1 file changed, 57 insertions(+), 62 deletions(-)
@@ -1830,15 +1830,23 @@ static unsigned int pnv_pci_ioda_pe_dma_weight(struct pnv_ioda_pe *pe)returnweight;}+staticunsignedintpnv_pci_ioda_total_dma_weight(structpnv_phb*phb)+{+unsignedintweight=0;++pci_walk_bus(phb->hose->bus,pnv_pci_ioda_dev_dma_weight,&weight);+returnweight;+}+staticvoidpnv_pci_ioda1_setup_dma_pe(structpnv_phb*phb,-structpnv_ioda_pe*pe,-unsignedintbase,-unsignedintsegs)+structpnv_ioda_pe*pe){structpage*tce_mem=NULL;structiommu_table*tbl;-unsignedinttce32_segsz,i;+unsignedintweight,total_weight;+unsignedinttce32_segsz,base,segs,i;+boolfound;int64_trc;void*addr;
@@ -1846,12 +1854,55 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,/* XXX FIXME: Provide 64-bit DMA facilities & non-4K TCE tables etc.. *//* XXX FIXME: Allocate multi-level tables on PHB3 */+total_weight=pnv_pci_ioda_total_dma_weight(phb);+weight=pnv_pci_ioda_pe_dma_weight(pe);+if(!total_weight||!weight)+return;++segs=(weight*phb->ioda.dma32_count)/total_weight;+if(!segs)+segs=1;++/*+*AllocatecontinuousDMA32segments.Webeginwiththeexpected+*numberofsegments.Withonemoreattempt,thenumberofDMA32+*segmentstobeallocatedisdecreasedbyoneuntilonesegment+*isallocatedsuccessfully.+*/+while(segs){+found=false;+for(base=0;base<=phb->ioda.dma32_count-segs;base++){+for(i=base;i<base+segs;i++){+if(phb->ioda.dma32_segmap[i]!=+IODA_INVALID_PE)+break;+}++if(i>=base+segs){+found=true;+break;+}+}++if(found)+break;++segs--;+}++if(!segs){+pe_warn(pe,"No available DMA32 resource\n");+return;+}+tbl=pnv_pci_table_alloc(phb->hose->node);iommu_register_group(&pe->table_group,phb->hose->global_number,pe->pe_number);pnv_pci_link_table_and_group(phb->hose->node,0,tbl,&pe->table_group);/* Grab a 32-bit TCE table */+pe_info(pe,"DMA weight %d (%d), assigned %d DMA32 segments\n",+weight,total_weight,segs);pe_info(pe," Setting up 32-bit TCE table at %08x..%08x\n",base*PNV_IODA1_DMA32_SEGSIZE,(base+segs)*PNV_IODA1_DMA32_SEGSIZE-1);
@@ -2362,68 +2413,12 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,staticvoidpnv_pci_ioda1_setup_dma(structpnv_phb*phb){-structpci_controller*hose=phb->hose;-unsignedintweight,total_weight,dma_pe_count;-unsignedintresidual,remaining,segs,base;structpnv_ioda_pe*pe;-total_weight=0;-dma_pe_count=0;-list_for_each_entry(pe,&phb->ioda.pe_list,list){-weight=pnv_pci_ioda_pe_dma_weight(pe);-if(weight>0)-dma_pe_count++;--total_weight+=weight;-}--/* If we have more PE# than segments available, hand out one-*perPEuntilwerunoutandlettherestfail.Ifnot,-*thenweassignatleastonesegmentperPE,plusmorebased-*ontheamountofdevicesunderthatPE-*/-if(dma_pe_count>phb->ioda.dma32_count)-residual=0;-else-residual=phb->ioda.dma32_count-dma_pe_count;--pr_info("PCI: Domain %04x has %ld available 32-bit DMA segments\n",-hose->global_number,phb->ioda.dma32_count);-pr_info("PCI: %d PE# for a total weight of %d\n",-dma_pe_count,total_weight);-pnv_pci_ioda_setup_opal_tce_kill(phb);-/* Walk our PE list and configure their DMA segments, hand them-*outonebasesegmentplusanyresidualsegmentsbasedon-*weight-*/-remaining=phb->ioda.dma32_count;-base=0;-list_for_each_entry(pe,&phb->ioda.pe_list,list){-weight=pnv_pci_ioda_pe_dma_weight(pe);-if(!weight)-continue;--if(!remaining){-pe_warn(pe,"No DMA32 resources available\n");-continue;-}-segs=1;-if(residual){-segs+=((weight*residual)+(total_weight/2))/-total_weight;-if(segs>remaining)-segs=remaining;-}--pe_info(pe,"DMA weight %d, assigned %d DMA32 segments\n",-weight,segs);-pnv_pci_ioda1_setup_dma_pe(phb,pe,base,segs);--remaining-=segs;-base+=segs;-}+list_for_each_entry(pe,&phb->ioda.pe_list,list)+pnv_pci_ioda1_setup_dma_pe(phb,pe);}staticvoidpnv_pci_ioda2_setup_dma(structpnv_phb*phb)
Each PHB maintains an array helping to translate 2-bytes Request
ID (RID) to PE# with the assumption that PE# takes one byte, meaning
that we can't have more than 256 PEs. However, pci_dn->pe_number
already had 4-bytes for the PE#.
This extends the PE# capacity so that each of them will be 4-bytes
long. Then we can reuse IODA_INVALID_PE to check the PE# stored in
phb->pe_rmap[] is valid or not.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 6 +++++-
arch/powerpc/platforms/powernv/pci.h | 7 ++-----
2 files changed, 7 insertions(+), 6 deletions(-)
In current implementation, the PEs that are allocated or picked
from the reserved list are identified by PE number. The PE instance
has to be picked according to the PE number eventually. We have
same issue when PE is released.
For pnv_ioda_pick_m64_pe() and pnv_ioda_alloc_pe(), this returns
PE instance so that pnv_ioda_setup_bus_PE() can use the allocated
or reserved PE instance directly. Also, pnv_ioda_setup_bus_PE()
returns the reserved/allocated PE instance to be used in subsequent
patches. On the other hand, pnv_ioda_free_pe() uses PE instance
(not number) as its argument. No logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 81 +++++++++++++++++--------------
arch/powerpc/platforms/powernv/pci.h | 2 +-
2 files changed, 46 insertions(+), 37 deletions(-)
@@ -160,19 +160,20 @@ static int pnv_ioda_alloc_pe(struct pnv_phb *phb)pe=find_next_zero_bit(phb->ioda.pe_alloc,phb->ioda.total_pe_num,0);if(pe>=phb->ioda.total_pe_num)-returnIODA_INVALID_PE;+returnNULL;}while(test_and_set_bit(pe,phb->ioda.pe_alloc));-pnv_ioda_init_pe(phb,pe);-returnpe;+returnpnv_ioda_init_pe(phb,pe);}-staticvoidpnv_ioda_free_pe(structpnv_phb*phb,intpe)+staticvoidpnv_ioda_free_pe(structpnv_ioda_pe*pe){-WARN_ON(phb->ioda.pe_array[pe].pdev);+structpnv_phb*phb=pe->phb;++WARN_ON(pe->pdev);-memset(&phb->ioda.pe_array[pe],0,sizeof(structpnv_ioda_pe));-clear_bit(pe,phb->ioda.pe_alloc);+memset(pe,0,sizeof(structpnv_ioda_pe));+clear_bit(pe->pe_number,phb->ioda.pe_alloc);}/* The default M64 BAR is shared by all PEs */
@@ -342,7 +343,7 @@ static int pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)/* Root bus shouldn't use M64 */if(pci_is_root_bus(bus))-returnIODA_INVALID_PE;+returnNULL;/* Allocate bitmap */size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));
@@ -350,7 +351,7 @@ static int pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)if(!pe_alloc){pr_warn("%s: Out of memory !\n",__func__);-returnIODA_INVALID_PE;+returnNULL;}/* Figure out reserved PE numbers by the PE */
@@ -988,28 +989,26 @@ static void pnv_ioda_setup_same_PE(struct pci_bus *bus, struct pnv_ioda_pe *pe)*subordinatePCIdevicesandbuses.ThesecondtypeofPEisnormally*orgiriatedbyPCIe-to-PCIbridgeorPLXswitchdownstreamports.*/-staticvoidpnv_ioda_setup_bus_PE(structpci_bus*bus,boolall)+staticstructpnv_ioda_pe*pnv_ioda_setup_bus_PE(structpci_bus*bus,boolall){structpci_controller*hose=pci_bus_to_host(bus);structpnv_phb*phb=hose->private_data;-structpnv_ioda_pe*pe;-intpe_num=IODA_INVALID_PE;+structpnv_ioda_pe*pe=NULL;/* Check if PE is determined by M64 */if(phb->pick_m64_pe)-pe_num=phb->pick_m64_pe(bus,all);+pe=phb->pick_m64_pe(bus,all);/* The PE number isn't pinned by M64 */-if(pe_num==IODA_INVALID_PE)-pe_num=pnv_ioda_alloc_pe(phb);+if(!pe)+pe=pnv_ioda_alloc_pe(phb);-if(pe_num==IODA_INVALID_PE){+if(!pe){pr_warning("%s: Not enough PE# available for PCI bus %04x:%02x\n",__func__,pci_domain_nr(bus),bus->number);-return;+returnNULL;}-pe=&phb->ioda.pe_array[pe_num];pe->flags|=(all?PNV_IODA_PE_BUS_ALL:PNV_IODA_PE_BUS);pe->pbus=bus;pe->pdev=NULL;
@@ -1018,17 +1017,16 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)if(all)pe_info(pe,"Secondary bus %d..%d associated with PE#%d\n",-bus->busn_res.start,bus->busn_res.end,pe_num);+bus->busn_res.start,bus->busn_res.end,pe->pe_number);elsepe_info(pe,"Secondary bus %d associated with PE#%d\n",-bus->busn_res.start,pe_num);+bus->busn_res.start,pe->pe_number);if(pnv_ioda_configure_pe(phb,pe)){/* XXX What do we do here ? */-if(pe_num)-pnv_ioda_free_pe(phb,pe_num);+pnv_ioda_free_pe(pe);pe->pbus=NULL;-return;+returnNULL;}/* Associate it with all child devices */
@@ -1036,6 +1034,8 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)/* Put PE to the list */list_add_tail(&pe->list,&phb->ioda.pe_list);++returnpe;}staticvoidpnv_ioda_setup_PEs(structpci_bus*bus)
@@ -1354,9 +1358,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs)if(pnv_ioda_configure_pe(phb,pe)){/* XXX What do we do here ? */-if(pe_num)-pnv_ioda_free_pe(phb,pe_num);pe->pdev=NULL;+pnv_ioda_free_pe(pe);continue;}
@@ -1374,6 +1377,7 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)structpci_bus*bus;structpci_controller*hose;structpnv_phb*phb;+structpnv_ioda_pe*pe;structpci_dn*pdn;intret;u16i;
@@ -1416,11 +1420,13 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)/* Calculate available PE for required VFs */if(pdn->m64_single_mode){for(i=0;i<num_vfs;i++){-pdn->pe_num_map[i]=pnv_ioda_alloc_pe(phb);-if(pdn->pe_num_map[i]==IODA_INVALID_PE){+pe=pnv_ioda_alloc_pe(phb);+if(!pe){ret=-EBUSY;gotom64_failed;}++pdn->pe_num_map[i]=pe->pe_number;}}else{mutex_lock(&phb->ioda.pe_alloc_mutex);
@@ -1465,8 +1471,11 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)m64_failed:if(pdn->m64_single_mode){for(i=0;i<num_vfs;i++){-if(pdn->pe_num_map[i]!=IODA_INVALID_PE)-pnv_ioda_free_pe(phb,pdn->pe_num_map[i]);+if(pdn->pe_num_map[i]==IODA_INVALID_PE)+continue;++pe=&phb->ioda.pe_array[pdn->pe_num_map[i]];+pnv_ioda_free_pe(pe);}}elsebitmap_clear(phb->ioda.pe_alloc,*pdn->pe_num_map,num_vfs);
We're going to reserve/assign PEs when pcibios_setup_bridge() is
called. The function won't be called for root bus as it doesn't
have parent bridge. However, the root bus still needs a PE to be
covered.
This reserves PE numbers that are adjacent to the reserved one
for root buses.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 33 ++++++++++++++++++++++---------
arch/powerpc/platforms/powernv/pci.h | 1 +
2 files changed, 25 insertions(+), 9 deletions(-)
Currently, the PEs and their associated resources are assigned
in ppc_md.pcibios_fixup() except those used by SRIOV VFs. The
function is called for once after PCI probing and resources
assignment is completed. So it isn't hotplug friendly.
This creates PEs dynamically by ppc_md.pcibios_setup_bridge(), which
is called on the event during system bootup and PCI hotplug: updating
PCI bridge's windows after resource assignment/reassignment are done.
For partial hotplug case, where not all PCI devices belonging to the
PE are unplugged and plugged again, we just need unbinding/binding
the affected PCI devices with the corresponding PE without creating
new one.
As there is no upstream bridge for root bus that needs to be covered
by PE, we have to create PE for root bus in ppc_md.pcibios_setup_bridge()
before any other PEs can be created, as PE for root bus is the ancestor
to anyone else.
On the other hand, the windows of root port or the upstream port
of PCIe switch behind root port are extended to be PHB's aperatuses
to accommodate the additonal resources needed by newly plugged devices
based on the fact: hotpluggable slot is behind root port or downstream
port of the PCIe switch behind root port. The extension for those
PCI brdiges' windows is done in ppc_md.pcibios_setup_bridge() as
well.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 240 +++++++++++++++++-------------
arch/powerpc/platforms/powernv/pci.h | 1 +
2 files changed, 138 insertions(+), 103 deletions(-)
@@ -992,9 +1001,26 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)structpci_controller*hose=pci_bus_to_host(bus);structpnv_phb*phb=hose->private_data;structpnv_ioda_pe*pe=NULL;+intpe_num;++/*+*Inpartialhotplugcase,thePEinstancemightbestillalive.+*Weshouldreuseitinsteadofallocatinganewone.+*/+pe_num=phb->ioda.pe_rmap[bus->number<<8];+if(pe_num!=IODA_INVALID_PE){+pe=&phb->ioda.pe_array[pe_num];+pnv_ioda_setup_same_PE(bus,pe);+returnNULL;+}++/* PE number for root bus should have been reserved */+if(pci_is_root_bus(bus)&&+phb->ioda.root_pe_idx!=IODA_INVALID_PE)+pe=&phb->ioda.pe_array[phb->ioda.root_pe_idx];/* Check if PE is determined by M64 */-if(phb->pick_m64_pe)+if(!pe&&phb->pick_m64_pe)pe=phb->pick_m64_pe(bus,all);/* The PE number isn't pinned by M64 */
@@ -3019,6 +2951,104 @@ static resource_size_t pnv_pci_window_alignment(struct pci_bus *bus,returnphb->ioda.io_segsize;}+/*+*Weareupdatingrootportortheupstreamportofthe+*bridgebehindtherootportwithPHB'swindowsinorder+*toaccommodatethechangesonrequiredresourcesduring+*PCI(slot)hotplug,whichisconnectedtoeitherroot+*portorthedownstreamportsofPCIeswitchbehindthe+*rootport.+*/+staticvoidpnv_pci_fixup_bridge_resources(structpci_bus*bus,+unsignedlongtype)+{+structpci_controller*hose=pci_bus_to_host(bus);+structpnv_phb*phb=hose->private_data;+structpci_dev*bridge=bus->self;+structresource*r,*w;+inti;++/* Check if we need apply fixup to the bridge's windows */+if(!pci_is_root_bus(bridge->bus)&&+!pci_is_root_bus(bridge->bus->self->bus))+return;++/* Fixup the resoureces */+for(i=0;i<PCI_BRIDGE_RESOURCE_NUM;i++){+r=&bridge->resource[PCI_BRIDGE_RESOURCES+i];+if(!r->flags||!r->parent)+continue;++w=NULL;+if(r->flags&type&IORESOURCE_IO)+w=&hose->io_resource;+elseif(pnv_pci_is_mem_pref_64(r->flags)&&+(type&IORESOURCE_PREFETCH)&&+phb->ioda.m64_segsize)+w=&hose->mem_resources[1];+elseif(r->flags&type&IORESOURCE_MEM)+w=&hose->mem_resources[0];++r->start=w->start;+r->end=w->end;+}+}++staticvoidpnv_pci_setup_bridge(structpci_bus*bus,+unsignedlongtype)+{+structpci_controller*hose=pci_bus_to_host(bus);+structpnv_phb*phb=hose->private_data;+structpci_dev*bridge=bus->self;+structpnv_ioda_pe*pe;+boolall=(pci_pcie_type(bridge)==PCI_EXP_TYPE_PCI_BRIDGE);++/* The PE for root bus should be realized before any one else */+if(!phb->ioda.root_pe_populated){+pe=pnv_ioda_setup_bus_PE(phb->hose->bus,false);+if(pe){+phb->ioda.root_pe_idx=pe->pe_number;+phb->ioda.root_pe_populated=true;+}+}++/* Extend bridge's windows if necessary */+pnv_pci_fixup_bridge_resources(bus,type);++/* Don't assign PE to PCI bus, which doesn't have subordinate devices */+if(list_empty(&bus->devices))+return;++/* Reserve PEs according to used M64 resources */+if(phb->reserve_m64_pe)+phb->reserve_m64_pe(bus,NULL,all);++/*+*AssignPE.Wemightrunherebecauseofpartialhotplug.+*Forthecase,wejustpickuptheexistingPEandshould+*notallocateresourcesagain.+*/+pe=pnv_ioda_setup_bus_PE(bus,all);+if(!pe)+return;++/* Setup MMIO mapping */+pnv_ioda_setup_pe_seg(pe);++/* Setup DMA */+switch(phb->type){+casePNV_PHB_IODA1:+pnv_pci_ioda1_setup_dma_pe(phb,pe);+break;+casePNV_PHB_IODA2:+pnv_pci_ioda2_setup_dma_pe(phb,pe);+break;+default:+pr_warn("%s: No DMA for PHB#%d (type %d)\n",+__func__,phb->hose->global_number,phb->type);+}+}+#ifdef CONFIG_PCI_IOVstaticresource_size_tpnv_pci_iov_resource_alignment(structpci_dev*pdev,intresno)
This adds a reference count of PE, representing the number of PCI
devices associated with the PE. The reference count is increased
or decreased when PCI devices join or leave the PE. Once it becomes
zero, the PE together with its used resources (IO, MMIO, DMA, PELTM,
PELTV) are released to support PCI hot unplug.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 245 ++++++++++++++++++++++++++----
arch/powerpc/platforms/powernv/pci.h | 1 +
2 files changed, 218 insertions(+), 28 deletions(-)
@@ -129,6 +129,215 @@ static inline bool pnv_pci_is_mem_pref_64(unsigned long flags)(IORESOURCE_MEM_64|IORESOURCE_PREFETCH));}+staticvoidpnv_pci_ioda1_release_dma_pe(structpnv_ioda_pe*pe)+{+structpnv_phb*phb=pe->phb;+structiommu_table*tbl;+intstart,count,i;+int64_trc;++/* Search for the used DMA32 segments */+start=-1;+count=0;+for(i=0;i<phb->ioda.dma32_count;i++){+if(phb->ioda.dma32_segmap[i]!=pe->pe_number)+continue;++count++;+if(start<0)+start=i;+}++if(!count)+return;++/* Unlink IOMMU table from group */+tbl=pe->table_group.tables[0];+pnv_pci_unlink_table_and_group(tbl,&pe->table_group);+if(pe->table_group.group){+iommu_group_put(pe->table_group.group);+WARN_ON(pe->table_group.group);+}++/* Release IOMMU table */+pnv_pci_ioda2_table_free_pages(tbl);+iommu_free_table(tbl,of_node_full_name(pci_bus_to_OF_node(pe->pbus)));++/* Disable TVE */+for(i=start;i<start+count;i++){+rc=opal_pci_map_pe_dma_window(phb->opal_id,pe->pe_number,+i,0,0ul,0ul,0ul);+if(rc)+pe_warn(pe,"Error %ld unmapping DMA32 seg#%d\n",+rc,i);++phb->ioda.dma32_segmap[i]=IODA_INVALID_PE;+}+}++staticunsignedintpnv_pci_ioda_pe_dma_weight(structpnv_ioda_pe*pe);+staticlongpnv_pci_ioda2_unset_window(structiommu_table_group*table_group,+intnum);+staticvoidpnv_pci_ioda2_set_bypass(structpnv_ioda_pe*pe,boolenable);++staticvoidpnv_pci_ioda2_release_dma_pe(structpnv_ioda_pe*pe)+{+structiommu_table*tbl;+unsignedintweight=pnv_pci_ioda_pe_dma_weight(pe);+int64_trc;++if(!weight)+return;++tbl=pe->table_group.tables[0];+rc=pnv_pci_ioda2_unset_window(&pe->table_group,0);+if(rc)+pe_warn(pe,"OPAL error %ld release DMA window\n",rc);++pnv_pci_ioda2_set_bypass(pe,false);+if(pe->table_group.group){+iommu_group_put(pe->table_group.group);+WARN_ON(pe->table_group.group);+}++pnv_pci_ioda2_table_free_pages(tbl);+iommu_free_table(tbl,"pnv");+}++staticvoidpnv_ioda_release_dma_pe(structpnv_ioda_pe*pe)+{+structpnv_phb*phb=pe->phb;++switch(phb->type){+casePNV_PHB_IODA1:+pnv_pci_ioda1_release_dma_pe(pe);+break;+casePNV_PHB_IODA2:+pnv_pci_ioda2_release_dma_pe(pe);+break;+default:+WARN_ON(1);+}+}++staticvoidpnv_ioda_release_window(structpnv_ioda_pe*pe,intwin)+{+structpnv_phb*phb=pe->phb;+intindex,*segmap=NULL;+int64_trc;++switch(win){+caseOPAL_IO_WINDOW_TYPE:+segmap=phb->ioda.io_segmap;+break;+caseOPAL_M32_WINDOW_TYPE:+segmap=phb->ioda.m32_segmap;+break;+caseOPAL_M64_WINDOW_TYPE:+if(phb->type!=PNV_PHB_IODA1)+return;+segmap=phb->ioda.m64_segmap;+break;+default:+return;+}++for(index=0;index<phb->ioda.total_pe_num;index++){+if(segmap[index]!=pe->pe_number)+continue;++if(win==OPAL_M64_WINDOW_TYPE)+rc=opal_pci_map_pe_mmio_window(phb->opal_id,+phb->ioda.reserved_pe_idx,win,+index/PNV_IODA1_M64_SEGS,+index%PNV_IODA1_M64_SEGS);+else+rc=opal_pci_map_pe_mmio_window(phb->opal_id,+phb->ioda.reserved_pe_idx,win,+0,index);++if(rc!=OPAL_SUCCESS)+pe_warn(pe,"Error %ld unmapping (%d) segment#%d\n",+rc,win,index);++segmap[index]=IODA_INVALID_PE;+}+}++staticvoidpnv_ioda_release_pe_seg(structpnv_ioda_pe*pe)+{+structpnv_phb*phb=pe->phb;+intwin;++for(win=OPAL_M32_WINDOW_TYPE;win<=OPAL_IO_WINDOW_TYPE;win++){+if(phb->type==PNV_PHB_IODA2&&win==OPAL_IO_WINDOW_TYPE)+continue;++pnv_ioda_release_window(pe,win);+}+}++staticintpnv_ioda_deconfigure_pe(structpnv_phb*phb,+structpnv_ioda_pe*pe);+staticvoidpnv_ioda_free_pe(structpnv_ioda_pe*pe);+staticvoidpnv_ioda_release_pe(structpnv_ioda_pe*pe)+{+structpnv_ioda_pe*tmp,*slave;++/* Release slave PEs in compound PE */+if(pe->flags&PNV_IODA_PE_MASTER){+list_for_each_entry_safe(slave,tmp,&pe->slaves,list)+pnv_ioda_release_pe(slave);+}++/* Remove the PE from the list */+list_del(&pe->list);++/* Release resources */+pnv_ioda_release_dma_pe(pe);+pnv_ioda_release_pe_seg(pe);+pnv_ioda_deconfigure_pe(pe->phb,pe);++pnv_ioda_free_pe(pe);+}++staticinlinestructpnv_ioda_pe*pnv_ioda_pe_get(structpnv_ioda_pe*pe)+{+if(!pe)+returnNULL;++pe->device_count++;+returnpe;+}++staticinlinevoidpnv_ioda_pe_put(structpnv_ioda_pe*pe)+{+if(!pe)+return;++pe->device_count--;+WARN_ON(pe->device_count<0);+if(pe->device_count==0)+pnv_ioda_release_pe(pe);+}++staticvoidpnv_pci_release_device(structpci_dev*pdev)+{+structpci_controller*hose=pci_bus_to_host(pdev->bus);+structpnv_phb*phb=hose->private_data;+structpci_dn*pdn=pci_get_pdn(pdev);+structpnv_ioda_pe*pe;++if(pdev->is_virtfn)+return;++if(!pdn||pdn->pe_number==IODA_INVALID_PE)+return;++pe=&phb->ioda.pe_array[pdn->pe_number];+pnv_ioda_pe_put(pe);+}+staticstructpnv_ioda_pe*pnv_ioda_init_pe(structpnv_phb*phb,intpe_no){phb->ioda.pe_array[pe_no].phb=phb;
@@ -724,7 +933,6 @@ static int pnv_ioda_set_peltv(struct pnv_phb *phb,return0;}-#ifdef CONFIG_PCI_IOVstaticintpnv_ioda_deconfigure_pe(structpnv_phb*phb,structpnv_ioda_pe*pe){structpci_dev*parent;
@@ -1047,9 +1259,8 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)bus->busn_res.start,pe->pe_number);if(pnv_ioda_configure_pe(phb,pe)){-/* XXX What do we do here ? */-pnv_ioda_free_pe(pe);pe->pbus=NULL;+pnv_ioda_release_pe(pe);returnNULL;}
@@ -30,6 +30,7 @@ struct pnv_phb;structpnv_ioda_pe{unsignedlongflags;structpnv_phb*phb;+intdevice_count;/* A PE can be associated with a single device or an*entirebus(&children).Intheformercase,pdev
@@ -260,10 +260,10 @@ static inline struct eeh_dev *pdn_to_eeh_dev(struct pci_dn *pdn)externstructpci_bus*pcibios_find_pci_bus(structdevice_node*dn);/** Remove all of the PCI devices under this bus */-externvoidpcibios_remove_pci_devices(structpci_bus*bus);+externvoidpci_remove_pci_devices(structpci_bus*bus);/** Discover new pci devices under this bus, and add them */-externvoidpcibios_add_pci_devices(structpci_bus*bus);+externvoidpci_add_pci_devices(structpci_bus*bus);externvoidisa_bridge_find_early(structpci_controller*hose);
@@ -38,20 +38,20 @@ void pcibios_release_device(struct pci_dev *dev)}/**-*pcibios_remove_pci_devices-removealldevicesunderthisbus+*pci_remove_pci_devices-removealldevicesunderthisbus*@bus:theindicatedPCIbus**RemoveallofthePCIdevicesunderthisbusbothfromthe*linuxpcidevicetree,andfromthepowerpcEEHaddresscache.*/-voidpcibios_remove_pci_devices(structpci_bus*bus)+voidpci_remove_pci_devices(structpci_bus*bus){structpci_dev*dev,*tmp;structpci_bus*child_bus;/* First go down child busses */list_for_each_entry(child_bus,&bus->children,node)-pcibios_remove_pci_devices(child_bus);+pci_remove_pci_devices(child_bus);pr_debug("PCI: Removing devices on bus %04x:%02x\n",pci_domain_nr(bus),bus->number);
This moves pci_find_bus_by_node() from arch/powerpc/platforms/
pseries/pci_dlpar.c to arch/powerpc/kernel/pci-hotplug.c so that
the function can be used by pSeries and PowerNV platform at the
same time. Also, below cleanup applied. No functional changes
introduced.
* Remove variable "busdn" in find_bus_among_children()
* Use PCI_DN() to convert device node to pci_dn
Signed-off-by: Gavin Shan <redacted>
Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
arch/powerpc/kernel/pci-hotplug.c | 29 ++++++++++++++++++++++++++++
arch/powerpc/platforms/pseries/pci_dlpar.c | 31 ------------------------------
2 files changed, 29 insertions(+), 31 deletions(-)
This renames update_dn_pci_info() to pci_add_device_node_info()
with corresponding adjustment on the parameter type and exports it.
The function is used to create pdn (struct pci_dn) for the indicated
device node. Another function add_pdn(), almost wrapper of
pci_add_device_node_info(), to be used in traverse_pci_devices(). No
logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/pci-bridge.h | 3 ++-
arch/powerpc/kernel/pci_dn.c | 30 +++++++++++++++++++-----------
arch/powerpc/platforms/pseries/setup.c | 2 +-
3 files changed, 22 insertions(+), 13 deletions(-)
@@ -429,8 +438,7 @@ void pci_devs_phb_init_dynamic(struct pci_controller *phb)structpci_dn*pdn;/* PHB nodes themselves must not match */-update_dn_pci_info(dn,phb);-pdn=dn->data;+pdn=pci_add_device_node_info(phb,dn);if(pdn){pdn->devfn=pdn->busno=-1;pdn->vendor_id=pdn->device_id=pdn->class_code=0;
@@ -439,7 +447,7 @@ void pci_devs_phb_init_dynamic(struct pci_controller *phb)}/* Update dn->phb ptrs for new phb and children devices */-traverse_pci_devices(dn,update_dn_pci_info,phb);+traverse_pci_devices(dn,add_pdn,phb);}/**
This implements and exports pci_remove_device_node_info(). It's
used to remove the pdn (struct pci_dn) for the indicated device
node. The function is going to be used by PowerNV PCI hotplug
driver.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/pci-bridge.h | 1 +
arch/powerpc/kernel/pci_dn.c | 23 +++++++++++++++++++++++
2 files changed, 24 insertions(+)
The pdn (struct pci_dn) instances are allocated from memblock or
bootmem when creating PCI controller (hoses) in setup_arch(). PCI
hotplug, which will be supported by proceeding patches, release
PCI device nodes and their corresponding pdn on unplugging event.
The memory chunks for pdn instances allocated from memblock or
bootmem are hard to reused after being released.
This delays creating pdn in core_initcall_sync(eeh_dev_phb_init) so
that they are allocated from slab. In turn, the memory chunks for
them can be reused after being released without problem. Since the
pdn and eeh_dev has same life cycle, the eeh_dev is created when
pdn is populated. We needn't create eeh_dev with another initcall.
The time to create PHB PEs is delayed a bit from core_initcall() to
core_initcall_sync().
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/ppc-pci.h | 2 --
arch/powerpc/kernel/eeh_dev.c | 19 ++++-------------
arch/powerpc/kernel/pci_dn.c | 20 ++++++++++++++++--
arch/powerpc/platforms/maple/pci.c | 34 ++++++++++++++++++------------
arch/powerpc/platforms/pasemi/pci.c | 3 ---
arch/powerpc/platforms/powermac/pci.c | 38 +++++++++++++++++++++-------------
arch/powerpc/platforms/powernv/pci.c | 3 ---
arch/powerpc/platforms/pseries/setup.c | 6 +-----
9 files changed, 69 insertions(+), 58 deletions(-)
@@ -80,16 +79,8 @@ void *eeh_dev_init(struct pci_dn *pdn, void *data)*/voideeh_dev_phb_init_dynamic(structpci_controller*phb){-structpci_dn*root=phb->pci_data;-/* EEH PE for PHB */eeh_phb_pe_create(phb);--/* EEH device for PHB */-eeh_dev_init(root,phb);--/* EEH devices for children OF nodes */-traverse_pci_dn(root,eeh_dev_init,phb);}/**
@@ -105,9 +96,7 @@ static int __init eeh_dev_phb_init(void)list_for_each_entry_safe(phb,tmp,&hose_list,list_node)eeh_dev_phb_init_dynamic(phb);-pr_info("EEH: devices created\n");-return0;}-core_initcall(eeh_dev_phb_init);+core_initcall_sync(eeh_dev_phb_init);
@@ -486,15 +498,19 @@ void pci_devs_phb_init_dynamic(struct pci_controller *phb)*pcidevicefoundunderneath.Thisroutinerunsonce,*earlyinthebootsequence.*/-void__initpci_devs_phb_init(void)+staticint__initpci_devs_phb_init(void){structpci_controller*phb,*tmp;/* This must be done first so the device nodes have valid pci info! */list_for_each_entry_safe(phb,tmp,&hose_list,list_node)pci_devs_phb_init_dynamic(phb);++return0;}+core_initcall(pci_devs_phb_init);+staticvoidpci_dev_pdn_setup(structpci_dev*pdev){structpci_dn*pdn;
@@ -568,6 +568,26 @@ void maple_pci_irq_fixup(struct pci_dev *dev)DBG(" <- maple_pci_irq_fixup\n");}+staticintmaple_pci_root_bridge_prepare(structpci_host_bridge*bridge)+{+structpci_controller*hose=pci_bus_to_host(bridge->bus);+structdevice_node*np,*child;++if(hose!=u3_agp)+return0;++/* Fixup the PCI<->OF mapping for U3 AGP due to bus renumbering. We+*assumethereisnoP2PbridgeontheAGPbus,whichshouldbea+*safeassumptionshopefully.+*/+np=hose->dn;+PCI_DN(np)->busno=0xf0;+for_each_child_of_node(np,child)+PCI_DN(child)->busno=0xf0;++return0;+}+void__initmaple_pci_init(void){structdevice_node*np,*root;
@@ -605,19 +625,7 @@ void __init maple_pci_init(void)if(ht&&maple_add_bridge(ht)!=0)of_node_put(ht);-/* Setup the linkage between OF nodes and PHBs */-pci_devs_phb_init();--/* Fixup the PCI<->OF mapping for U3 AGP due to bus renumbering. We-*assumethereisnoP2PbridgeontheAGPbus,whichshouldbea-*safeassumptionshopefully.-*/-if(u3_agp){-structdevice_node*np=u3_agp->dn;-PCI_DN(np)->busno=0xf0;-for(np=np->child;np;np=np->sibling)-PCI_DN(np)->busno=0xf0;-}+ppc_md.pcibios_root_bridge_prepare=maple_pci_root_bridge_prepare;/* Tell pci.c to not change any resource allocations. */pci_add_flags(PCI_PROBE_ONLY);
@@ -229,9 +229,6 @@ void __init pas_pci_init(void)of_node_get(np);of_node_put(root);--/* Setup the linkage between OF nodes and PHBs */-pci_devs_phb_init();}void__iomem*pasemi_pci_getcfgaddr(structpci_dev*dev,intoffset)
@@ -878,6 +878,29 @@ void pmac_pci_irq_fixup(struct pci_dev *dev)#endif /* CONFIG_PPC32 */}+#ifdef CONFIG_PPC64+staticintpmac_pci_root_bridge_prepare(structpci_host_bridge*bridge)+{+structpci_controller*hose=pci_bus_to_host(bridge->bus);+structdevice_node*np,*child;++if(hose!=u3_agp)+return0;++/* Fixup the PCI<->OF mapping for U3 AGP due to bus renumbering. We+*assumethereisnoP2PbridgeontheAGPbus,whichshouldbea+*safeassumptionsfornow.Weshoulddosomethingbetterinthe+*futurethough+*/+np=hose->dn;+PCI_DN(np)->busno=0xf0;+for_each_child_of_node(np,child)+PCI_DN(child)->busno=0xf0;++return0;+}+#endif /* CONFIG_PPC64 */+void__initpmac_pci_init(void){structdevice_node*np,*root;
@@ -914,20 +937,7 @@ void __init pmac_pci_init(void)if(ht&&pmac_add_bridge(ht)!=0)of_node_put(ht);-/* Setup the linkage between OF nodes and PHBs */-pci_devs_phb_init();--/* Fixup the PCI<->OF mapping for U3 AGP due to bus renumbering. We-*assumethereisnoP2PbridgeontheAGPbus,whichshouldbea-*safeassumptionsfornow.Weshoulddosomethingbetterinthe-*futurethough-*/-if(u3_agp){-structdevice_node*np=u3_agp->dn;-PCI_DN(np)->busno=0xf0;-for(np=np->child;np;np=np->sibling)-PCI_DN(np)->busno=0xf0;-}+ppc_md.pcibios_root_bridge_prepare=pmac_pci_root_bridge_prepare;/* pmac_check_ht_link(); */#else /* CONFIG_PPC64 */
In hotplug case, function pcibios_add_pci_devices() is called to
rescan the specified PCI bus, which might not have any child devices.
Access to the PCI bus's child device node will cause kernel crash
without exception.
This adds condition to skip scanning PCI bus that doesn't have any
subordinate devices, in order to avoid kernel crash.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/kernel/pci-hotplug.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
This drops unnecessary nested if statements in pnv_eeh_reset() to
improve the code readability. After the changes, the unused local
variable "ret" is dropped as well. No logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 61 ++++++++++++----------------
1 file changed, 27 insertions(+), 34 deletions(-)
When pnv_pci_reset_secondary_bus() is called to issue reset on
the indicated secondary bus, the bus can't be root bus. So we
needn't consider root bus in the function.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
In pnv_pci_reset_secondary_bus(), we should issue fundamental
reset if any one subordinate device of the specified is requesting
that. Otherwise, the device might not come up after the reset.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
PowerNV platforms runs on top of skiboot firmware that includes
changes to support PCI slots. PCI slots are identified by PHB's
ID or the combo of that and PCI slot ID.
This changes the EEH PowerNV backend to support PCI slots:
* Rename arguments of opal_pci_reset() and opal_pci_poll().
* One more argument (PCI slot's state) added to opal_pci_poll().
* Drop pnv_eeh_phb_poll() and introduce a enhanced similar
function pnv_pci_poll() that will be used by PowerNV hotplug
backends.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/opal.h | 4 +--
arch/powerpc/platforms/powernv/eeh-powernv.c | 42 ++++++----------------------
arch/powerpc/platforms/powernv/pci.c | 21 ++++++++++++++
arch/powerpc/platforms/powernv/pci.h | 1 +
4 files changed, 32 insertions(+), 36 deletions(-)
@@ -729,28 +729,11 @@ static int pnv_eeh_get_state(struct eeh_pe *pe, int *delay)returnret;}-statics64pnv_eeh_phb_poll(structpnv_phb*phb)-{-s64rc=OPAL_HARDWARE;--while(1){-rc=opal_pci_poll(phb->opal_id);-if(rc<=0)-break;--if(system_state<SYSTEM_RUNNING)-udelay(1000*rc);-else-msleep(rc);-}--returnrc;-}-intpnv_eeh_phb_reset(structpci_controller*hose,intoption){structpnv_phb*phb=hose->private_data;s64rc=OPAL_HARDWARE;+intret;pr_debug("%s: Reset PHB#%x, option=%d\n",__func__,hose->global_number,option);
@@ -765,8 +748,6 @@ int pnv_eeh_phb_reset(struct pci_controller *hose, int option)rc=opal_pci_reset(phb->opal_id,OPAL_RESET_PHB_COMPLETE,OPAL_DEASSERT_RESET);-if(rc<0)-gotoout;/**PollstateofthePHBuntiltherequestisdone
@@ -774,24 +755,22 @@ int pnv_eeh_phb_reset(struct pci_controller *hose, int option)*resetfollowedbyhotresetonrootbus.Sowealso*needthePCIbussettlementdelay.*/-rc=pnv_eeh_phb_poll(phb);-if(option==EEH_RESET_DEACTIVATE){+ret=pnv_pci_poll(phb->opal_id,rc,NULL);+if(option==EEH_RESET_DEACTIVATE&&!ret){if(system_state<SYSTEM_RUNNING)udelay(1000*EEH_PE_RST_SETTLE_TIME);elsemsleep(EEH_PE_RST_SETTLE_TIME);}-out:-if(rc!=OPAL_SUCCESS)-return-EIO;-return0;+returnret;}staticintpnv_eeh_root_reset(structpci_controller*hose,intoption){structpnv_phb*phb=hose->private_data;s64rc=OPAL_HARDWARE;+intret;pr_debug("%s: Reset PHB#%x, option=%d\n",__func__,hose->global_number,option);
@@ -813,18 +792,13 @@ static int pnv_eeh_root_reset(struct pci_controller *hose, int option)rc=opal_pci_reset(phb->opal_id,OPAL_RESET_PCI_HOT,OPAL_DEASSERT_RESET);-if(rc<0)-gotoout;/* Poll state of the PHB until the request is done */-rc=pnv_eeh_phb_poll(phb);-if(option==EEH_RESET_DEACTIVATE)+ret=pnv_pci_poll(phb->opal_id,rc,NULL);+if(option==EEH_RESET_DEACTIVATE&&!ret)msleep(EEH_PE_RST_SETTLE_TIME);-out:-if(rc!=OPAL_SUCCESS)-return-EIO;-return0;+returnret;}staticintpnv_eeh_bridge_reset(structpci_dev*dev,intoption)
The skiboot firmware might provide the PCI slot reset capability
which is identified by property "ibm,reset-by-firmware" on the
PCI slot associated device node.
This checks the property. If it exists, the reset request is routed
to firmware. Otherwise, the reset is done by kernel as before.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 41 +++++++++++++++++++++++++++-
1 file changed, 40 insertions(+), 1 deletion(-)
@@ -801,7 +801,7 @@ static int pnv_eeh_root_reset(struct pci_controller *hose, int option)returnret;}-staticintpnv_eeh_bridge_reset(structpci_dev*dev,intoption)+staticint__pnv_eeh_bridge_reset(structpci_dev*dev,intoption){structpci_dn*pdn=pci_get_pdn_by_devfn(dev->bus,dev->devfn);structeeh_dev*edev=pdn_to_eeh_dev(pdn);
@@ -852,6 +852,45 @@ static int pnv_eeh_bridge_reset(struct pci_dev *dev, int option)return0;}+staticintpnv_eeh_bridge_reset(structpci_dev*pdev,intoption)+{+structpci_controller*hose;+structpnv_phb*phb;+structdevice_node*dn=pdev?pci_device_to_OF_node(pdev):NULL;+uint64_tid=(0x1ul<<60);+uint8_tscope;+int64_trc;++/*+*Ifthefirmwarecan'thandleit,wewillissuehotreset+*onthesecondarybusdespitetherequestedresettype.+*/+if(!dn||!of_get_property(dn,"ibm,reset-by-firmware",NULL))+return__pnv_eeh_bridge_reset(pdev,option);++/* The firmware can handle the request */+switch(option){+caseEEH_RESET_HOT:+scope=OPAL_RESET_PCI_HOT;+break;+caseEEH_RESET_FUNDAMENTAL:+scope=OPAL_RESET_PCI_FUNDAMENTAL;+break;+caseEEH_RESET_DEACTIVATE:+return0;+default:+dev_warn(&pdev->dev,"%s: Unsupported reset %d\n",+__func__,option);+return-EINVAL;+}++hose=pci_bus_to_host(pdev->bus);+phb=hose->private_data;+id|=(pdev->bus->number<<24)|(pdev->devfn<<16)|phb->opal_id;+rc=opal_pci_reset(id,scope,OPAL_ASSERT_RESET);+returnpnv_pci_poll(id,rc,NULL);+}+staticintpnv_pci_dev_reset_type(structpci_dev*pdev,void*data){int*freset=data;
This exports 4 functins, which base on the corresponding OPAL
APIs to get/set PCI slot status. Those functions are going to
be used by PowerNV PCI hotplug driver:
pnv_pci_get_device_tree() opal_get_device_tree()
pnv_pci_get_presence_state() opal_pci_get_presence_state()
pnv_pci_get_power_state() opal_pci_get_power_state()
pnv_pci_set_power_state() opal_pci_set_power_state()
Besides, the patch also exports pnv_pci_hotplug_notifier_{register,
unregister}() to allow registration and unregistration of PCI hotplug
notifier, which will be used to receive PCI hotplug message from
skiboot firmware in PowerNV PCI hotplug driver.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/include/asm/opal-api.h | 17 ++++++-
arch/powerpc/include/asm/opal.h | 4 ++
arch/powerpc/include/asm/pnv-pci.h | 7 +++
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 ++
arch/powerpc/platforms/powernv/pci.c | 66 ++++++++++++++++++++++++++
5 files changed, 97 insertions(+), 1 deletion(-)
The device tree will change dynamically in PowerNV PCI hotplug
driver. This enables CONFIG_OF_DYNAMIC to support that.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/Kconfig | 1 +
1 file changed, 1 insertion(+)
The function unflatten_dt_node() is called recursively to unflatten
device nodes and properties in the FDT blob. It looks complicated
and hard to be understood.
This splits the function into 3 functions: populate_properties(),
populate_node() and unflatten_dt_node(). populate_properties(),
which is called by populate_node(), creates properties for the
indicated device node. The later one creates the device nodes
from FDT blob. populate_node() gets the offset in FDT blob for
next device nodes and then calls populate_node(). No logical
changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
drivers/of/fdt.c | 275 ++++++++++++++++++++++++++++++++-----------------------
1 file changed, 160 insertions(+), 115 deletions(-)
@@ -160,39 +160,127 @@ static void *unflatten_dt_alloc(void **mem, unsigned long size,returnres;}-/**-*unflatten_dt_node-Allocandpopulateadevice_nodefromtheflattree-*@blob:Theparentdevicetreeblob-*@mem:Memorychunktouseforallocatingdevicenodesandproperties-*@poffset:pointertonodeinflattree-*@dad:Parentstructdevice_node-*@nodepp:Thedevice_nodetreecreatedbythecall-*@fpsize:Sizeofthenodepathupatthecurrentdepth.-*@dryrun:Iftrue,donotallocatedevicenodesbutstillcalculateneeded-*memorysize-*/-staticvoid*unflatten_dt_node(constvoid*blob,-void*mem,-int*poffset,-structdevice_node*dad,-structdevice_node**nodepp,-unsignedlongfpsize,+staticvoidpopulate_properties(constvoid*blob,+intoffset,+void**mem,+structdevice_node*np,+constchar*nodename,booldryrun){-const__be32*p;+structproperty*pp,**pprev=NULL;+intcur;+boolhas_name=false;++pprev=&np->properties;+cur=fdt_first_property_offset(blob,offset);+while(cur>=0){+const__be32*val;+constchar*pname;+u32sz;++val=fdt_getprop_by_offset(blob,cur,&pname,&sz);+if(!val){+pr_warn("%s: Cannot locate property at 0x%x\n",+__func__,cur);+gotonext;+}++if(!pname){+pr_warn("%s: Cannot find property name at 0x%x\n",+__func__,cur);+gotonext;+}elseif(!strcmp(pname,"name")){+has_name=true;+}++pp=unflatten_dt_alloc(mem,sizeof(structproperty),+__alignof__(structproperty));+if(!dryrun){+/* We accept flattened tree phandles either in+*ePAPR-style"phandle"properties,orthe+*legacy"linux,phandle"properties.Ifboth+*appearandhavedifferentvalues,things+*willgetweird.Don'tdothat.+*/+if(!strcmp(pname,"phandle")||+!strcmp(pname,"linux,phandle")){+if(!np->phandle)+np->phandle=be32_to_cpup(val);+}++/* And we process the "ibm,phandle" property+*usedinpSeriesdynamicdevicetree+*stuff+*/+if(!strcmp(pname,"ibm,phandle"))+np->phandle=be32_to_cpup(val);++pp->name=(char*)pname;+pp->length=sz;+pp->value=(__be32*)val;+*pprev=pp;+pprev=&pp->next;+}+next:+cur=fdt_next_property_offset(blob,cur);+}++/* With version 0x10 we may not have the name property,+*recreateitherefromtheunitnameifabsent+*/+if(!has_name){+constchar*p=nodename,*ps=p,*pa=NULL;+intlen;++while(*p){+if((*p)=='@')+pa=p;+elseif((*p)=='/')+ps=p+1;+p++;+}++if(pa<ps)+pa=p;+len=(pa-ps)+1;+pp=unflatten_dt_alloc(mem,sizeof(structproperty)+len,+__alignof__(structproperty));+if(!dryrun){+pp->name="name";+pp->length=len;+pp->value=pp+1;+*pprev=pp;+pprev=&pp->next;+memcpy(pp->value,ps,len-1);+((char*)pp->value)[len-1]=0;+pr_debug("fixed up name for %s -> %s\n",+nodename,(char*)pp->value);+}+}++if(!dryrun)+*pprev=NULL;+}++staticunsignedlongpopulate_node(constvoid*blob,+intoffset,+void**mem,+structdevice_node*dad,+unsignedlongfpsize,+structdevice_node**pnp,+booldryrun)+{structdevice_node*np;-structproperty*pp,**prev_pp=NULL;constchar*pathp;unsignedintl,allocl;-staticintdepth=0;-intold_depth;-intoffset;-inthas_name=0;-intnew_format=0;+boolnew_format=false;+char*fname;-pathp=fdt_get_name(blob,*poffset,&l);-if(!pathp)-returnmem;+pathp=fdt_get_name(blob,offset,&l);+if(!pathp){+*pnp=NULL;+return0;+}allocl=++l;
@@ -202,7 +290,7 @@ static void * unflatten_dt_node(const void *blob,*not'/'.*/if((*pathp)!='/'){-new_format=1;+new_format=true;if(fpsize==0){/* root node: special case. fpsize accounts for path*plusterminatingzero.rootnodeonlyhas'/',so
@@ -222,112 +310,38 @@ static void * unflatten_dt_node(const void *blob,}}-np=unflatten_dt_alloc(&mem,sizeof(structdevice_node)+allocl,+np=unflatten_dt_alloc(mem,sizeof(structdevice_node)+allocl,__alignof__(structdevice_node));if(!dryrun){-char*fn;of_node_init(np);-np->full_name=fn=((char*)np)+sizeof(*np);+np->full_name=fname=((char*)np)+sizeof(*np);if(new_format){-/* rebuild full path for new format */+/* Rebuild full path for new format */if(dad&&dad->parent){-strcpy(fn,dad->full_name);+strcpy(fname,dad->full_name);#ifdef DEBUG-if((strlen(fn)+l+1)!=allocl){+if((strlen(fname)+l+1)!=allocl){pr_debug("%s: p: %d, l: %d, a: %d\n",-pathp,(int)strlen(fn),-l,allocl);+pathp,(int)strlen(fn),+l,allocl);}#endif-fn+=strlen(fn);+fname+=strlen(fname);}-*(fn++)='/';+*(fname++)='/';}-memcpy(fn,pathp,l);+memcpy(fname,pathp,l);-prev_pp=&np->properties;-if(dad!=NULL){+if(dad){np->parent=dad;np->sibling=dad->child;dad->child=np;}}-/* process properties */-for(offset=fdt_first_property_offset(blob,*poffset);-(offset>=0);-(offset=fdt_next_property_offset(blob,offset))){-constchar*pname;-u32sz;-if(!(p=fdt_getprop_by_offset(blob,offset,&pname,&sz))){-offset=-FDT_ERR_INTERNAL;-break;-}--if(pname==NULL){-pr_info("Can't find property name in list !\n");-break;-}-if(strcmp(pname,"name")==0)-has_name=1;-pp=unflatten_dt_alloc(&mem,sizeof(structproperty),-__alignof__(structproperty));-if(!dryrun){-/* We accept flattened tree phandles either in-*ePAPR-style"phandle"properties,orthe-*legacy"linux,phandle"properties.Ifboth-*appearandhavedifferentvalues,things-*willgetweird.Don'tdothat.*/-if((strcmp(pname,"phandle")==0)||-(strcmp(pname,"linux,phandle")==0)){-if(np->phandle==0)-np->phandle=be32_to_cpup(p);-}-/* And we process the "ibm,phandle" property-*usedinpSeriesdynamicdevicetree-*stuff*/-if(strcmp(pname,"ibm,phandle")==0)-np->phandle=be32_to_cpup(p);-pp->name=(char*)pname;-pp->length=sz;-pp->value=(__be32*)p;-*prev_pp=pp;-prev_pp=&pp->next;-}-}-/* with version 0x10 we may not have the name property, recreate-*itherefromtheunitnameifabsent-*/-if(!has_name){-constchar*p1=pathp,*ps=pathp,*pa=NULL;-intsz;--while(*p1){-if((*p1)=='@')-pa=p1;-if((*p1)=='/')-ps=p1+1;-p1++;-}-if(pa<ps)-pa=p1;-sz=(pa-ps)+1;-pp=unflatten_dt_alloc(&mem,sizeof(structproperty)+sz,-__alignof__(structproperty));-if(!dryrun){-pp->name="name";-pp->length=sz;-pp->value=pp+1;-*prev_pp=pp;-prev_pp=&pp->next;-memcpy(pp->value,ps,sz-1);-((char*)pp->value)[sz-1]=0;-pr_debug("fixed up name for %s -> %s\n",pathp,-(char*)pp->value);-}-}+/* Populate the properties */+populate_properties(blob,offset,mem,np,pathp,dryrun);if(!dryrun){-*prev_pp=NULL;np->name=of_get_property(np,"name",NULL);np->type=of_get_property(np,"device_type",NULL);
In current implementation, unflatten_dt_node() is called recursively
to unflatten device nodes in FDT blob. It's stress to limited stack
capacity.
This avoids calling the function recursively, meaning the device
nodes are unflattened in one call on unflatten_dt_node(): two arrays
are introduced to track the parent path size and the device node of
current level of depth, which will be used by the device node on next
level of depth to be unflattened. Also, the parameter "poffset" and
"fpsize" are unused and dropped.
Signed-off-by: Gavin Shan <redacted>
---
drivers/of/fdt.c | 94 +++++++++++++++++++++++++++++++++-----------------------
1 file changed, 56 insertions(+), 38 deletions(-)
@@ -355,61 +355,82 @@ static unsigned long populate_node(const void *blob,returnfpsize;}+staticvoidreverse_nodes(structdevice_node*parent)+{+structdevice_node*child,*next;++/* In-depth first */+child=parent->child;+while(child){+reverse_nodes(child);++child=child->sibling;+}++/* Reverse the nodes in the child list */+child=parent->child;+parent->child=NULL;+while(child){+next=child->sibling;++child->sibling=parent->child;+parent->child=child;+child=next;+}+}+/***unflatten_dt_node-Allocandpopulateadevice_nodefromtheflattree*@blob:Theparentdevicetreeblob*@mem:Memorychunktouseforallocatingdevicenodesandproperties-*@poffset:pointertonodeinflattree*@dad:Parentstructdevice_node*@nodepp:Thedevice_nodetreecreatedbythecall-*@fpsize:Sizeofthenodepathupatthecurrentdepth.*@dryrun:Iftrue,donotallocatedevicenodesbutstillcalculateneeded*memorysize*/staticvoid*unflatten_dt_node(constvoid*blob,void*mem,-int*poffset,structdevice_node*dad,structdevice_node**nodepp,-unsignedlongfpsize,booldryrun){-structdevice_node*np;-staticintdepth;-intold_depth;--fpsize=populate_node(blob,*poffset,&mem,dad,fpsize,&np,dryrun);-if(!fpsize)-returnmem;+structdevice_node*root;+intoffset=0,depth=0;+unsignedlongfpsizes[64];+structdevice_node*nps[64];-old_depth=depth;-*poffset=fdt_next_node(blob,*poffset,&depth);-if(depth<0)-depth=0;-while(*poffset>0&&depth>old_depth)-mem=unflatten_dt_node(blob,mem,poffset,np,NULL,-fpsize,dryrun);+if(nodepp)+*nodepp=NULL;++root=dad;+fpsizes[depth]=dad?strlen(of_node_full_name(dad)):0;+nps[depth++]=dad;+while(offset>=0&&depth<64){+fpsizes[depth]=populate_node(blob,offset,&mem,+nps[depth-1],+fpsizes[depth-1],+&nps[depth],dryrun);+if(!fpsizes[depth])+returnmem;++if(!dryrun&&nodepp&&!*nodepp)+*nodepp=nps[depth];+if(!dryrun&&!root)+root=nps[depth];++offset=fdt_next_node(blob,offset,&depth);+}-if(*poffset<0&&*poffset!=-FDT_ERR_NOTFOUND)-pr_err("unflatten: error %d processing FDT\n",*poffset);+if(offset<0&&offset!=-FDT_ERR_NOTFOUND)+pr_err("%s: Error %d processing FDT\n",+__func__,offset);/**Reversethechildlist.Somedriversassumesnodeordermatches.dts*nodeorder*/-if(!dryrun&&np->child){-structdevice_node*child=np->child;-np->child=NULL;-while(child){-structdevice_node*next=child->sibling;-child->sibling=np->child;-np->child=child;-child=next;-}-}--if(nodepp)-*nodepp=np;+if(!dryrun)+reverse_nodes(root);returnmem;}
@@ -452,8 +472,7 @@ static void __unflatten_device_tree(const void *blob,}/* First pass, scan for size */-start=0;-size=(unsignedlong)unflatten_dt_node(blob,NULL,&start,NULL,NULL,0,true);+size=(unsignedlong)unflatten_dt_node(blob,NULL,NULL,NULL,true);size=ALIGN(size,4);pr_debug(" size is %lx, allocating...\n",size);
@@ -467,8 +486,7 @@ static void __unflatten_device_tree(const void *blob,pr_debug(" unflattening %p...\n",mem);/* Second pass, do actual unflattening */-start=0;-unflatten_dt_node(blob,mem,&start,NULL,mynodes,0,false);+unflatten_dt_node(blob,mem,NULL,mynodes,false);if(be32_to_cpup(mem+size)!=0xdeadbeef)pr_warning("End of tree marker overwritten: %08x\n",be32_to_cpup(mem+size));
@@ -472,7 +472,7 @@ static void __unflatten_device_tree(const void *blob,}/* First pass, scan for size */-size=(unsignedlong)unflatten_dt_node(blob,NULL,NULL,NULL,true);+size=(unsignedlong)unflatten_dt_nodes(blob,NULL,NULL,NULL,true);size=ALIGN(size,4);pr_debug(" size is %lx, allocating...\n",size);
@@ -486,7 +486,7 @@ static void __unflatten_device_tree(const void *blob,pr_debug(" unflattening %p...\n",mem);/* Second pass, do actual unflattening */-unflatten_dt_node(blob,mem,NULL,mynodes,false);+unflatten_dt_nodes(blob,mem,NULL,mynodes,false);if(be32_to_cpup(mem+size)!=0xdeadbeef)pr_warning("End of tree marker overwritten: %08x\n",be32_to_cpup(mem+size));
This returns the allocate memory chunk, storing the unflattened device
tree, from of_fdt_unflatten_tree() so that memory chunk can be released
on demand in PowerNV PCI hotplug driver.
Signed-off-by: Gavin Shan <redacted>
Acked-by: Rob Herring <robh@kernel.org>
---
drivers/of/fdt.c | 25 ++++++++++++++++---------
include/linux/of_fdt.h | 6 +++---
2 files changed, 19 insertions(+), 12 deletions(-)
This adds standalone driver to support PCI hotplug for PowerPC PowerNV
platform that runs on top of skiboot firmware. The firmware identifies
hotpluggable slots and marked their device tree node with proper
"ibm,slot-pluggable" and "ibm,reset-by-firmware". The driver scans device
tree nodes to create/register PCI hotplug slot accordingly.
If the skiboot firmware doesn't support slot status retrieval, the PCI
slot device node shouldn't have property "ibm,reset-by-firmware". In
that case, none of valid PCI slots will be detected from device tree.
The skiboot firmware doesn't export the capability to access attention
LEDs yet and it's something for TBD.
Signed-off-by: Gavin Shan <redacted>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
---
MAINTAINERS | 6 +
drivers/pci/hotplug/Kconfig | 12 +
drivers/pci/hotplug/Makefile | 3 +
drivers/pci/hotplug/pnv_php.c | 866 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 887 insertions(+)
create mode 100644 drivers/pci/hotplug/pnv_php.c
@@ -0,0 +1,866 @@+/*+*PCIHotplugDriverforPowerPCPowerNVplatform.+*+*CopyrightGavinShan,IBMCorporation2015.+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseaspublishedby+*theFreeSoftwareFoundation;eitherversion2oftheLicense,or+*(atyouroption)anylaterversion.+*/++#include<linux/pci.h>+#include<linux/pci_hotplug.h>+#include<linux/module.h>++#include<asm/opal.h>+#include<asm/pnv-pci.h>+#include<asm/ppc-pci.h>++#define DRIVER_VERSION "0.1"+#define DRIVER_AUTHOR "Gavin Shan, IBM Corporation"+#define DRIVER_DESC "PowerPC PowerNV PCI Hotplug Driver"++structpnv_php_slot{+structhotplug_slotphp_slot;+structhotplug_slot_infophp_slot_info;+uint64_tid;+char*name;+intslot_no;+structkrefkref;+intstate;+#define PNV_PHP_STATE_INIT 0+#define PNV_PHP_STATE_REGISTER 1+#define PNV_PHP_STATE_POPULATED 2+structdevice_node*dn;+structpci_dev*pdev;+structpci_bus*bus;+boolpower_state_check;+intpower_state_confirmed;+#define PNV_PHP_POWER_CONFIRMED_INVALID 0+#define PNV_PHP_POWER_CONFIRMED_SUCCESS 1+#define PNV_PHP_POWER_CONFIRMED_FAIL 2+structopal_msg*msg;+void*fdt;+void*dt;+structof_changesetocs;+structwork_structwork;+wait_queue_head_tqueue;+structpnv_php_slot*parent;+structlist_headchildren;+structlist_headlink;+};++staticLIST_HEAD(pnv_php_slot_list);+staticDEFINE_SPINLOCK(pnv_php_lock);++staticvoidpnv_php_register(structdevice_node*dn);+staticvoidpnv_php_unregister_one(structdevice_node*dn);+staticvoidpnv_php_unregister(structdevice_node*dn);++staticinlinestructpnv_php_slot*pnv_php_get_slot(structpnv_php_slot*slot)+{+if(slot){+kref_get(&slot->kref);+returnslot;+}++returnNULL;+}++staticvoidpnv_php_free_slot(structkref*kref)+{+structpnv_php_slot*slot=container_of(kref,+structpnv_php_slot,+kref);++WARN_ON(!list_empty(&slot->children));+kfree(slot->name);+kfree(slot);+}++staticinlinevoidpnv_php_put_slot(structpnv_php_slot*slot)+{+if(!slot)+return;++kref_put(&slot->kref,pnv_php_free_slot);+}++staticstructpnv_php_slot*pnv_php_match(structdevice_node*dn,+structpnv_php_slot*slot)+{+structpnv_php_slot*target,*tmp;++if(slot->dn==dn)+returnpnv_php_get_slot(slot);++list_for_each_entry(tmp,&slot->children,link){+target=pnv_php_match(dn,tmp);+if(target)+returntarget;+}++returnNULL;+}++staticstructpnv_php_slot*pnv_php_find_slot(structdevice_node*dn)+{+structpnv_php_slot*slot,*tmp;+unsignedlongflags;++spin_lock_irqsave(&pnv_php_lock,flags);+list_for_each_entry(tmp,&pnv_php_slot_list,link){+slot=pnv_php_match(dn,tmp);+if(slot){+spin_unlock_irqrestore(&pnv_php_lock,flags);+returnslot;+}+}+spin_unlock_irqrestore(&pnv_php_lock,flags);++returnNULL;+}++/*+*Removepdnforallchildrenoftheindicateddevicenode.+*Thefunctionshouldremovepdninadepth-firstmanner.+*/+staticvoidpnv_php_rmv_pdns(structdevice_node*dn)+{+structdevice_node*child;++for_each_child_of_node(dn,child){+pnv_php_rmv_pdns(child);++pci_remove_device_node_info(child);+}+}++/*+*Removeallchildnodesoftheindicateddevicenodes.The+*functionshouldremovedevicenodesindepth-firstmanner.+*/+staticintpnv_php_rmv_device_nodes(structdevice_node*parent)+{+structdevice_node*dn,*child;+intret=0;++for_each_child_of_node(parent,dn){+ret=pnv_php_rmv_device_nodes(dn);+if(ret)+returnret;++child=of_get_next_child(dn,NULL);+if(child){+of_node_put(child);+of_node_put(dn);+pr_err("%s: Alive children of node <%s>\n",+__func__,of_node_full_name(dn));+return-EBUSY;+}++of_detach_node(dn);+of_node_put(dn);+}++return0;+}++/*+*Thefunctionprocessesthemessagesentbyfirmware+*toremovealldevicetreenodesbeneaththeslot's+*nodesandtheassociatedauxiliarydata.+*/+staticvoidpnv_php_handle_poweroff(structpnv_php_slot*slot)+{+intret;++pnv_php_rmv_pdns(slot->dn);++/*+*Ifthedevicesub-treewascreatedfromOFchangeset,simply+*torevertthat.Otherwise,thedevicenodesinthesub-tree+*needtobeiteratedanddetached.+*/+if(slot->fdt){+of_changeset_destroy(&slot->ocs);+kfree(slot->dt);+kfree(slot->fdt);+slot->dt=NULL;+slot->dn->child=NULL;+slot->fdt=NULL;+slot->power_state_confirmed=PNV_PHP_POWER_CONFIRMED_SUCCESS;+gotoconfirm;+}++ret=pnv_php_rmv_device_nodes(slot->dn);+if(!ret){+slot->power_state_confirmed=PNV_PHP_POWER_CONFIRMED_SUCCESS;+}else{+slot->power_state_confirmed=PNV_PHP_POWER_CONFIRMED_FAIL;+dev_warn(&slot->pdev->dev,"Error %d freeing nodes\n",+ret);+}++confirm:+wake_up_interruptible(&slot->queue);+}++staticintpnv_php_populate_changeset(structof_changeset*ocs,+structdevice_node*dn)+{+structdevice_node*child;+intret=0;++for_each_child_of_node(dn,child){+ret=of_changeset_attach_node(ocs,child);+if(ret)+returnret;++ret=pnv_php_populate_changeset(ocs,child);+}++returnret;+}++staticvoid*pnv_php_add_one_pdn(structdevice_node*dn,void*data)+{+structpci_controller*hose=(structpci_controller*)data;+structpci_dn*pdn;++pdn=pci_add_device_node_info(hose,dn);+if(!pdn)+returnERR_PTR(-ENOMEM);++returnNULL;+}++staticvoidpnv_php_add_pdns(structpnv_php_slot*slot)+{+structpci_controller*hose=pci_bus_to_host(slot->bus);++pci_traverse_device_nodes(slot->dn,pnv_php_add_one_pdn,hose);+}++staticvoidpnv_php_handle_poweron(structpnv_php_slot*slot)+{+void*fdt,*dt;+uint64_tlen;+intconfirm=PNV_PHP_POWER_CONFIRMED_SUCCESS;+intret;++/* We don't know the FDT blob size. It tries with incremental+*sizedmemorychunk.+*/+for(len=0x2000;len<=0x10000;len+=0x2000){+fdt=kzalloc(len,GFP_KERNEL);+if(!fdt)+break;++ret=pnv_pci_get_device_tree(slot->dn->phandle,fdt,len);+if(!ret)+break;++kfree(fdt);+}++if(len>0x10000){+dev_warn(&slot->pdev->dev,"Cannot alloc FDT blob\n");+gotoout;+}++/* Unflatten device tree blob */+dt=of_fdt_unflatten_tree(fdt,slot->dn,NULL);+if(!dt){+dev_warn(&slot->pdev->dev,"Cannot unflatten FDT\n");+gotofree_fdt;+}++/* Initialize and apply the changeset */+of_changeset_init(&slot->ocs);+ret=pnv_php_populate_changeset(&slot->ocs,slot->dn);+if(ret){+dev_warn(&slot->pdev->dev,"Error %d populating changeset\n",+ret);+gotofree_dt;+}++slot->dn->child=NULL;+ret=of_changeset_apply(&slot->ocs);+if(ret){+dev_warn(&slot->pdev->dev,"Error %d applying changeset\n",+ret);+gotodestroy_changeset;+}++/* Add device node firmware data */+pnv_php_add_pdns(slot);+slot->fdt=fdt;+slot->dt=dt;+gotoout;++destroy_changeset:+of_changeset_destroy(&slot->ocs);+free_dt:+kfree(dt);+slot->dn->child=NULL;+free_fdt:+kfree(fdt);+confirm=PNV_PHP_POWER_CONFIRMED_FAIL;+out:+/* Confirm status change */+slot->power_state_confirmed=confirm;+wake_up_interruptible(&slot->queue);+}++staticvoidpnv_php_work(structwork_struct*data)+{+structpnv_php_slot*slot=container_of(data,+structpnv_php_slot,work);+uint64_tevent=be64_to_cpu(slot->msg->params[0]);++if(event==OPAL_PCI_SLOT_POWER_OFF)+pnv_php_handle_poweroff(slot);+else+pnv_php_handle_poweron(slot);++pnv_php_put_slot(slot);+}++staticintpnv_php_handle_msg(structnotifier_block*nb,+unsignedlongtype,+void*message)+{+phandleh;+structdevice_node*dn;+structpnv_php_slot*slot;+structopal_msg*msg=message;++if(type!=OPAL_MSG_PCI_HOTPLUG){+pr_warn("%s: Invalid message %ld received!\n",+__func__,type);+returnNOTIFY_DONE;+}++h=(phandle)be64_to_cpu(msg->params[1]);+dn=of_find_node_by_phandle(h);+if(!dn){+pr_warn("%s: No device node for phandle 0x%x\n",+__func__,h);+returnNOTIFY_DONE;+}++slot=pnv_php_find_slot(dn);+of_node_put(dn);+if(!slot){+pr_warn("%s: No slot found for node <%s>\n",+__func__,of_node_full_name(dn));+of_node_put(dn);+returnNOTIFY_DONE;+}++slot->msg=msg;+schedule_work(&slot->work);+returnNOTIFY_OK;+}++staticintpnv_php_set_power_state(structhotplug_slot*php_slot,u8state)+{+structpnv_php_slot*slot=php_slot->private;+intret;++slot->power_state_confirmed=PNV_PHP_POWER_CONFIRMED_INVALID;+ret=pnv_pci_set_power_state(slot->id,state);+if(ret){+dev_warn(&slot->pdev->dev,"Error %d powering %s slot\n",+ret,state?"on":"off");+returnret;+}++/* Continue to PCI probing after finalized device-tree. The+*device-treemighthavebeenupdatedcompletelyatthis+*point.Thuswedon'thavetoalwayswaitingforthat.+*/+if(slot->power_state_confirmed==PNV_PHP_POWER_CONFIRMED_SUCCESS)+return0;+elseif(slot->power_state_confirmed==PNV_PHP_POWER_CONFIRMED_FAIL)+return-EBUSY;++ret=wait_event_timeout(slot->queue,+slot->power_state_confirmed,10*HZ);+if(!ret){+dev_warn(&slot->pdev->dev,"Error %d waiting for power-%s\n",+ret,state?"on":"off");+return-EBUSY;+}++if(slot->power_state_confirmed==PNV_PHP_POWER_CONFIRMED_SUCCESS)+return0;++dev_warn(&slot->pdev->dev,"Error status %d for power-%s\n",+slot->power_state_confirmed,state?"on":"off");+return-EBUSY;+}++staticintpnv_php_get_power_state(structhotplug_slot*php_slot,u8*state)+{+structpnv_php_slot*slot=php_slot->private;+uint8_tpower_state;+intret;++/*+*Retrievepowerstatusfromfirmware.Ifwefail+*gettingthat,thepowerstatusfailsbackto+*beon.+*/+ret=pnv_pci_get_power_state(slot->id,&power_state);+if(ret){+*state=OPAL_PCI_SLOT_POWER_ON;+dev_warn(&slot->pdev->dev,"Error %d getting power status\n",+ret);+}else{+*state=power_state;+php_slot->info->power_status=power_state;+}++return0;+}++staticintpnv_php_get_adapter_state(structhotplug_slot*php_slot,u8*state)+{+structpnv_php_slot*slot=php_slot->private;+uint8_tpresence;+intret;++/*+*Retrievepresencestatusfromfirmware.Ifwecan't+*getthat,itwillfailbacktobeempty.+*/+ret=pnv_pci_get_presence_state(slot->id,&presence);+if(ret>=0){+*state=presence;+php_slot->info->adapter_status=presence;+ret=0;+}else{+*state=OPAL_PCI_SLOT_EMPTY;+dev_warn(&slot->pdev->dev,"Error %d getting presence\n",+ret);+}++returnret;+}++staticintpnv_php_set_attention_state(structhotplug_slot*php_slot,u8state)+{+/* FIXME: Make it real once firmware supports it */+php_slot->info->attention_status=state;++return0;+}++staticintpnv_php_enable(structpnv_php_slot*slot,boolrescan)+{+structhotplug_slot*php_slot=&slot->php_slot;+uint8_tpresence,power_status;+intret;++/* Check if the slot has been configured */+if(slot->state!=PNV_PHP_STATE_REGISTER)+return0;++/* Retrieve slot presence status */+ret=php_slot->ops->get_adapter_status(php_slot,&presence);+if(ret)+returnret;++/* Proceed if there have nothing behind the slot */+if(presence==OPAL_PCI_SLOT_EMPTY)+gotoscan;++/*+*Ifwedon'tdetectsomethingbehindtheslot,weneed+*makesurethepowersuplytotheslotison.Otherwise,+*theslotdownstreamPCIelinkturnshouldbedown.+*+*Onthefirsttime,wedon'tchangethepowerstatusto+*boostsystembootwithassumptionthatthefirmware+*suppliesconsistentslotpowerstatus:emptyslotalways+*hasitspoweroffandnon-emptyslothasitspoweron.+*/+if(!slot->power_state_check){+slot->power_state_check=true;+gotoscan;+}++/* Check the power status. Scan the slot if that's already on */+ret=php_slot->ops->get_power_status(php_slot,&power_status);+if(ret)+returnret;++if(power_status==OPAL_PCI_SLOT_POWER_ON)+gotoscan;++/* Power is off, turn it on and then scan the slot */+ret=pnv_php_set_power_state(php_slot,OPAL_PCI_SLOT_POWER_ON);+if(ret)+returnret;++scan:+if(presence==OPAL_PCI_SLOT_PRESENT){+if(rescan){+pci_lock_rescan_remove();+pci_add_pci_devices(slot->bus);+pci_unlock_rescan_remove();+}++/* Rescan for child hotpluggable slots */+slot->state=PNV_PHP_STATE_POPULATED;+if(rescan)+pnv_php_register(slot->dn);+}else{+slot->state=PNV_PHP_STATE_POPULATED;+}++return0;+}++staticintpnv_php_enable_slot(structhotplug_slot*php_slot)+{+structpnv_php_slot*slot=container_of(php_slot,+structpnv_php_slot,+php_slot);++returnpnv_php_enable(slot,true);+}++staticintpnv_php_disable_slot(structhotplug_slot*php_slot)+{+structpnv_php_slot*slot=php_slot->private;+uint8_tpower_state;+intret;++if(slot->state!=PNV_PHP_STATE_POPULATED)+return0;++/* Remove all devices behind the slot */+pci_lock_rescan_remove();+pci_remove_pci_devices(slot->bus);+pci_unlock_rescan_remove();++/* Detach the child hotpluggable slots */+pnv_php_unregister(slot->dn);++/*+*Checkthepowerstatusandturnitoffifnecessary.Ifwe+*failtogetthepowerstatus,thepowerwillbeforcedto+*beoff.+*/+ret=php_slot->ops->get_power_status(php_slot,&power_state);+if(ret||power_state==OPAL_PCI_SLOT_POWER_ON){+ret=pnv_php_set_power_state(php_slot,+OPAL_PCI_SLOT_POWER_OFF);+if(ret)+dev_warn(&slot->pdev->dev,"Error %d powering off\n",+ret);+}++/* Update slot state */+slot->state=PNV_PHP_STATE_REGISTER;+return0;+}++staticstructhotplug_slot_opsphp_slot_ops={+.get_power_status=pnv_php_get_power_state,+.get_adapter_status=pnv_php_get_adapter_state,+.set_attention_status=pnv_php_set_attention_state,+.enable_slot=pnv_php_enable_slot,+.disable_slot=pnv_php_disable_slot,+};++staticvoidpnv_php_release(structhotplug_slot*hp_slot)+{+structpnv_php_slot*slot=hp_slot->private;+unsignedlongflags;++/* Remove from global or child list */+spin_lock_irqsave(&pnv_php_lock,flags);+list_del(&slot->link);+spin_unlock_irqrestore(&pnv_php_lock,flags);++/* Detach from parent */+pnv_php_put_slot(slot);+pnv_php_put_slot(slot->parent);+}++staticintpnv_php_get_slot_id(structdevice_node*dn,uint64_t*id)+{+structdevice_node*parent=dn;+const__be64*prop64;+const__be32*prop32;++/*+*ThehotpluggableslotalwayshasacompoundId,which+*consistsof16-bitsPHBId,16bitsbus/slot/function+*number,andcompoundindicator+*/+*id=(0x1ul<<63);++/* Bus/Slot/Function number */+prop32=of_get_property(dn,"reg",NULL);+if(!prop32)+return-ENXIO;+*id|=((of_read_number(prop32,1)&0x00ffff00)<<8);++/* PHB Id */+while((parent=of_get_parent(parent))){+if(!PCI_DN(parent)){+of_node_put(parent);+break;+}++if(!of_device_is_compatible(parent,"ibm,ioda2-phb")&&+!of_device_is_compatible(parent,"ibm,ioda-phb")){+of_node_put(parent);+continue;+}++prop64=of_get_property(parent,"ibm,opal-phbid",NULL);+if(!prop64){+of_node_put(parent);+return-ENXIO;+}++*id|=be64_to_cpup(prop64);+of_node_put(parent);+return0;+}++return-ENODEV;+}++staticstructpnv_php_slot*pnv_php_alloc_slot(structdevice_node*dn)+{+structpnv_php_slot*slot;+structpci_bus*bus;+constchar*label;+uint64_tid;++label=of_get_property(dn,"ibm,slot-label",NULL);+if(!label)+returnNULL;++if(pnv_php_get_slot_id(dn,&id))+returnNULL;++bus=pci_find_bus_by_node(dn);+if(!bus)+returnNULL;++slot=kzalloc(sizeof(*slot),GFP_KERNEL);+if(!slot)+returnNULL;++slot->name=kstrdup(label,GFP_KERNEL);+if(!slot->name){+kfree(slot);+returnNULL;+}++if(dn->child&&PCI_DN(dn->child))+slot->slot_no=PCI_SLOT(PCI_DN(dn->child)->devfn);+else+slot->slot_no=-1;/* Placeholder slot */++kref_init(&slot->kref);+slot->state=PNV_PHP_STATE_INIT;+slot->dn=dn;+slot->pdev=bus->self;+slot->bus=bus;+slot->id=id;+slot->power_state_check=false;+slot->power_state_confirmed=PNV_PHP_POWER_CONFIRMED_INVALID;+slot->php_slot.ops=&php_slot_ops;+slot->php_slot.info=&slot->php_slot_info;+slot->php_slot.release=pnv_php_release;+slot->php_slot.private=slot;++INIT_WORK(&slot->work,pnv_php_work);+init_waitqueue_head(&slot->queue);+INIT_LIST_HEAD(&slot->children);+INIT_LIST_HEAD(&slot->link);++returnslot;+}++staticintpnv_php_register_slot(structpnv_php_slot*slot)+{+structpnv_php_slot*parent;+structdevice_node*dn=slot->dn;+unsignedlongflags;+intret;++/* Check if the slot exists or not */+parent=pnv_php_find_slot(slot->dn);+if(parent){+pnv_php_put_slot(parent);+return-EEXIST;+}++/* Register PCI slot */+ret=pci_hp_register(&slot->php_slot,slot->bus,+slot->slot_no,slot->name);+if(ret){+dev_warn(&slot->pdev->dev,"Error %d registering slot\n",+ret);+returnret;+}++/* Attach to the parent's child list or global list */+while((dn=of_get_parent(dn))){+if(!PCI_DN(dn)){+of_node_put(dn);+break;+}++parent=pnv_php_find_slot(dn);+if(parent){+of_node_put(dn);+break;+}+}++spin_lock_irqsave(&pnv_php_lock,flags);+slot->parent=parent;+if(parent)+list_add_tail(&slot->link,&parent->children);+else+list_add_tail(&slot->link,&pnv_php_slot_list);+spin_unlock_irqrestore(&pnv_php_lock,flags);++slot->state=PNV_PHP_STATE_REGISTER;+return0;+}++staticintpnv_php_register_one(structdevice_node*dn)+{+structpnv_php_slot*slot;+const__be32*prop32;+intret;++/* Check if it's hotpluggable slot */+prop32=of_get_property(dn,"ibm,slot-pluggable",NULL);+if(!prop32||!of_read_number(prop32,1))+return-ENXIO;++prop32=of_get_property(dn,"ibm,reset-by-firmware",NULL);+if(!prop32||!of_read_number(prop32,1))+return-ENXIO;++slot=pnv_php_alloc_slot(dn);+if(!slot)+return-ENODEV;++ret=pnv_php_register_slot(slot);+if(ret)+gotofree_slot;++ret=pnv_php_enable(slot,false);+if(ret)+gotounregister_slot;++return0;++unregister_slot:+pnv_php_unregister_one(slot->dn);+free_slot:+pnv_php_put_slot(slot);+returnret;+}++staticvoidpnv_php_register(structdevice_node*dn)+{+structdevice_node*child;++/*+*Theparentslotsshouldberegisteredbeforetheir+*childslots.+*/+for_each_child_of_node(dn,child){+pnv_php_register_one(child);+pnv_php_register(child);+}+}++staticvoidpnv_php_unregister_one(structdevice_node*dn)+{+structpnv_php_slot*slot;++slot=pnv_php_find_slot(dn);+if(!slot)+return;++pnv_php_put_slot(slot);+pci_hp_deregister(&slot->php_slot);+}++staticvoidpnv_php_unregister(structdevice_node*dn)+{+structdevice_node*child;++/* The child slots should go before their parent slots */+for_each_child_of_node(dn,child){+pnv_php_unregister(child);+pnv_php_unregister_one(child);+}+}++staticstructnotifier_blockphp_msg_nb={+.notifier_call=pnv_php_handle_msg,+.next=NULL,+.priority=0,+};++staticint__initpnv_php_init(void)+{+structdevice_node*dn;+intret;++pr_info(DRIVER_DESC" version: "DRIVER_VERSION"\n");++/* Register hotplug message handler */+ret=pnv_pci_hotplug_notifier_register(&php_msg_nb);+if(ret){+pr_warn("%s: Error %d registering hotplug notifier\n",+__func__,ret);+returnret;+}++/* Scan PHB nodes and their children */+for_each_compatible_node(dn,NULL,"ibm,ioda-phb")+pnv_php_register(dn);+for_each_compatible_node(dn,NULL,"ibm,ioda2-phb")+pnv_php_register(dn);++return0;+}++staticvoid__exitpnv_php_exit(void)+{+structdevice_node*dn;++for_each_compatible_node(dn,NULL,"ibm,ioda-phb")+pnv_php_unregister(dn);+for_each_compatible_node(dn,NULL,"ibm,ioda2-phb")+pnv_php_unregister(dn);++pnv_pci_hotplug_notifier_unregister(&php_msg_nb);+}++module_init(pnv_php_init);+module_exit(pnv_php_exit);++MODULE_VERSION(DRIVER_VERSION);+MODULE_LICENSE("GPL v2");+MODULE_AUTHOR(DRIVER_AUTHOR);+MODULE_DESCRIPTION(DRIVER_DESC);
From: Rob Herring <hidden> Date: 2015-11-04 16:07:50
On Wed, Nov 4, 2015 at 7:12 AM, Gavin Shan [off-list ref] wrote:
In current implementation, unflatten_dt_node() is called recursively
to unflatten device nodes in FDT blob. It's stress to limited stack
capacity.
Did you actually hit a problem?
Now we have a max depth of 64. Seems like that should be plenty... Any
idea how this compares to when we run out of stack space?
This avoids calling the function recursively, meaning the device
nodes are unflattened in one call on unflatten_dt_node(): two arrays
are introduced to track the parent path size and the device node of
current level of depth, which will be used by the device node on next
level of depth to be unflattened. Also, the parameter "poffset" and
"fpsize" are unused and dropped.
Yay. I'm happy to see parameters removed instead of added to this function.
@@ -355,61 +355,82 @@ static unsigned long populate_node(const void *blob,returnfpsize;}+staticvoidreverse_nodes(structdevice_node*parent)+{+structdevice_node*child,*next;++/* In-depth first */+child=parent->child;+while(child){+reverse_nodes(child);++child=child->sibling;+}++/* Reverse the nodes in the child list */+child=parent->child;+parent->child=NULL;+while(child){+next=child->sibling;++child->sibling=parent->child;+parent->child=child;+child=next;+}+}+/***unflatten_dt_node-Allocandpopulateadevice_nodefromtheflattree*@blob:Theparentdevicetreeblob*@mem:Memorychunktouseforallocatingdevicenodesandproperties-*@poffset:pointertonodeinflattree*@dad:Parentstructdevice_node*@nodepp:Thedevice_nodetreecreatedbythecall-*@fpsize:Sizeofthenodepathupatthecurrentdepth.*@dryrun:Iftrue,donotallocatedevicenodesbutstillcalculateneeded*memorysize*/staticvoid*unflatten_dt_node(constvoid*blob,void*mem,-int*poffset,structdevice_node*dad,structdevice_node**nodepp,-unsignedlongfpsize,booldryrun)
We can probably further simplify things by returning an int with
negative being errors and positive being the size. Also, dryrun can be
dropped and implied by mem and/or nodepp being NULL.
{
- struct device_node *np;
- static int depth;
- int old_depth;
-
- fpsize = populate_node(blob, *poffset, &mem, dad, fpsize, &np, dryrun);
- if (!fpsize)
- return mem;
+ struct device_node *root;
+ int offset = 0, depth = 0;
+ unsigned long fpsizes[64];
+ struct device_node *nps[64];
What about depth == 64 case? I think the behavior should be a WARN and
ignore those nodes so we at least can continue to boot and see the
error. Of course, if there is a phandle pointing to ignored nodes, we
have to handle that too.
quoted hunk
/*
* Reverse the child list. Some drivers assumes node order matches .dts
* node order
*/
- if (!dryrun && np->child) {
- struct device_node *child = np->child;
- np->child = NULL;
- while (child) {
- struct device_node *next = child->sibling;
- child->sibling = np->child;
- np->child = child;
- child = next;
- }
- }
-
- if (nodepp)
- *nodepp = np;
+ if (!dryrun)
+ reverse_nodes(root);
return mem;
}
From: Rob Herring <hidden> Date: 2015-11-04 18:43:08
On Wed, Nov 4, 2015 at 7:12 AM, Gavin Shan [off-list ref] wrote:
quoted hunk
The function unflatten_dt_node() is called recursively to unflatten
device nodes and properties in the FDT blob. It looks complicated
and hard to be understood.
This splits the function into 3 functions: populate_properties(),
populate_node() and unflatten_dt_node(). populate_properties(),
which is called by populate_node(), creates properties for the
indicated device node. The later one creates the device nodes
from FDT blob. populate_node() gets the offset in FDT blob for
next device nodes and then calls populate_node(). No logical
changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
drivers/of/fdt.c | 275 ++++++++++++++++++++++++++++++++-----------------------
1 file changed, 160 insertions(+), 115 deletions(-)
This could be better written as a for loop to avoid the gotos:
for (cur = fdt_first_property_offset(blob, offset); cur >=0; cur =
fdt_next_property_offset(blob, cur))
If you kept "fn" that would cut down the diff and make it a bit easier
to review.
if (new_format) {
- /* rebuild full path for new format */
+ /* Rebuild full path for new format */
if (dad && dad->parent) {
- strcpy(fn, dad->full_name);
+ strcpy(fname, dad->full_name);
#ifdef DEBUG
- if ((strlen(fn) + l + 1) != allocl) {
+ if ((strlen(fname) + l + 1) != allocl) {
pr_debug("%s: p: %d, l: %d, a: %d\n",
- pathp, (int)strlen(fn),
- l, allocl);
+ pathp, (int)strlen(fn),
This won't compile if enabled (should be fname).
+ l, allocl);
}
#endif
- fn += strlen(fn);
+ fname += strlen(fname);
}
- *(fn++) = '/';
+ *(fname++) = '/';
}
- memcpy(fn, pathp, l);
+ memcpy(fname, pathp, l);
- prev_pp = &np->properties;
- if (dad != NULL) {
+ if (dad) {
np->parent = dad;
np->sibling = dad->child;
dad->child = np;
}
}
- /* process properties */
- for (offset = fdt_first_property_offset(blob, *poffset);
- (offset >= 0);
- (offset = fdt_next_property_offset(blob, offset))) {
- const char *pname;
- u32 sz;
- if (!(p = fdt_getprop_by_offset(blob, offset, &pname, &sz))) {
- offset = -FDT_ERR_INTERNAL;
- break;
- }
-
- if (pname == NULL) {
- pr_info("Can't find property name in list !\n");
- break;
- }
- if (strcmp(pname, "name") == 0)
- has_name = 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property),
- __alignof__(struct property));
- if (!dryrun) {
- /* We accept flattened tree phandles either in
- * ePAPR-style "phandle" properties, or the
- * legacy "linux,phandle" properties. If both
- * appear and have different values, things
- * will get weird. Don't do that. */
- if ((strcmp(pname, "phandle") == 0) ||
- (strcmp(pname, "linux,phandle") == 0)) {
- if (np->phandle == 0)
- np->phandle = be32_to_cpup(p);
- }
- /* And we process the "ibm,phandle" property
- * used in pSeries dynamic device tree
- * stuff */
- if (strcmp(pname, "ibm,phandle") == 0)
- np->phandle = be32_to_cpup(p);
- pp->name = (char *)pname;
- pp->length = sz;
- pp->value = (__be32 *)p;
- *prev_pp = pp;
- prev_pp = &pp->next;
- }
- }
- /* with version 0x10 we may not have the name property, recreate
- * it here from the unit name if absent
- */
- if (!has_name) {
- const char *p1 = pathp, *ps = pathp, *pa = NULL;
- int sz;
-
- while (*p1) {
- if ((*p1) == '@')
- pa = p1;
- if ((*p1) == '/')
- ps = p1 + 1;
- p1++;
- }
- if (pa < ps)
- pa = p1;
- sz = (pa - ps) + 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property) + sz,
- __alignof__(struct property));
- if (!dryrun) {
- pp->name = "name";
- pp->length = sz;
- pp->value = pp + 1;
- *prev_pp = pp;
- prev_pp = &pp->next;
- memcpy(pp->value, ps, sz - 1);
- ((char *)pp->value)[sz - 1] = 0;
- pr_debug("fixed up name for %s -> %s\n", pathp,
- (char *)pp->value);
- }
- }
+ /* Populate the properties */
@@ -337,6 +351,37 @@ static void * unflatten_dt_node(const void *blob, np->type = "<NULL>"; }+ *pnp = np;+ return fpsize;+}++/**+ * unflatten_dt_node - Alloc and populate a device_node from the flat tree+ * @blob: The parent device tree blob+ * @mem: Memory chunk to use for allocating device nodes and properties+ * @poffset: pointer to node in flat tree+ * @dad: Parent struct device_node+ * @nodepp: The device_node tree created by the call+ * @fpsize: Size of the node path up at the current depth.+ * @dryrun: If true, do not allocate device nodes but still calculate needed+ * memory size+ */+static void *unflatten_dt_node(const void *blob,+ void *mem,+ int *poffset,+ struct device_node *dad,+ struct device_node **nodepp,+ unsigned long fpsize,+ bool dryrun)+{+ struct device_node *np;+ static int depth;+ int old_depth;++ fpsize = populate_node(blob, *poffset, &mem, dad, fpsize, &np, dryrun);
Doesn't this give a warning assigning a ptr to long?
Looks like np can be moved internal to populate_node.
On Wed, Nov 04, 2015 at 12:43:08PM -0600, Rob Herring wrote:
On Wed, Nov 4, 2015 at 7:12 AM, Gavin Shan [off-list ref] wrote:
quoted
The function unflatten_dt_node() is called recursively to unflatten
device nodes and properties in the FDT blob. It looks complicated
and hard to be understood.
This splits the function into 3 functions: populate_properties(),
populate_node() and unflatten_dt_node(). populate_properties(),
which is called by populate_node(), creates properties for the
indicated device node. The later one creates the device nodes
from FDT blob. populate_node() gets the offset in FDT blob for
next device nodes and then calls populate_node(). No logical
changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
drivers/of/fdt.c | 275 ++++++++++++++++++++++++++++++++-----------------------
1 file changed, 160 insertions(+), 115 deletions(-)
I'd like to make dryrun implicit. It is basically a function of NULL
or near NULL pointers.
[1] The condition would be something like below:
if ((unsigned long)(*mem) < limit)
dryrun = true;
else
dryrun = false;
The question here is how to choose a sane @limit in practice. In !dryrun case,
the memory is allocated from memblock at system booting time, or slab when system
is up. memblock could assign memory in bottom-up fasion.
When @limit is too small, the condition wouldn't be comprehensive in !dryrun
case. When @limit is too large, the !dryrun cases can be regarded as dryrun case.
This could be better written as a for loop to avoid the gotos:
for (cur = fdt_first_property_offset(blob, offset); cur >=0; cur =
fdt_next_property_offset(blob, cur))
Thanks, the changes will be included in next revision.
If you kept "fn" that would cut down the diff and make it a bit easier
to review.
Agree, I'll drop the rename in next revision. I perhaps have separate
patch to do the renaming after this patch. In that way, unrelated code
changes will be avoided in this one.
quoted
if (new_format) {
- /* rebuild full path for new format */
+ /* Rebuild full path for new format */
if (dad && dad->parent) {
- strcpy(fn, dad->full_name);
+ strcpy(fname, dad->full_name);
#ifdef DEBUG
- if ((strlen(fn) + l + 1) != allocl) {
+ if ((strlen(fname) + l + 1) != allocl) {
pr_debug("%s: p: %d, l: %d, a: %d\n",
- pathp, (int)strlen(fn),
- l, allocl);
+ pathp, (int)strlen(fn),
This won't compile if enabled (should be fname).
Indeed, I even didn't try to compile this piece of debugging code. Will change
accordingly in next revision.
quoted
+ l, allocl);
}
#endif
- fn += strlen(fn);
+ fname += strlen(fname);
}
- *(fn++) = '/';
+ *(fname++) = '/';
}
- memcpy(fn, pathp, l);
+ memcpy(fname, pathp, l);
- prev_pp = &np->properties;
- if (dad != NULL) {
+ if (dad) {
np->parent = dad;
np->sibling = dad->child;
dad->child = np;
}
}
- /* process properties */
- for (offset = fdt_first_property_offset(blob, *poffset);
- (offset >= 0);
- (offset = fdt_next_property_offset(blob, offset))) {
- const char *pname;
- u32 sz;
- if (!(p = fdt_getprop_by_offset(blob, offset, &pname, &sz))) {
- offset = -FDT_ERR_INTERNAL;
- break;
- }
-
- if (pname == NULL) {
- pr_info("Can't find property name in list !\n");
- break;
- }
- if (strcmp(pname, "name") == 0)
- has_name = 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property),
- __alignof__(struct property));
- if (!dryrun) {
- /* We accept flattened tree phandles either in
- * ePAPR-style "phandle" properties, or the
- * legacy "linux,phandle" properties. If both
- * appear and have different values, things
- * will get weird. Don't do that. */
- if ((strcmp(pname, "phandle") == 0) ||
- (strcmp(pname, "linux,phandle") == 0)) {
- if (np->phandle == 0)
- np->phandle = be32_to_cpup(p);
- }
- /* And we process the "ibm,phandle" property
- * used in pSeries dynamic device tree
- * stuff */
- if (strcmp(pname, "ibm,phandle") == 0)
- np->phandle = be32_to_cpup(p);
- pp->name = (char *)pname;
- pp->length = sz;
- pp->value = (__be32 *)p;
- *prev_pp = pp;
- prev_pp = &pp->next;
- }
- }
- /* with version 0x10 we may not have the name property, recreate
- * it here from the unit name if absent
- */
- if (!has_name) {
- const char *p1 = pathp, *ps = pathp, *pa = NULL;
- int sz;
-
- while (*p1) {
- if ((*p1) == '@')
- pa = p1;
- if ((*p1) == '/')
- ps = p1 + 1;
- p1++;
- }
- if (pa < ps)
- pa = p1;
- sz = (pa - ps) + 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property) + sz,
- __alignof__(struct property));
- if (!dryrun) {
- pp->name = "name";
- pp->length = sz;
- pp->value = pp + 1;
- *prev_pp = pp;
- prev_pp = &pp->next;
- memcpy(pp->value, ps, sz - 1);
- ((char *)pp->value)[sz - 1] = 0;
- pr_debug("fixed up name for %s -> %s\n", pathp,
- (char *)pp->value);
- }
- }
+ /* Populate the properties */
@@ -337,6 +351,37 @@ static void * unflatten_dt_node(const void *blob, np->type = "<NULL>"; }+ *pnp = np;+ return fpsize;+}++/**+ * unflatten_dt_node - Alloc and populate a device_node from the flat tree+ * @blob: The parent device tree blob+ * @mem: Memory chunk to use for allocating device nodes and properties+ * @poffset: pointer to node in flat tree+ * @dad: Parent struct device_node+ * @nodepp: The device_node tree created by the call+ * @fpsize: Size of the node path up at the current depth.+ * @dryrun: If true, do not allocate device nodes but still calculate needed+ * memory size+ */+static void *unflatten_dt_node(const void *blob,+ void *mem,+ int *poffset,+ struct device_node *dad,+ struct device_node **nodepp,+ unsigned long fpsize,+ bool dryrun)+{+ struct device_node *np;+ static int depth;+ int old_depth;++ fpsize = populate_node(blob, *poffset, &mem, dad, fpsize, &np, dryrun);
Doesn't this give a warning assigning a ptr to long?
Looks like np can be moved internal to populate_node.
@fpsize and the return value of populate_node() are "unsigned long".
Nope, @np will be used in next patch in this function, which tracks the
device node unflattened in last level of depth.
quoted
+ if (!fpsize)
+ return mem;
+
old_depth = depth;
*poffset = fdt_next_node(blob, *poffset, &depth);
if (depth < 0)
On Wed, Nov 04, 2015 at 10:07:50AM -0600, Rob Herring wrote:
On Wed, Nov 4, 2015 at 7:12 AM, Gavin Shan [off-list ref] wrote:
quoted
In current implementation, unflatten_dt_node() is called recursively
to unflatten device nodes in FDT blob. It's stress to limited stack
capacity.
Did you actually hit a problem?
Now we have a max depth of 64. Seems like that should be plenty... Any
idea how this compares to when we run out of stack space?
When I rebased last revision (v6), particular below patch, to 4.3.rc6,
the kernel won't boot in P7 and P8 boxes. On P7 boxes, the stack overruns
according to the printed kernel messages. On P8 boxes, the /bin/init in
initramfs image can't be loaded/executed properly and it's potentially
caused by memory corruption. That's why I reworked it to avoid recursive
calling to unflatten_dt_node().
The max depth "64" wasn't selected based on the stack usage. I was thinking
the device tree is converted to friendly *.dts format and it's using TAB
as the prefix for each line. If the device tree has 64 depth, Each line
in *.dts for leaf nodes have to be wrapped and spanning multiple lines.
That's why I choosed 64, maybe 32 is enough. Did you see a device-tree
that has more than 16 depth in field? :-)
quoted
This avoids calling the function recursively, meaning the device
nodes are unflattened in one call on unflatten_dt_node(): two arrays
are introduced to track the parent path size and the device node of
current level of depth, which will be used by the device node on next
level of depth to be unflattened. Also, the parameter "poffset" and
"fpsize" are unused and dropped.
Yay. I'm happy to see parameters removed instead of added to this function.
@@ -355,61 +355,82 @@ static unsigned long populate_node(const void *blob,returnfpsize;}+staticvoidreverse_nodes(structdevice_node*parent)+{+structdevice_node*child,*next;++/* In-depth first */+child=parent->child;+while(child){+reverse_nodes(child);++child=child->sibling;+}++/* Reverse the nodes in the child list */+child=parent->child;+parent->child=NULL;+while(child){+next=child->sibling;++child->sibling=parent->child;+parent->child=child;+child=next;+}+}+/***unflatten_dt_node-Allocandpopulateadevice_nodefromtheflattree*@blob:Theparentdevicetreeblob*@mem:Memorychunktouseforallocatingdevicenodesandproperties-*@poffset:pointertonodeinflattree*@dad:Parentstructdevice_node*@nodepp:Thedevice_nodetreecreatedbythecall-*@fpsize:Sizeofthenodepathupatthecurrentdepth.*@dryrun:Iftrue,donotallocatedevicenodesbutstillcalculateneeded*memorysize*/staticvoid*unflatten_dt_node(constvoid*blob,void*mem,-int*poffset,structdevice_node*dad,structdevice_node**nodepp,-unsignedlongfpsize,booldryrun)
We can probably further simplify things by returning an int with
negative being errors and positive being the size. Also, dryrun can be
dropped and implied by mem and/or nodepp being NULL.
Yeah, I think it's reasonable to return "size" from this function. "dryrun"
can be dropped and implied by NULL @mem. @nodepp can't be NULL. I perhaps
have separate patch to address it in next revision.
quoted
{
- struct device_node *np;
- static int depth;
- int old_depth;
-
- fpsize = populate_node(blob, *poffset, &mem, dad, fpsize, &np, dryrun);
- if (!fpsize)
- return mem;
+ struct device_node *root;
+ int offset = 0, depth = 0;
+ unsigned long fpsizes[64];
+ struct device_node *nps[64];
Use a define here.
Fair enough, will do in next revision. I'm not good at naming. Would
"FDT_MAX_DEPTH" is a good one?
What about depth == 64 case? I think the behavior should be a WARN and
ignore those nodes so we at least can continue to boot and see the
error. Of course, if there is a phandle pointing to ignored nodes, we
have to handle that too.
Yeah, I'll have a WARN_ON(depth >= 64) in next revision. Sorry, I didn't
get the 2nd part of your comments: When depth > 64, the system won't work.
It might boot up. Why the phandle pointing to the ignored node has to be
dropped?
quoted
/*
* Reverse the child list. Some drivers assumes node order matches .dts
* node order
*/
- if (!dryrun && np->child) {
- struct device_node *child = np->child;
- np->child = NULL;
- while (child) {
- struct device_node *next = child->sibling;
- child->sibling = np->child;
- np->child = child;
- child = next;
- }
- }
-
- if (nodepp)
- *nodepp = np;
+ if (!dryrun)
+ reverse_nodes(root);
return mem;
}
On Thu, Nov 05, 2015 at 10:23:15AM +1100, Gavin Shan wrote:
On Wed, Nov 04, 2015 at 10:07:50AM -0600, Rob Herring wrote:
quoted
On Wed, Nov 4, 2015 at 7:12 AM, Gavin Shan [off-list ref] wrote:
quoted
In current implementation, unflatten_dt_node() is called recursively
to unflatten device nodes in FDT blob. It's stress to limited stack
capacity.
Did you actually hit a problem?
Now we have a max depth of 64. Seems like that should be plenty... Any
idea how this compares to when we run out of stack space?
When I rebased last revision (v6), particular below patch, to 4.3.rc6,
the kernel won't boot in P7 and P8 boxes. On P7 boxes, the stack overruns
according to the printed kernel messages. On P8 boxes, the /bin/init in
initramfs image can't be loaded/executed properly and it's potentially
caused by memory corruption. That's why I reworked it to avoid recursive
calling to unflatten_dt_node().
The max depth "64" wasn't selected based on the stack usage. I was thinking
the device tree is converted to friendly *.dts format and it's using TAB
as the prefix for each line. If the device tree has 64 depth, Each line
in *.dts for leaf nodes have to be wrapped and spanning multiple lines.
That's why I choosed 64, maybe 32 is enough. Did you see a device-tree
that has more than 16 depth in field? :-)
quoted
quoted
This avoids calling the function recursively, meaning the device
nodes are unflattened in one call on unflatten_dt_node(): two arrays
are introduced to track the parent path size and the device node of
current level of depth, which will be used by the device node on next
level of depth to be unflattened. Also, the parameter "poffset" and
"fpsize" are unused and dropped.
Yay. I'm happy to see parameters removed instead of added to this function.
@@ -355,61 +355,82 @@ static unsigned long populate_node(const void *blob,returnfpsize;}+staticvoidreverse_nodes(structdevice_node*parent)+{+structdevice_node*child,*next;++/* In-depth first */+child=parent->child;+while(child){+reverse_nodes(child);++child=child->sibling;+}++/* Reverse the nodes in the child list */+child=parent->child;+parent->child=NULL;+while(child){+next=child->sibling;++child->sibling=parent->child;+parent->child=child;+child=next;+}+}+/***unflatten_dt_node-Allocandpopulateadevice_nodefromtheflattree*@blob:Theparentdevicetreeblob*@mem:Memorychunktouseforallocatingdevicenodesandproperties-*@poffset:pointertonodeinflattree*@dad:Parentstructdevice_node*@nodepp:Thedevice_nodetreecreatedbythecall-*@fpsize:Sizeofthenodepathupatthecurrentdepth.*@dryrun:Iftrue,donotallocatedevicenodesbutstillcalculateneeded*memorysize*/staticvoid*unflatten_dt_node(constvoid*blob,void*mem,-int*poffset,structdevice_node*dad,structdevice_node**nodepp,-unsignedlongfpsize,booldryrun)
We can probably further simplify things by returning an int with
negative being errors and positive being the size. Also, dryrun can be
dropped and implied by mem and/or nodepp being NULL.
Yeah, I think it's reasonable to return "size" from this function. "dryrun"
can be dropped and implied by NULL @mem. @nodepp can't be NULL. I perhaps
have separate patch to address it in next revision.
quoted
quoted
{
- struct device_node *np;
- static int depth;
- int old_depth;
-
- fpsize = populate_node(blob, *poffset, &mem, dad, fpsize, &np, dryrun);
- if (!fpsize)
- return mem;
+ struct device_node *root;
+ int offset = 0, depth = 0;
+ unsigned long fpsizes[64];
+ struct device_node *nps[64];
Use a define here.
Fair enough, will do in next revision. I'm not good at naming. Would
"FDT_MAX_DEPTH" is a good one?
What about depth == 64 case? I think the behavior should be a WARN and
ignore those nodes so we at least can continue to boot and see the
error. Of course, if there is a phandle pointing to ignored nodes, we
have to handle that too.
Yeah, I'll have a WARN_ON(depth >= 64) in next revision. Sorry, I didn't
get the 2nd part of your comments: When depth > 64, the system won't work.
It might boot up. Why the phandle pointing to the ignored node has to be
dropped?
quoted
quoted
/*
* Reverse the child list. Some drivers assumes node order matches .dts
* node order
*/
- if (!dryrun && np->child) {
- struct device_node *child = np->child;
- np->child = NULL;
- while (child) {
- struct device_node *next = child->sibling;
- child->sibling = np->child;
- np->child = child;
- child = next;
- }
- }
-
- if (nodepp)
- *nodepp = np;
+ if (!dryrun)
+ reverse_nodes(root);
return mem;
}
Very much a nit-pick, but I thought we were trying to move towards using
phb instead of hose in new code?
Apart from that this looks good. I would probably have merged it with
the previous patch, but I know Alexey has been suggesting a lot of
splitting and merging previously, so whatever he prefers here is OK.
+
+ if (hose->controller_ops.setup_bridge)
+ hose->controller_ops.setup_bridge(bus, type);
+}
+
void pcibios_reset_secondary_bus(struct pci_dev *dev)
{
struct pci_controller *phb = pci_bus_to_host(dev->bus);
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Daniel Axtens <hidden> Date: 2015-11-05 22:28:20
Gavin Shan [off-list ref] writes:
This cleans up on pnv_pci_ioda_controller_ops struct to use tab
instead of space indent of statement to avoid complains from
scripts/checkpatch.pl. No logical changes introduced.
Oh, that was my code :/ Sorry I missed that, thanks for cleaning it up!
Reviewed-by: Daniel Axtens <redacted>
Very much a nit-pick, but I thought we were trying to move towards using
phb instead of hose in new code?
Take PowerNV platform as an example, "hose" means "struct pci_controller",
but "phb" means "struct pnv_phb". We don't have the movement to use "phb"
to represent "struct pci_controller".
Apart from that this looks good. I would probably have merged it with
the previous patch, but I know Alexey has been suggesting a lot of
splitting and merging previously, so whatever he prefers here is OK.
I'd like to keep them separate as they're for different subsystem:
PCI generic subsystem and PowerPC subsystem. Separate maintainers
for them can pick them as they will.
quoted
+
+ if (hose->controller_ops.setup_bridge)
+ hose->controller_ops.setup_bridge(bus, type);
+}
+
void pcibios_reset_secondary_bus(struct pci_dev *dev)
{
struct pci_controller *phb = pci_bus_to_host(dev->bus);
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Nov 06, 2015 at 09:28:20AM +1100, Daniel Axtens wrote:
Gavin Shan [off-list ref] writes:
quoted
This cleans up on pnv_pci_ioda_controller_ops struct to use tab
instead of space indent of statement to avoid complains from
scripts/checkpatch.pl. No logical changes introduced.
Oh, that was my code :/ Sorry I missed that, thanks for cleaning it up!
Reviewed-by: Daniel Axtens <redacted>
nah, that's fine, Daniel. You might be saying: I didn't review your code
carefully enough, which leaves me the chance to clean it up :-)
Thanks,
Gavin
On Thu, Nov 05, 2015 at 12:12:00AM +1100, Gavin Shan wrote:
This series of patches rebases on powerpc/next branch, plus below additional
patches:
https://patchwork.ozlabs.org/patch/534804/ (PATCH[1/1] Andrew's EEH fix)
https://patchwork.ozlabs.org/patch/534154/ (PATCH[7/7] Richard's SRIOV Rework)
commit 3b0e21e Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/scottwood/linux into next
The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skiboot@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.
The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 27] are related to this part.
quoted
From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. Conversely,
the device-tree node should be removed from the system when the PCI adapter is going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[28 - 43] are
doing the related work.
The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[44 - 49].
The last one, PATCH[50], is the standalone PCI hotplug driver for PowerPC PowerNV
platform.
Changelog
=========
v7:
* Reworked revision to some extent.
* Rebased to powerpc/next repository.
* Reorder/split/merge/drop according - Alexey.
* Defined macros and use array to track IO/M32/M64/DMA32 segments - Alexey.
* Merged 3 files to one for the hotplug driver - Alexey.
* As part of OPAL API, defined macros for PCI slot power state, hotplug
message type. Defined macros for PCI slot power confirmed state in
hotplug driver.
* Misc comments from Alexey.
* Reworked unflatten_dt_node() to avoid recursive function calls.
* Use EXPORT_SYMBOL_GPL() and document function's input/output - Rob/Frank.
v6:
* Patch reorder, split, squash - Alexey.
* Minor coding style - Alexey.
* Better function names for pcibios_{add,remove}_pci_devices - Bjorn
* Replace pr_warn() with dev_warn() in PowerNV hotplug driver - Bjorn
* Concurrent depth as parameter passed to __unflatten_dt_node() - Grant / Alexey
* Replace overlay with of_changeset - Grant
v5:
* Rebased to 4.1.rc6 and some unmerged patches as below:
Alexey's DDW patchset (v11);
Gavin's EEH error injection support (in mpe's next branch);
Richard's EEH cleanup patches (in mpe's next branch);
Richard's EEH support for VF (v7);
Gavin's misc EEH fixes for 4.2;
* The revision bases on skiboot corresponding patches (v7):
https://patchwork.ozlabs.org/patch/480437/
* Utilize OF overlay to update device-tree with help of newly introduced
OPAL API opal_get_overlay_dt().
* Split patches for easy review according to aik's comments.
* Fix coding style from checkpatchc.pl as pointed by aik.
* Code cleanup and misc fixup according to aik's input.
v4:
* Rebased to 4.1.RC1
* Added API to unflatten FDT blob to device node sub-tree, which is attached
the indicated parent device node. The original mechanism based on formatted
string stream has been dropped.
* The PATCH[v3 09/21] ("powerpc/eeh: Delay probing EEH device during hotplug")
was picked up sent to linux-ppc@ separately for review as Richard's "VF EEH
Support" depends on that.
v3:
* Rebased to 4.1.RC0
* PowerNV PCI infrasturcture is total refactored in order to support PCI
hotplug. The PowerNV hotplug driver is also reworked a lot because of
the changes in skiboot in order to support PCI hotplug.
Gavin Shan (50):
PCI: Add pcibios_setup_bridge()
powerpc/pci: Override pcibios_setup_bridge()
powerpc/pci: Cleanup on struct pci_controller_ops
powerpc/powernv: Cleanup on pnv_pci_ioda_controller_ops
powerpc/powernv: Drop pnv_ioda_setup_dev_PE()
powerpc/powernv: Drop phb->bdfn_to_pe()
powerpc/powernv: Reorder fields in struct pnv_phb
powerpc/powernv: Rename PE# fields in struct pnv_phb
powerpc/powernv: Fix initial IO and M32 segmap
powerpc/powernv: Simplify pnv_ioda_setup_pe_seg()
powerpc/powernv: IO and M32 mapping based on PCI device resources
powerpc/powernv: Track M64 segment consumption
powerpc/powernv: Rename M64 related functions
powerpc/powernv: M64 support on P7IOC
powerpc/powernv: Rename pnv_pci_ioda_setup_dma_pe()
powerpc/powernv: Define PNV_IODA1_DMA32_SEGSIZE
powerpc/powernv: Avoid calculating DMA32 segments on PHB3
powerpc/powernv: Remove DMA32 PE list
powerpc/powernv: Track DMA32 segment consumption
powerpc/powernv: Improve DMA32 segment calculation
powerpc/powernv: Increase PE# capacity
powerpc/powernv: Introduce pnv_ioda_init_pe()
powerpc/powernv: Use PE instead of number during setup and release
powerpc/powernv: Allocate PE# in reverse order
powerpc/powernv: Reserve PE for root bus
powerpc/powernv: Create PEs at PCI hot plugging time
powerpc/powernv: Dynamically release PEs
powerpc/pci: Rename pcibios_{add,remove}_pci_devices()
powerpc/pci: Rename pcibios_find_pci_bus()
powerpc/pci: Move pci_find_bus_by_node() around
powerpc/pci: Export pci_add_device_node_info()
powerpc/pci: Introduce pci_remove_device_node_info()
powerpc/pci: Export pci_traverse_device_nodes()
powerpc/pci: Delay populating pdn
powerpc/pci: Don't scan empty slot
powerpc/pci: Update bridge windows on PCI plug
powerpc/powernv: Simplify pnv_eeh_reset()
powerpc/powernv: Exclude root bus in pnv_pci_reset_secondary_bus()
powerpc/powernv: Fundamental reset in pnv_pci_reset_secondary_bus()
powerpc/powernv: Support PCI slot ID
powerpc/powernv: Use firmware PCI slot reset infrastructure
powerpc/powernv: Functions to get/set PCI slot status
powerpc/powernv: Select OF_DYNAMIC
drivers/of: Split unflatten_dt_node()
drivers/of: Avoid recursively calling unflatten_dt_node()
drivers/of: Rename unflatten_dt_node()
drivers/of: Specify parent node in of_fdt_unflatten_tree()
drivers/of: Return allocated memory from of_fdt_unflatten_tree()
drivers/of: Export OF changeset functions
PCI/hotplug: PowerPC PowerNV PCI hotplug driver
MAINTAINERS | 6 +
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/opal-api.h | 17 +-
arch/powerpc/include/asm/opal.h | 8 +-
arch/powerpc/include/asm/pci-bridge.h | 25 +-
arch/powerpc/include/asm/pnv-pci.h | 7 +
arch/powerpc/include/asm/ppc-pci.h | 8 +-
arch/powerpc/kernel/eeh_dev.c | 19 +-
arch/powerpc/kernel/eeh_driver.c | 12 +-
arch/powerpc/kernel/pci-common.c | 16 +-
arch/powerpc/kernel/pci-hotplug.c | 47 +-
arch/powerpc/kernel/pci_dn.c | 85 +-
arch/powerpc/platforms/maple/pci.c | 34 +-
arch/powerpc/platforms/pasemi/pci.c | 3 -
arch/powerpc/platforms/powermac/pci.c | 38 +-
arch/powerpc/platforms/powernv/Kconfig | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 173 ++--
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 +
arch/powerpc/platforms/powernv/pci-ioda.c | 1251 +++++++++++++++---------
arch/powerpc/platforms/powernv/pci.c | 92 +-
arch/powerpc/platforms/powernv/pci.h | 62 +-
arch/powerpc/platforms/pseries/msi.c | 4 +-
arch/powerpc/platforms/pseries/pci_dlpar.c | 32 -
arch/powerpc/platforms/pseries/setup.c | 8 +-
drivers/of/dynamic.c | 65 +-
drivers/of/fdt.c | 378 ++++---
drivers/of/of_private.h | 2 +
drivers/of/overlay.c | 8 +-
drivers/of/unittest.c | 6 +-
drivers/pci/hotplug/Kconfig | 12 +
drivers/pci/hotplug/Makefile | 3 +
drivers/pci/hotplug/pnv_php.c | 866 ++++++++++++++++
drivers/pci/hotplug/rpadlpar_core.c | 8 +-
drivers/pci/hotplug/rpaphp_core.c | 4 +-
drivers/pci/hotplug/rpaphp_pci.c | 4 +-
drivers/pci/setup-bus.c | 5 +
include/linux/of_fdt.h | 5 +-
include/linux/pci.h | 1 +
38 files changed, 2389 insertions(+), 932 deletions(-)
create mode 100644 drivers/pci/hotplug/pnv_php.c
--
2.1.0
The series of patches intend to support PCI slot for PowerPC PowerNV =
platform,
quoted
which is running on top of skiboot firmware. The patchset requires =
corresponding
quoted
changes from skiboot firmware, which is sent to =
skiboot@lists.ozlabs.org
quoted
for review. The PCI slots are exposed by skiboot with device node =
properties,
quoted
and kernel utilizes those properties to populated PCI slots =
accordingly.
quoted
=20
The original PCI infrastructure on PowerNV platform can't support =
hotplug
quoted
because the PE is assigned during PHB fixup time, which is called for =
once
quoted
during system boot time. For this, the PCI infrastructure on PowerNV =
platform
quoted
has been reworked for a lot. After that, the PE and its corresponding =
resources
quoted
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned =
upon updating
quoted
PCI bridge's resources, which might decide PE# assigned to the PE =
(e.g. M64
quoted
resources, on P8 strictly speaking). Each PE will maintain a =
reference count,
quoted
which is (number of child PCI devices + 1). That indicates when last =
child PCI
quoted
device leaves the PE, the PE and its included resources will be =
relased and put
quoted
back into free pool again. With this design, the PE will be released =
when EEH PE
quoted
is released. PATCH[1 - 27] are related to this part.
=20
quoted
=46rom skiboot perspective, PCI slot is providing =
(hot/fundamental/complete)
quoted
resets to EEH. The kernel gets to know if skiboot supports various =
reset on one
quoted
particular PCI slot through device-tree node. If it does, EEH will =
utilize the
quoted
functionality provided by skiboot. Besides, the device-tree nodes =
have to change
quoted
in order to support PCI hotplug. For example, when one PCI adapter =
inserted to
quoted
one slot, its device-tree node should be added to the system =
dynamically. Conversely,
quoted
the device-tree node should be removed from the system when the PCI =
adapter is going
quoted
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI =
device nodes,
quoted
they should be added/removed accordingly during PCI hotplug. PATCH[28 =
- 43] are
quoted
doing the related work.
=20
The OF driver is changed to support unflattening FDT blob for =
sub-stree, which
quoted
is covered by PATCH[44 - 49].
=20
The last one, PATCH[50], is the standalone PCI hotplug driver for =
PowerPC PowerNV
quoted
platform.=20
=20
Changelog
=3D=3D=3D=3D=3D=3D=3D=3D=3D
v7:
* Reworked revision to some extent.
* Rebased to powerpc/next repository.
* Reorder/split/merge/drop according - Alexey.
* Defined macros and use array to track IO/M32/M64/DMA32 segments - =
Alexey.
quoted
* Merged 3 files to one for the hotplug driver - Alexey.
* As part of OPAL API, defined macros for PCI slot power state, =
hotplug
quoted
message type. Defined macros for PCI slot power confirmed state in
hotplug driver.
* Misc comments from Alexey.
* Reworked unflatten_dt_node() to avoid recursive function calls.
* Use EXPORT_SYMBOL_GPL() and document function's input/output - =
Rob/Frank.
quoted
v6:
* Patch reorder, split, squash - Alexey.
* Minor coding style - Alexey.
* Better function names for pcibios_{add,remove}_pci_devices - Bjorn
* Replace pr_warn() with dev_warn() in PowerNV hotplug driver - =
Bjorn
quoted
* Concurrent depth as parameter passed to __unflatten_dt_node() - =
Grant / Alexey
quoted
* Replace overlay with of_changeset - Grant
v5:
* Rebased to 4.1.rc6 and some unmerged patches as below:
Alexey's DDW patchset (v11);
Gavin's EEH error injection support (in mpe's next branch);
Richard's EEH cleanup patches (in mpe's next branch);
Richard's EEH support for VF (v7);
Gavin's misc EEH fixes for 4.2;
* The revision bases on skiboot corresponding patches (v7):
https://patchwork.ozlabs.org/patch/480437/
* Utilize OF overlay to update device-tree with help of newly =
introduced
quoted
OPAL API opal_get_overlay_dt().
* Split patches for easy review according to aik's comments.
* Fix coding style from checkpatchc.pl as pointed by aik.
* Code cleanup and misc fixup according to aik's input.
v4:
* Rebased to 4.1.RC1
* Added API to unflatten FDT blob to device node sub-tree, which is =
attached
quoted
the indicated parent device node. The original mechanism based on =
formatted
quoted
string stream has been dropped.
* The PATCH[v3 09/21] ("powerpc/eeh: Delay probing EEH device during =
hotplug")
quoted
was picked up sent to linux-ppc@ separately for review as =
Richard's "VF EEH
quoted
Support" depends on that.
v3:
* Rebased to 4.1.RC0
* PowerNV PCI infrasturcture is total refactored in order to support =
PCI
quoted
hotplug. The PowerNV hotplug driver is also reworked a lot because =
of
quoted
the changes in skiboot in order to support PCI hotplug.
=20
Gavin Shan (50):
PCI: Add pcibios_setup_bridge()
powerpc/pci: Override pcibios_setup_bridge()
powerpc/pci: Cleanup on struct pci_controller_ops
powerpc/powernv: Cleanup on pnv_pci_ioda_controller_ops
powerpc/powernv: Drop pnv_ioda_setup_dev_PE()
powerpc/powernv: Drop phb->bdfn_to_pe()
powerpc/powernv: Reorder fields in struct pnv_phb
powerpc/powernv: Rename PE# fields in struct pnv_phb
powerpc/powernv: Fix initial IO and M32 segmap
powerpc/powernv: Simplify pnv_ioda_setup_pe_seg()
powerpc/powernv: IO and M32 mapping based on PCI device resources
powerpc/powernv: Track M64 segment consumption
powerpc/powernv: Rename M64 related functions
powerpc/powernv: M64 support on P7IOC
powerpc/powernv: Rename pnv_pci_ioda_setup_dma_pe()
powerpc/powernv: Define PNV_IODA1_DMA32_SEGSIZE
powerpc/powernv: Avoid calculating DMA32 segments on PHB3
powerpc/powernv: Remove DMA32 PE list
powerpc/powernv: Track DMA32 segment consumption
powerpc/powernv: Improve DMA32 segment calculation
powerpc/powernv: Increase PE# capacity
powerpc/powernv: Introduce pnv_ioda_init_pe()
powerpc/powernv: Use PE instead of number during setup and release
powerpc/powernv: Allocate PE# in reverse order
powerpc/powernv: Reserve PE for root bus
powerpc/powernv: Create PEs at PCI hot plugging time
powerpc/powernv: Dynamically release PEs
powerpc/pci: Rename pcibios_{add,remove}_pci_devices()
powerpc/pci: Rename pcibios_find_pci_bus()
powerpc/pci: Move pci_find_bus_by_node() around
powerpc/pci: Export pci_add_device_node_info()
powerpc/pci: Introduce pci_remove_device_node_info()
powerpc/pci: Export pci_traverse_device_nodes()
powerpc/pci: Delay populating pdn
powerpc/pci: Don't scan empty slot
powerpc/pci: Update bridge windows on PCI plug
powerpc/powernv: Simplify pnv_eeh_reset()
powerpc/powernv: Exclude root bus in pnv_pci_reset_secondary_bus()
powerpc/powernv: Fundamental reset in pnv_pci_reset_secondary_bus()
powerpc/powernv: Support PCI slot ID
powerpc/powernv: Use firmware PCI slot reset infrastructure
powerpc/powernv: Functions to get/set PCI slot status
powerpc/powernv: Select OF_DYNAMIC
drivers/of: Split unflatten_dt_node()
drivers/of: Avoid recursively calling unflatten_dt_node()
drivers/of: Rename unflatten_dt_node()
drivers/of: Specify parent node in of_fdt_unflatten_tree()
drivers/of: Return allocated memory from of_fdt_unflatten_tree()
drivers/of: Export OF changeset functions
PCI/hotplug: PowerPC PowerNV PCI hotplug driver
=20
MAINTAINERS | 6 +
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/opal-api.h | 17 +-
arch/powerpc/include/asm/opal.h | 8 +-
arch/powerpc/include/asm/pci-bridge.h | 25 +-
arch/powerpc/include/asm/pnv-pci.h | 7 +
arch/powerpc/include/asm/ppc-pci.h | 8 +-
arch/powerpc/kernel/eeh_dev.c | 19 +-
arch/powerpc/kernel/eeh_driver.c | 12 +-
arch/powerpc/kernel/pci-common.c | 16 +-
arch/powerpc/kernel/pci-hotplug.c | 47 +-
arch/powerpc/kernel/pci_dn.c | 85 +-
arch/powerpc/platforms/maple/pci.c | 34 +-
arch/powerpc/platforms/pasemi/pci.c | 3 -
arch/powerpc/platforms/powermac/pci.c | 38 +-
arch/powerpc/platforms/powernv/Kconfig | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 173 ++--
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 +
arch/powerpc/platforms/powernv/pci-ioda.c | 1251 =
On Sun, Nov 08, 2015 at 08:24:37PM -0800, Pramod Sudheendra wrote:
quoted
On Nov 8, 2015, at 7:09 PM, Gavin Shan [off-list ref] wrote:
On Thu, Nov 05, 2015 at 12:12:00AM +1100, Gavin Shan wrote:
quoted
This series of patches rebases on powerpc/next branch, plus below additional
patches:
https://patchwork.ozlabs.org/patch/534804/ (PATCH[1/1] Andrew's EEH fix)
https://patchwork.ozlabs.org/patch/534154/ (PATCH[7/7] Richard's SRIOV Rework)
commit 3b0e21e Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/scottwood/linux into next
Yeah, I dropped that before it's populated completely as I was told it's disallowed
by my employer. I have to push it into IBM internal git server and it's only visible
to IBM. Sorry for the inconvienence...
quoted
quoted
The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skiboot@lists.ozlabs.org
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.
The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 27] are related to this part.
quoted
From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. Conversely,
the device-tree node should be removed from the system when the PCI adapter is going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[28 - 43] are
doing the related work.
The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[44 - 49].
The last one, PATCH[50], is the standalone PCI hotplug driver for PowerPC PowerNV
platform.
Changelog
=========
v7:
* Reworked revision to some extent.
* Rebased to powerpc/next repository.
* Reorder/split/merge/drop according - Alexey.
* Defined macros and use array to track IO/M32/M64/DMA32 segments - Alexey.
* Merged 3 files to one for the hotplug driver - Alexey.
* As part of OPAL API, defined macros for PCI slot power state, hotplug
message type. Defined macros for PCI slot power confirmed state in
hotplug driver.
* Misc comments from Alexey.
* Reworked unflatten_dt_node() to avoid recursive function calls.
* Use EXPORT_SYMBOL_GPL() and document function's input/output - Rob/Frank.
v6:
* Patch reorder, split, squash - Alexey.
* Minor coding style - Alexey.
* Better function names for pcibios_{add,remove}_pci_devices - Bjorn
* Replace pr_warn() with dev_warn() in PowerNV hotplug driver - Bjorn
* Concurrent depth as parameter passed to __unflatten_dt_node() - Grant / Alexey
* Replace overlay with of_changeset - Grant
v5:
* Rebased to 4.1.rc6 and some unmerged patches as below:
Alexey's DDW patchset (v11);
Gavin's EEH error injection support (in mpe's next branch);
Richard's EEH cleanup patches (in mpe's next branch);
Richard's EEH support for VF (v7);
Gavin's misc EEH fixes for 4.2;
* The revision bases on skiboot corresponding patches (v7):
https://patchwork.ozlabs.org/patch/480437/
* Utilize OF overlay to update device-tree with help of newly introduced
OPAL API opal_get_overlay_dt().
* Split patches for easy review according to aik's comments.
* Fix coding style from checkpatchc.pl as pointed by aik.
* Code cleanup and misc fixup according to aik's input.
v4:
* Rebased to 4.1.RC1
* Added API to unflatten FDT blob to device node sub-tree, which is attached
the indicated parent device node. The original mechanism based on formatted
string stream has been dropped.
* The PATCH[v3 09/21] ("powerpc/eeh: Delay probing EEH device during hotplug")
was picked up sent to linux-ppc@ separately for review as Richard's "VF EEH
Support" depends on that.
v3:
* Rebased to 4.1.RC0
* PowerNV PCI infrasturcture is total refactored in order to support PCI
hotplug. The PowerNV hotplug driver is also reworked a lot because of
the changes in skiboot in order to support PCI hotplug.
Gavin Shan (50):
PCI: Add pcibios_setup_bridge()
powerpc/pci: Override pcibios_setup_bridge()
powerpc/pci: Cleanup on struct pci_controller_ops
powerpc/powernv: Cleanup on pnv_pci_ioda_controller_ops
powerpc/powernv: Drop pnv_ioda_setup_dev_PE()
powerpc/powernv: Drop phb->bdfn_to_pe()
powerpc/powernv: Reorder fields in struct pnv_phb
powerpc/powernv: Rename PE# fields in struct pnv_phb
powerpc/powernv: Fix initial IO and M32 segmap
powerpc/powernv: Simplify pnv_ioda_setup_pe_seg()
powerpc/powernv: IO and M32 mapping based on PCI device resources
powerpc/powernv: Track M64 segment consumption
powerpc/powernv: Rename M64 related functions
powerpc/powernv: M64 support on P7IOC
powerpc/powernv: Rename pnv_pci_ioda_setup_dma_pe()
powerpc/powernv: Define PNV_IODA1_DMA32_SEGSIZE
powerpc/powernv: Avoid calculating DMA32 segments on PHB3
powerpc/powernv: Remove DMA32 PE list
powerpc/powernv: Track DMA32 segment consumption
powerpc/powernv: Improve DMA32 segment calculation
powerpc/powernv: Increase PE# capacity
powerpc/powernv: Introduce pnv_ioda_init_pe()
powerpc/powernv: Use PE instead of number during setup and release
powerpc/powernv: Allocate PE# in reverse order
powerpc/powernv: Reserve PE for root bus
powerpc/powernv: Create PEs at PCI hot plugging time
powerpc/powernv: Dynamically release PEs
powerpc/pci: Rename pcibios_{add,remove}_pci_devices()
powerpc/pci: Rename pcibios_find_pci_bus()
powerpc/pci: Move pci_find_bus_by_node() around
powerpc/pci: Export pci_add_device_node_info()
powerpc/pci: Introduce pci_remove_device_node_info()
powerpc/pci: Export pci_traverse_device_nodes()
powerpc/pci: Delay populating pdn
powerpc/pci: Don't scan empty slot
powerpc/pci: Update bridge windows on PCI plug
powerpc/powernv: Simplify pnv_eeh_reset()
powerpc/powernv: Exclude root bus in pnv_pci_reset_secondary_bus()
powerpc/powernv: Fundamental reset in pnv_pci_reset_secondary_bus()
powerpc/powernv: Support PCI slot ID
powerpc/powernv: Use firmware PCI slot reset infrastructure
powerpc/powernv: Functions to get/set PCI slot status
powerpc/powernv: Select OF_DYNAMIC
drivers/of: Split unflatten_dt_node()
drivers/of: Avoid recursively calling unflatten_dt_node()
drivers/of: Rename unflatten_dt_node()
drivers/of: Specify parent node in of_fdt_unflatten_tree()
drivers/of: Return allocated memory from of_fdt_unflatten_tree()
drivers/of: Export OF changeset functions
PCI/hotplug: PowerPC PowerNV PCI hotplug driver
MAINTAINERS | 6 +
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/opal-api.h | 17 +-
arch/powerpc/include/asm/opal.h | 8 +-
arch/powerpc/include/asm/pci-bridge.h | 25 +-
arch/powerpc/include/asm/pnv-pci.h | 7 +
arch/powerpc/include/asm/ppc-pci.h | 8 +-
arch/powerpc/kernel/eeh_dev.c | 19 +-
arch/powerpc/kernel/eeh_driver.c | 12 +-
arch/powerpc/kernel/pci-common.c | 16 +-
arch/powerpc/kernel/pci-hotplug.c | 47 +-
arch/powerpc/kernel/pci_dn.c | 85 +-
arch/powerpc/platforms/maple/pci.c | 34 +-
arch/powerpc/platforms/pasemi/pci.c | 3 -
arch/powerpc/platforms/powermac/pci.c | 38 +-
arch/powerpc/platforms/powernv/Kconfig | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 173 ++--
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 +
arch/powerpc/platforms/powernv/pci-ioda.c | 1251 +++++++++++++++---------
arch/powerpc/platforms/powernv/pci.c | 92 +-
arch/powerpc/platforms/powernv/pci.h | 62 +-
arch/powerpc/platforms/pseries/msi.c | 4 +-
arch/powerpc/platforms/pseries/pci_dlpar.c | 32 -
arch/powerpc/platforms/pseries/setup.c | 8 +-
drivers/of/dynamic.c | 65 +-
drivers/of/fdt.c | 378 ++++---
drivers/of/of_private.h | 2 +
drivers/of/overlay.c | 8 +-
drivers/of/unittest.c | 6 +-
drivers/pci/hotplug/Kconfig | 12 +
drivers/pci/hotplug/Makefile | 3 +
drivers/pci/hotplug/pnv_php.c | 866 ++++++++++++++++
drivers/pci/hotplug/rpadlpar_core.c | 8 +-
drivers/pci/hotplug/rpaphp_core.c | 4 +-
drivers/pci/hotplug/rpaphp_pci.c | 4 +-
drivers/pci/setup-bus.c | 5 +
include/linux/of_fdt.h | 5 +-
include/linux/pci.h | 1 +
38 files changed, 2389 insertions(+), 932 deletions(-)
create mode 100644 drivers/pci/hotplug/pnv_php.c
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2015-11-09 06:43:44
On Mon, 2015-11-09 at 15:29 +1100, Gavin Shan wrote:
Yeah, I dropped that before it's populated completely as I was told
it's disallowed
by my employer. I have to push it into IBM internal git server and
it's only visible
to IBM. Sorry for the inconvienence...
I think that's a misinterpretation of the rule, I'll sort that out
tomorrow, there should be no problem publishing that tree on github as
long as you take a couple of precautions.
Cheers,
Ben.
quoted
quoted
quoted
The series of patches intend to support PCI slot for PowerPC
PowerNV platform,
which is running on top of skiboot firmware. The patchset
requires corresponding
changes from skiboot firmware, which is sent to skiboot@lists.o
zlabs.org
for review. The PCI slots are exposed by skiboot with device
node properties,
and kernel utilizes those properties to populated PCI slots
accordingly.
The original PCI infrastructure on PowerNV platform can't
support hotplug
because the PE is assigned during PHB fixup time, which is
called for once
during system boot time. For this, the PCI infrastructure on
PowerNV platform
has been reworked for a lot. After that, the PE and its
corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are
assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the
PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a
reference count,
which is (number of child PCI devices + 1). That indicates when
last child PCI
device leaves the PE, the PE and its included resources will be
relased and put
back into free pool again. With this design, the PE will be
released when EEH PE
is released. PATCH[1 - 27] are related to this part.
quoted
From skiboot perspective, PCI slot is providing
(hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports
various reset on one
particular PCI slot through device-tree node. If it does, EEH
will utilize the
functionality provided by skiboot. Besides, the device-tree
nodes have to change
in order to support PCI hotplug. For example, when one PCI
adapter inserted to
one slot, its device-tree node should be added to the system
dynamically. Conversely,
the device-tree node should be removed from the system when the
PCI adapter is going
to be offline. Since pci_dn and eeh_dev have same life cyle as
PCI device nodes,
they should be added/removed accordingly during PCI hotplug.
PATCH[28 - 43] are
doing the related work.
The OF driver is changed to support unflattening FDT blob for
sub-stree, which
is covered by PATCH[44 - 49].
The last one, PATCH[50], is the standalone PCI hotplug driver
for PowerPC PowerNV
platform.
Changelog
=========
v7:
* Reworked revision to some extent.
* Rebased to powerpc/next repository.
* Reorder/split/merge/drop according - Alexey.
* Defined macros and use array to track IO/M32/M64/DMA32
segments - Alexey.
* Merged 3 files to one for the hotplug driver - Alexey.
* As part of OPAL API, defined macros for PCI slot power
state, hotplug
message type. Defined macros for PCI slot power confirmed
state in
hotplug driver.
* Misc comments from Alexey.
* Reworked unflatten_dt_node() to avoid recursive function
calls.
* Use EXPORT_SYMBOL_GPL() and document function's input/output
- Rob/Frank.
v6:
* Patch reorder, split, squash - Alexey.
* Minor coding style - Alexey.
* Better function names for pcibios_{add,remove}_pci_devices -
Bjorn
* Replace pr_warn() with dev_warn() in PowerNV hotplug driver
- Bjorn
* Concurrent depth as parameter passed to
__unflatten_dt_node() - Grant / Alexey
* Replace overlay with of_changeset - Grant
v5:
* Rebased to 4.1.rc6 and some unmerged patches as below:
Alexey's DDW patchset (v11);
Gavin's EEH error injection support (in mpe's next branch);
Richard's EEH cleanup patches (in mpe's next branch);
Richard's EEH support for VF (v7);
Gavin's misc EEH fixes for 4.2;
* The revision bases on skiboot corresponding patches (v7):
https://patchwork.ozlabs.org/patch/480437/
* Utilize OF overlay to update device-tree with help of newly
introduced
OPAL API opal_get_overlay_dt().
* Split patches for easy review according to aik's comments.
* Fix coding style from checkpatchc.pl as pointed by aik.
* Code cleanup and misc fixup according to aik's input.
v4:
* Rebased to 4.1.RC1
* Added API to unflatten FDT blob to device node sub-tree,
which is attached
the indicated parent device node. The original mechanism
based on formatted
string stream has been dropped.
* The PATCH[v3 09/21] ("powerpc/eeh: Delay probing EEH device
during hotplug")
was picked up sent to linux-ppc@ separately for review as
Richard's "VF EEH
Support" depends on that.
v3:
* Rebased to 4.1.RC0
* PowerNV PCI infrasturcture is total refactored in order to
support PCI
hotplug. The PowerNV hotplug driver is also reworked a lot
because of
the changes in skiboot in order to support PCI hotplug.
Gavin Shan (50):
PCI: Add pcibios_setup_bridge()
powerpc/pci: Override pcibios_setup_bridge()
powerpc/pci: Cleanup on struct pci_controller_ops
powerpc/powernv: Cleanup on pnv_pci_ioda_controller_ops
powerpc/powernv: Drop pnv_ioda_setup_dev_PE()
powerpc/powernv: Drop phb->bdfn_to_pe()
powerpc/powernv: Reorder fields in struct pnv_phb
powerpc/powernv: Rename PE# fields in struct pnv_phb
powerpc/powernv: Fix initial IO and M32 segmap
powerpc/powernv: Simplify pnv_ioda_setup_pe_seg()
powerpc/powernv: IO and M32 mapping based on PCI device
resources
powerpc/powernv: Track M64 segment consumption
powerpc/powernv: Rename M64 related functions
powerpc/powernv: M64 support on P7IOC
powerpc/powernv: Rename pnv_pci_ioda_setup_dma_pe()
powerpc/powernv: Define PNV_IODA1_DMA32_SEGSIZE
powerpc/powernv: Avoid calculating DMA32 segments on PHB3
powerpc/powernv: Remove DMA32 PE list
powerpc/powernv: Track DMA32 segment consumption
powerpc/powernv: Improve DMA32 segment calculation
powerpc/powernv: Increase PE# capacity
powerpc/powernv: Introduce pnv_ioda_init_pe()
powerpc/powernv: Use PE instead of number during setup and
release
powerpc/powernv: Allocate PE# in reverse order
powerpc/powernv: Reserve PE for root bus
powerpc/powernv: Create PEs at PCI hot plugging time
powerpc/powernv: Dynamically release PEs
powerpc/pci: Rename pcibios_{add,remove}_pci_devices()
powerpc/pci: Rename pcibios_find_pci_bus()
powerpc/pci: Move pci_find_bus_by_node() around
powerpc/pci: Export pci_add_device_node_info()
powerpc/pci: Introduce pci_remove_device_node_info()
powerpc/pci: Export pci_traverse_device_nodes()
powerpc/pci: Delay populating pdn
powerpc/pci: Don't scan empty slot
powerpc/pci: Update bridge windows on PCI plug
powerpc/powernv: Simplify pnv_eeh_reset()
powerpc/powernv: Exclude root bus in
pnv_pci_reset_secondary_bus()
powerpc/powernv: Fundamental reset in
pnv_pci_reset_secondary_bus()
powerpc/powernv: Support PCI slot ID
powerpc/powernv: Use firmware PCI slot reset infrastructure
powerpc/powernv: Functions to get/set PCI slot status
powerpc/powernv: Select OF_DYNAMIC
drivers/of: Split unflatten_dt_node()
drivers/of: Avoid recursively calling unflatten_dt_node()
drivers/of: Rename unflatten_dt_node()
drivers/of: Specify parent node in of_fdt_unflatten_tree()
drivers/of: Return allocated memory from
of_fdt_unflatten_tree()
drivers/of: Export OF changeset functions
PCI/hotplug: PowerPC PowerNV PCI hotplug driver
MAINTAINERS | 6 +
arch/powerpc/include/asm/eeh.h | 2 +-
arch/powerpc/include/asm/opal-api.h | 17 +-
arch/powerpc/include/asm/opal.h | 8 +-
arch/powerpc/include/asm/pci-bridge.h | 25 +-
arch/powerpc/include/asm/pnv-pci.h | 7 +
arch/powerpc/include/asm/ppc-pci.h | 8 +-
arch/powerpc/kernel/eeh_dev.c | 19 +-
arch/powerpc/kernel/eeh_driver.c | 12 +-
arch/powerpc/kernel/pci-common.c | 16 +-
arch/powerpc/kernel/pci-hotplug.c | 47 +-
arch/powerpc/kernel/pci_dn.c | 85 +-
arch/powerpc/platforms/maple/pci.c | 34 +-
arch/powerpc/platforms/pasemi/pci.c | 3 -
arch/powerpc/platforms/powermac/pci.c | 38 +-
arch/powerpc/platforms/powernv/Kconfig | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 173 ++--
arch/powerpc/platforms/powernv/opal-wrappers.S | 4 +
arch/powerpc/platforms/powernv/pci-ioda.c | 1251
+++++++++++++++---------
arch/powerpc/platforms/powernv/pci.c | 92 +-
arch/powerpc/platforms/powernv/pci.h | 62 +-
arch/powerpc/platforms/pseries/msi.c | 4 +-
arch/powerpc/platforms/pseries/pci_dlpar.c | 32 -
arch/powerpc/platforms/pseries/setup.c | 8 +-
drivers/of/dynamic.c | 65 +-
drivers/of/fdt.c | 378 ++++---
drivers/of/of_private.h | 2 +
drivers/of/overlay.c | 8 +-
drivers/of/unittest.c | 6 +-
drivers/pci/hotplug/Kconfig | 12 +
drivers/pci/hotplug/Makefile | 3 +
drivers/pci/hotplug/pnv_php.c | 866
++++++++++++++++
drivers/pci/hotplug/rpadlpar_core.c | 8 +-
drivers/pci/hotplug/rpaphp_core.c | 4 +-
drivers/pci/hotplug/rpaphp_pci.c | 4 +-
drivers/pci/setup-bus.c | 5 +
include/linux/of_fdt.h | 5 +-
include/linux/pci.h | 1 +
38 files changed, 2389 insertions(+), 932 deletions(-)
create mode 100644 drivers/pci/hotplug/pnv_php.c
From: Daniel Axtens <hidden> Date: 2015-11-12 04:18:20
Looks good.
Will hold off on an official review until I can test the series.
Regards,
Daniel
Gavin Shan [off-list ref] writes:
quoted hunk
As we track M32 segment consumption, this introduces an array to
the PHB to track the mapping between M64 segment and PE number.
The information is going to be used to find M64 segment from the
PE number during PCI unplugging time in subsequent patches.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 10 ++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++-
2 files changed, 10 insertions(+), 3 deletions(-)
@@ -3103,6 +3104,8 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,/* Allocate aux data & arrays. We don't have IO ports on PHB3 */size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));+m64map_off=size;+size+=phb->ioda.total_pe_num*sizeof(phb->ioda.m64_segmap[0]);m32map_off=size;size+=phb->ioda.total_pe_num*sizeof(phb->ioda.m32_segmap[0]);if(phb->type==PNV_PHB_IODA1){
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
This is very minor, but is there a good reason to change the error
message from the one above to the one below? I just hesitate to change
error messages that people might be grepping the source for without a
good reason.
This is very minor, but is there a good reason to change the error
message from the one above to the one below? I just hesitate to change
error messages that people might be grepping the source for without a
good reason.
About 3 years ago, I think the error message printed by pr_warn() can't
exceed 80 lines each line. Otherwise, scripts/checkpatch.pl will report
warnings. The error message spans multiple lines to avoid that. However,
that turned to be wrong later. If people searchs the code from the error
or warning message, it'd better to keep it in one line, not in multiple
lines. That's the reason I merged them into one line since I have to
refactor the function. At same time, the message is shortened as "Error"
is shorter than "Failure" and "registers" in original message is meaningless.
On Thu, Nov 05, 2015 at 12:12:39AM +1100, Gavin Shan wrote:
In pnv_pci_reset_secondary_bus(), we should issue fundamental
reset if any one subordinate device of the specified is requesting
^^^^^^^^^^^^^^
the specified bus
I put the note reminding me to admend the changelog in next revision.
quoted hunk
that. Otherwise, the device might not come up after the reset.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
From: Daniel Axtens <hidden> Date: 2015-11-12 22:59:27
Gavin Shan [off-list ref] writes:
When pnv_pci_reset_secondary_bus() is called to issue reset on
the indicated secondary bus, the bus can't be root bus. So we
needn't consider root bus in the function.
It took me a while to convince myself that this is correct. For the
record, this is why it's correct:
pnv_pci_reset_secondary_bus fills the reset_secondary_bus callback in
the pci_controller_ops structure, and isn't used elsewhere.
In arch/powerpc/kernel/pci.c, that callback is called (if it exists) in
pcibios_reset_secondary_bus(). It's not called anywhere else.
The PPC pcibios_reset_secondary_bus overrides the weak version in
drivers/pci/pci.c. It's called from the same file by
pci_reset_bridge_secondary_device() (and nowhere else).
pci_reset_bridge_secondary_device() is nicely documented:
/**
* pci_reset_bridge_secondary_bus - Reset the secondary bus on a PCI bridge.
* @dev: Bridge device
*
* Use the bridge control register to assert reset on the secondary bus.
* Devices on the secondary bus are left in power-on state.
*/
Therefore, by the definiton of pci_reset_bridge_secondary_bus,
pnv_pci_reset_secondary_bus() can only be called with a bridge
device. As such, a bridge reset only is appropriate. If this breaks
anything, the caller is broken.
It might be worth including a condensed version of this in the commit
message.
Reviewed-by: Daniel Axtens <redacted>
Regards,
Daniel Axtens
@@ -880,16 +880,8 @@ static int pnv_eeh_bridge_reset(struct pci_dev *dev, int option)voidpnv_pci_reset_secondary_bus(structpci_dev*dev){-structpci_controller*hose;--if(pci_is_root_bus(dev->bus)){-hose=pci_bus_to_host(dev->bus);-pnv_eeh_root_reset(hose,EEH_RESET_HOT);-pnv_eeh_root_reset(hose,EEH_RESET_DEACTIVATE);-}else{-pnv_eeh_bridge_reset(dev,EEH_RESET_HOT);-pnv_eeh_bridge_reset(dev,EEH_RESET_DEACTIVATE);-}+pnv_eeh_bridge_reset(dev,EEH_RESET_HOT);+pnv_eeh_bridge_reset(dev,EEH_RESET_DEACTIVATE);}/**
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Nov 13, 2015 at 09:59:27AM +1100, Daniel Axtens wrote:
Gavin Shan [off-list ref] writes:
quoted
When pnv_pci_reset_secondary_bus() is called to issue reset on
the indicated secondary bus, the bus can't be root bus. So we
needn't consider root bus in the function.
It took me a while to convince myself that this is correct. For the
record, this is why it's correct:
pnv_pci_reset_secondary_bus fills the reset_secondary_bus callback in
the pci_controller_ops structure, and isn't used elsewhere.
In arch/powerpc/kernel/pci.c, that callback is called (if it exists) in
pcibios_reset_secondary_bus(). It's not called anywhere else.
The PPC pcibios_reset_secondary_bus overrides the weak version in
drivers/pci/pci.c. It's called from the same file by
pci_reset_bridge_secondary_device() (and nowhere else).
pci_reset_bridge_secondary_device() is nicely documented:
/**
* pci_reset_bridge_secondary_bus - Reset the secondary bus on a PCI bridge.
* @dev: Bridge device
*
* Use the bridge control register to assert reset on the secondary bus.
* Devices on the secondary bus are left in power-on state.
*/
Therefore, by the definiton of pci_reset_bridge_secondary_bus,
pnv_pci_reset_secondary_bus() can only be called with a bridge
device. As such, a bridge reset only is appropriate. If this breaks
anything, the caller is broken.
It might be worth including a condensed version of this in the commit
message.
Right. I'll add more description to the changelog in next revision.
@@ -880,16 +880,8 @@ static int pnv_eeh_bridge_reset(struct pci_dev *dev, int option)voidpnv_pci_reset_secondary_bus(structpci_dev*dev){-structpci_controller*hose;--if(pci_is_root_bus(dev->bus)){-hose=pci_bus_to_host(dev->bus);-pnv_eeh_root_reset(hose,EEH_RESET_HOT);-pnv_eeh_root_reset(hose,EEH_RESET_DEACTIVATE);-}else{-pnv_eeh_bridge_reset(dev,EEH_RESET_HOT);-pnv_eeh_bridge_reset(dev,EEH_RESET_DEACTIVATE);-}+pnv_eeh_bridge_reset(dev,EEH_RESET_HOT);+pnv_eeh_bridge_reset(dev,EEH_RESET_DEACTIVATE);}/**
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
According to the skiboot sources, fundamental reset isn't supported on
p5ioc2. As far as I can tell from your corresponding skiboot patches,
this is still the case after they are applied. Do we need a fallback to
EEH_RESET_HOT in this case? Otherwise there will be no reset performed
at all.
Likewise, if the FUNDAMENTAL reset fails for any reason, should we fall
back to a HOT reset?
Regards,
Daniel
According to the skiboot sources, fundamental reset isn't supported on
p5ioc2. As far as I can tell from your corresponding skiboot patches,
this is still the case after they are applied. Do we need a fallback to
EEH_RESET_HOT in this case? Otherwise there will be no reset performed
at all.
Likewise, if the FUNDAMENTAL reset fails for any reason, should we fall
back to a HOT reset?
P5IOC2 won't export any PCI slots. So kernel won't issue fundamental reset
to PCI buses on P5IOC2.
We had the failback: hot reset is picked if fundamental reset can't be
supported on the target PCI bus. In case fundamental reset fails, we
shouldn't go ahead try hot reset.
Thanks,
Gavin
From: Daniel Axtens <hidden> Date: 2015-11-13 00:23:27
Following some discussion on IRC, it looks like there are roughly 2
machines on the planet with skiboot and p5ioc2, so I'm not worried about
that any more.
I am still vaguely concerned about a failing fundamental reset.
Regards,
Daniel
This renames the fields related to PE number in "struct pnv_phb"
for better reflecting of their usages as Alexey suggested. No
logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 2 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 56 ++++++++++++++--------------
arch/powerpc/platforms/powernv/pci.c | 2 +-
arch/powerpc/platforms/powernv/pci.h | 4 +-
4 files changed, 32 insertions(+), 32 deletions(-)
@@ -134,7 +134,7 @@ static inline bool pnv_pci_is_mem_pref_64(unsigned long flags)staticvoidpnv_ioda_reserve_pe(structpnv_phb*phb,intpe_no){-if(!(pe_no>=0&&pe_no<phb->ioda.total_pe)){+if(!(pe_no>=0&&pe_no<phb->ioda.total_pe_num)){pr_warn("%s: Invalid PE %d on PHB#%x\n",__func__,pe_no,phb->hose->global_number);return;
@@ -154,8 +154,8 @@ static int pnv_ioda_alloc_pe(struct pnv_phb *phb)do{pe=find_next_zero_bit(phb->ioda.pe_alloc,-phb->ioda.total_pe,0);-if(pe>=phb->ioda.total_pe)+phb->ioda.total_pe_num,0);+if(pe>=phb->ioda.total_pe_num)returnIODA_INVALID_PE;}while(test_and_set_bit(pe,phb->ioda.pe_alloc));
@@ -209,13 +209,13 @@ static int pnv_ioda2_init_m64(struct pnv_phb *phb)*expectedtobe0orlastoneofPEcapabicity.*/r=&phb->hose->mem_resources[1];-if(phb->ioda.reserved_pe==0)+if(phb->ioda.reserved_pe_idx==0)r->start+=phb->ioda.m64_segsize;-elseif(phb->ioda.reserved_pe==(phb->ioda.total_pe-1))+elseif(phb->ioda.reserved_pe_idx==(phb->ioda.total_pe_num-1))r->end-=phb->ioda.m64_segsize;elsepr_warn(" Cannot strip M64 segment for reserved PE#%d\n",-phb->ioda.reserved_pe);+phb->ioda.reserved_pe_idx);return0;
@@ -284,7 +284,7 @@ static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all)returnIODA_INVALID_PE;/* Allocate bitmap */-size=_ALIGN_UP(phb->ioda.total_pe/8,sizeof(unsignedlong));+size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));pe_alloc=kzalloc(size,GFP_KERNEL);if(!pe_alloc){pr_warn("%s: Out of memory !\n",
@@ -465,7 +465,7 @@ static int pnv_ioda_get_pe_state(struct pnv_phb *phb, int pe_no)s64rc;/* Sanity check on PE number */-if(pe_no<0||pe_no>=phb->ioda.total_pe)+if(pe_no<0||pe_no>=phb->ioda.total_pe_num)returnOPAL_EEH_STOPPED_PERM_UNAVAIL;/*
@@ -1394,9 +1394,9 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)}else{mutex_lock(&phb->ioda.pe_alloc_mutex);*pdn->pe_num_map=bitmap_find_next_zero_area(-phb->ioda.pe_alloc,phb->ioda.total_pe,+phb->ioda.pe_alloc,phb->ioda.total_pe_num,0,num_vfs,0);-if(*pdn->pe_num_map>=phb->ioda.total_pe){+if(*pdn->pe_num_map>=phb->ioda.total_pe_num){mutex_unlock(&phb->ioda.pe_alloc_mutex);dev_info(&pdev->dev,"Failed to enable VF%d\n",num_vfs);kfree(pdn->pe_num_map);
@@ -3067,13 +3067,13 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,pr_err(" Failed to map registers !\n");/* Initialize more IODA stuff */-phb->ioda.total_pe=1;+phb->ioda.total_pe_num=1;prop32=of_get_property(np,"ibm,opal-num-pes",NULL);if(prop32)-phb->ioda.total_pe=be32_to_cpup(prop32);+phb->ioda.total_pe_num=be32_to_cpup(prop32);prop32=of_get_property(np,"ibm,opal-reserved-pe",NULL);if(prop32)-phb->ioda.reserved_pe=be32_to_cpup(prop32);+phb->ioda.reserved_pe_idx=be32_to_cpup(prop32);
It is not related to the patch but you initialize total_pe to 1 before
checking the device tree (which is ok) but you do not initialize
reserved_pe and I cannot find where @phb would be zeroed - it is allocated
by memblock_virt_alloc() which does not do that.
--
Alexey
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 86 +++++++++++++++++++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++
2 files changed, 86 insertions(+), 3 deletions(-)
@@ -325,6 +383,26 @@ static int pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)pe->master=master_pe;list_add_tail(&pe->list,&master_pe->slaves);}++/*+*P7IOCsupportsM64DT,whichhelpsmappingM64segment+*tooneparticularPE#.However,PHB3hasfixedmapping+*betweenM64segmentandPE#.Inordertohavesamelogic+*forP7IOCandPHB3,weenforcefixedmappingbetweenM64+*segmentandPE#onP7IOC.+*/+if(phb->type==PNV_PHB_IODA1){+int64_trc;++rc=opal_pci_map_pe_mmio_window(phb->opal_id,+pe->pe_number,OPAL_M64_WINDOW_TYPE,+pe->pe_number/PNV_IODA1_M64_SEGS,+pe->pe_number%PNV_IODA1_M64_SEGS);+if(rc!=OPAL_SUCCESS)+pr_warn("%s: Error %lld mapping M64 for PHB#%d-PE#%d\n",+__func__,rc,phb->hose->global_number,+pe->pe_number);+}}kfree(pe_alloc);
@@ -339,8 +417,7 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)constu32*r;u64pci_addr;-/* FIXME: Support M64 for P7IOC */-if(phb->type!=PNV_PHB_IODA2){+if(phb->type!=PNV_PHB_IODA1&&phb->type!=PNV_PHB_IODA2){pr_info(" Not support M64 window\n");return;}
@@ -373,7 +450,10 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)/* Use last M64 BAR to cover M64 window */phb->ioda.m64_bar_idx=15;-phb->init_m64=pnv_ioda2_init_m64;+if(phb->type==PNV_PHB_IODA1)+phb->init_m64=pnv_ioda1_init_m64;+else+phb->init_m64=pnv_ioda2_init_m64;phb->reserve_m64_pe=pnv_ioda_reserve_m64_pe;phb->pick_m64_pe=pnv_ioda_pick_m64_pe;
Nit: the callbacks initialization does not seem to relate to parsing any
window :) They could all go to where pnv_ioda_parse_m64_window() is called,
no separate patch is needed.
As we track M32 segment consumption, this introduces an array to
the PHB to track the mapping between M64 segment and PE number.
The information is going to be used to find M64 segment from the
PE number during PCI unplugging time in subsequent patches.
It would not hurt to put a few words about how we managed to live without
such a mapping for M64 before but we needed mapping for M32.
--
Alexey
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 86 +++++++++++++++++++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++
2 files changed, 86 insertions(+), 3 deletions(-)
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
I thought we decided (Ben suggested?) not to push P7IOC code now (or ever)
as there is no user for it, has this changed?
btw please put ioda1/ioda2/p7ioc/etc to the subject line to make it easier
to see how much work is there about particular PHB type. You rename quite
many functions and I generally want to ask you to group all renaming
patches first but it would also make sense to keep them close to (for
example) p7ioc-related patches so having more descriptive subject lines may
help. Thanks.
--
Alexey
From: Daniel Axtens <hidden> Date: 2015-11-17 00:29:26
Gavin Shan [off-list ref] writes:
Each PHB maintains an array helping to translate 2-bytes Request
ID (RID) to PE# with the assumption that PE# takes one byte, meaning
that we can't have more than 256 PEs. However, pci_dn->pe_number
already had 4-bytes for the PE#.
This extends the PE# capacity so that each of them will be 4-bytes
long. Then we can reuse IODA_INVALID_PE to check the PE# stored in
phb->pe_rmap[] is valid or not.
Just for clarity, could you make it clear in the commit message that
you're increasing the PE# capacity _in the PHB_? I just found it a bit
confusing the first time I read it.
With that clarified I'll be happy to add my reviewed-by tag.
Regards,
Daniel
@@ -162,11 +162,8 @@ struct pnv_phb {structlist_headpe_list;structmutexpe_list_mutex;-/* Reverse map of PEs, will have to extend if-*wearetosupportmorethan256PEs,indexed-*bus{bus,devfn}-*/-unsignedcharpe_rmap[0x10000];+/* Reverse map of PEs, indexed by {bus, devfn} */+intpe_rmap[0x10000];/* TCE cache invalidate registers (physical and*remapped)
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Nov 16, 2015 at 07:01:59PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
As we track M32 segment consumption, this introduces an array to
the PHB to track the mapping between M64 segment and PE number.
The information is going to be used to find M64 segment from the
PE number during PCI unplugging time in subsequent patches.
It would not hurt to put a few words about how we managed to live without
such a mapping for M64 before but we needed mapping for M32.
The M32 mapping (phb->ioda.m32_segmap[]) isn't used for anything before
this patcheset. There're no need for M64 mapping before this patchset
similarly, no need to add the words.
Thanks,
Gavin
In pnv_ioda_setup_dma(), it's unnecessary to calculate the DMA32
segments for PEs on PHB3 as the whole available DMA32 space can
be assigned to one specific PE on PHB3.
This splits pnv_ioda_setup_dma() to pnv_pci_ioda1_setup_dma() and
pnv_pci_ioda2_setup_dma() in order to avoid calculating DMA32
segments for PEs on PHB3. No logical changes introduced.
This patch is not needed as
[PATCH v7 20/50] powerpc/powernv: Improve DMA32 segment calculation
moves this calculation to another place (which already makes this patch
unnecessary) and
[PATCH v7 26/50] powerpc/powernv: Create PEs at PCI hot plugging time
removes just introduced pnv_pci_ioda1_setup_dma() - if you remove it, then
there is no point in fixing it in the first place.
On Mon, Nov 16, 2015 at 07:01:06PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
This renames the fields related to PE number in "struct pnv_phb"
for better reflecting of their usages as Alexey suggested. No
logical changes introduced.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/eeh-powernv.c | 2 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 56 ++++++++++++++--------------
arch/powerpc/platforms/powernv/pci.c | 2 +-
arch/powerpc/platforms/powernv/pci.h | 4 +-
4 files changed, 32 insertions(+), 32 deletions(-)
@@ -134,7 +134,7 @@ static inline bool pnv_pci_is_mem_pref_64(unsigned long flags)staticvoidpnv_ioda_reserve_pe(structpnv_phb*phb,intpe_no){-if(!(pe_no>=0&&pe_no<phb->ioda.total_pe)){+if(!(pe_no>=0&&pe_no<phb->ioda.total_pe_num)){pr_warn("%s: Invalid PE %d on PHB#%x\n",__func__,pe_no,phb->hose->global_number);return;
@@ -154,8 +154,8 @@ static int pnv_ioda_alloc_pe(struct pnv_phb *phb)do{pe=find_next_zero_bit(phb->ioda.pe_alloc,-phb->ioda.total_pe,0);-if(pe>=phb->ioda.total_pe)+phb->ioda.total_pe_num,0);+if(pe>=phb->ioda.total_pe_num)returnIODA_INVALID_PE;}while(test_and_set_bit(pe,phb->ioda.pe_alloc));
@@ -209,13 +209,13 @@ static int pnv_ioda2_init_m64(struct pnv_phb *phb)*expectedtobe0orlastoneofPEcapabicity.*/r=&phb->hose->mem_resources[1];-if(phb->ioda.reserved_pe==0)+if(phb->ioda.reserved_pe_idx==0)r->start+=phb->ioda.m64_segsize;-elseif(phb->ioda.reserved_pe==(phb->ioda.total_pe-1))+elseif(phb->ioda.reserved_pe_idx==(phb->ioda.total_pe_num-1))r->end-=phb->ioda.m64_segsize;elsepr_warn(" Cannot strip M64 segment for reserved PE#%d\n",-phb->ioda.reserved_pe);+phb->ioda.reserved_pe_idx);return0;
@@ -284,7 +284,7 @@ static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all)returnIODA_INVALID_PE;/* Allocate bitmap */-size=_ALIGN_UP(phb->ioda.total_pe/8,sizeof(unsignedlong));+size=_ALIGN_UP(phb->ioda.total_pe_num/8,sizeof(unsignedlong));pe_alloc=kzalloc(size,GFP_KERNEL);if(!pe_alloc){pr_warn("%s: Out of memory !\n",
@@ -465,7 +465,7 @@ static int pnv_ioda_get_pe_state(struct pnv_phb *phb, int pe_no)s64rc;/* Sanity check on PE number */-if(pe_no<0||pe_no>=phb->ioda.total_pe)+if(pe_no<0||pe_no>=phb->ioda.total_pe_num)returnOPAL_EEH_STOPPED_PERM_UNAVAIL;/*
@@ -1394,9 +1394,9 @@ int pnv_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)}else{mutex_lock(&phb->ioda.pe_alloc_mutex);*pdn->pe_num_map=bitmap_find_next_zero_area(-phb->ioda.pe_alloc,phb->ioda.total_pe,+phb->ioda.pe_alloc,phb->ioda.total_pe_num,0,num_vfs,0);-if(*pdn->pe_num_map>=phb->ioda.total_pe){+if(*pdn->pe_num_map>=phb->ioda.total_pe_num){mutex_unlock(&phb->ioda.pe_alloc_mutex);dev_info(&pdev->dev,"Failed to enable VF%d\n",num_vfs);kfree(pdn->pe_num_map);
@@ -3067,13 +3067,13 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,pr_err(" Failed to map registers !\n");/* Initialize more IODA stuff */-phb->ioda.total_pe=1;+phb->ioda.total_pe_num=1;prop32=of_get_property(np,"ibm,opal-num-pes",NULL);if(prop32)-phb->ioda.total_pe=be32_to_cpup(prop32);+phb->ioda.total_pe_num=be32_to_cpup(prop32);prop32=of_get_property(np,"ibm,opal-reserved-pe",NULL);if(prop32)-phb->ioda.reserved_pe=be32_to_cpup(prop32);+phb->ioda.reserved_pe_idx=be32_to_cpup(prop32);
It is not related to the patch but you initialize total_pe to 1 before
checking the device tree (which is ok) but you do not initialize reserved_pe
and I cannot find where @phb would be zeroed - it is allocated by
memblock_virt_alloc() which does not do that.
There is a call "memset(ptr, 0, size)" in memblock_virt_alloc_internal().
Thanks,
Gavin
On Mon, Nov 16, 2015 at 07:01:46PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 86 +++++++++++++++++++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++
2 files changed, 86 insertions(+), 3 deletions(-)
@@ -325,6 +383,26 @@ static int pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)pe->master=master_pe;list_add_tail(&pe->list,&master_pe->slaves);}++/*+*P7IOCsupportsM64DT,whichhelpsmappingM64segment+*tooneparticularPE#.However,PHB3hasfixedmapping+*betweenM64segmentandPE#.Inordertohavesamelogic+*forP7IOCandPHB3,weenforcefixedmappingbetweenM64+*segmentandPE#onP7IOC.+*/+if(phb->type==PNV_PHB_IODA1){+int64_trc;++rc=opal_pci_map_pe_mmio_window(phb->opal_id,+pe->pe_number,OPAL_M64_WINDOW_TYPE,+pe->pe_number/PNV_IODA1_M64_SEGS,+pe->pe_number%PNV_IODA1_M64_SEGS);+if(rc!=OPAL_SUCCESS)+pr_warn("%s: Error %lld mapping M64 for PHB#%d-PE#%d\n",+__func__,rc,phb->hose->global_number,+pe->pe_number);+}}kfree(pe_alloc);
@@ -339,8 +417,7 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)constu32*r;u64pci_addr;-/* FIXME: Support M64 for P7IOC */-if(phb->type!=PNV_PHB_IODA2){+if(phb->type!=PNV_PHB_IODA1&&phb->type!=PNV_PHB_IODA2){pr_info(" Not support M64 window\n");return;}
@@ -373,7 +450,10 @@ static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)/* Use last M64 BAR to cover M64 window */phb->ioda.m64_bar_idx=15;-phb->init_m64=pnv_ioda2_init_m64;+if(phb->type==PNV_PHB_IODA1)+phb->init_m64=pnv_ioda1_init_m64;+else+phb->init_m64=pnv_ioda2_init_m64;phb->reserve_m64_pe=pnv_ioda_reserve_m64_pe;phb->pick_m64_pe=pnv_ioda_pick_m64_pe;
Nit: the callbacks initialization does not seem to relate to parsing any
window :) They could all go to where pnv_ioda_parse_m64_window() is called,
no separate patch is needed.
Well, what's the benifit for that? I personally prefer the way I had: initialize
all callbacks in one place, not in separate places. However, if you have good
reason to support your suggestion, I'll change accordingly for sure.
On Mon, Nov 16, 2015 at 07:02:03PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 86 +++++++++++++++++++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 3 ++
2 files changed, 86 insertions(+), 3 deletions(-)
On Mon, Nov 16, 2015 at 07:02:18PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
This enables M64 window on P7IOC, which has been enabled on PHB3.
Different from PHB3 where 16 M64 BARs are supported and each of
them can be owned by one particular PE# exclusively or divided
evenly to 256 segments, every P7IOC PHB has 16 M64 BARs and each
of them are divided to 8 segments. So every P7IOC PHB supports
128 M64 segments in total. P7IOC has M64DT, which helps mapping
one particular M64 segment# to arbitrary PE#. PHB3 doesn't have
M64DT, indicating that one M64 segment can only be pinned to the
fixed PE#. In order to have same code to support M64 on P7IOC and
PHB3, we just provide 128 M64 segments on every P7IOC PHB and each
of them is pinned to the fixed PE# by bypassing the function of
M64DT. In turn, we just need different phb->init_m64() for P7IOC
and PHB3 to support M64.
I thought we decided (Ben suggested?) not to push P7IOC code now (or ever) as
there is no user for it, has this changed?
Remember that the code is mixed for P7IOC/PHB3. It's not harmful to support
M64 window on P7IOC, which is much larger than M32.
btw please put ioda1/ioda2/p7ioc/etc to the subject line to make it easier to
see how much work is there about particular PHB type. You rename quite many
functions and I generally want to ask you to group all renaming patches first
but it would also make sense to keep them close to (for example)
p7ioc-related patches so having more descriptive subject lines may help.
Thanks.
As the code is mixed for P7IOC/PHB3, I'm not following the line (IODA1/IODA2/p7ioc/phb3)
in this patchset. Instead, the sequence of patchset is order related to: cod refactoring,
IO/M32/M64, DMA, PE allocation/releaseing.
Thanks,
Gavin
PEs are put into PHB DMA32 list (phb->ioda.pe_dma_list) according
to their DMA32 weight. The PEs on the list are iterated to setup
their TCE32 tables at system booting time. The list is used for
once and there is no good reason for it to survive.
From the above I concluded that you need a list, just do not need to keep
after the configuration is done but in fact you remove the list completely
so just remove "to survive" (s/for it to survive/for keep having it/) :)
quoted hunk
This moves the logic calculating DMA32 weight of PHB and PE to
pnv_pci_ioda1_setup_dma() to drop PHB's DMA32 list.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 150 ++++++++++++++----------------
arch/powerpc/platforms/powernv/pci.h | 19 ----
2 files changed, 68 insertions(+), 101 deletions(-)
@@ -891,44 +891,6 @@ out:return0;}-staticvoidpnv_ioda_link_pe_by_weight(structpnv_phb*phb,-structpnv_ioda_pe*pe)-{-structpnv_ioda_pe*lpe;--list_for_each_entry(lpe,&phb->ioda.pe_dma_list,dma_link){-if(lpe->dma_weight<pe->dma_weight){-list_add_tail(&pe->dma_link,&lpe->dma_link);-return;-}-}-list_add_tail(&pe->dma_link,&phb->ioda.pe_dma_list);-}--staticunsignedintpnv_ioda_dma_weight(structpci_dev*dev)-{-/* This is quite simplistic. The "base" weight of a device-*is10.0meansnoDMAistobeaccountedforit.-*/--/* If it's a bridge, no DMA */-if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)-return0;--/* Reduce the weight of slow USB controllers */-if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||-dev->class==PCI_CLASS_SERIAL_USB_OHCI||-dev->class==PCI_CLASS_SERIAL_USB_EHCI)-return3;--/* Increase the weight of RAID (includes Obsidian) */-if((dev->class>>8)==PCI_CLASS_STORAGE_RAID)-return15;--/* Default */-return10;-}-#ifdef CONFIG_PCI_IOVstaticintpnv_pci_vf_resource_shift(structpci_dev*dev,intoffset){
@@ -1046,10 +1007,8 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)pe->flags|=(all?PNV_IODA_PE_BUS_ALL:PNV_IODA_PE_BUS);pe->pbus=bus;pe->pdev=NULL;-pe->tce32_seg=-1;pe->mve_number=-1;pe->rid=bus->busn_res.start<<8;-pe->dma_weight=0;if(all)pe_info(pe,"Secondary bus %d..%d associated with PE#%d\n",
@@ -1071,17 +1030,6 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)/* Put PE to the list */list_add_tail(&pe->list,&phb->ioda.pe_list);--/* Account for one DMA PE if at least one DMA capable device exist-*belowthebridge-*/-if(pe->dma_weight!=0){-phb->ioda.dma_weight+=pe->dma_weight;-phb->ioda.dma_pe_count++;-}--/* Link the PE */-pnv_ioda_link_pe_by_weight(phb,pe);}staticvoidpnv_ioda_setup_PEs(structpci_bus*bus)
@@ -1842,6 +1789,47 @@ static struct iommu_table_ops pnv_ioda2_iommu_ops = {.free=pnv_ioda2_table_free,};+staticintpnv_pci_ioda_dev_dma_weight(structpci_dev*dev,void*data)+{+unsignedint*weight=(unsignedint*)data;++/* This is quite simplistic. The "base" weight of a device+*is10.0meansnoDMAistobeaccountedforit.+*/++if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)+return0;++if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||+dev->class==PCI_CLASS_SERIAL_USB_OHCI||+dev->class==PCI_CLASS_SERIAL_USB_EHCI)+*weight+=3;+elseif((dev->class>>8)==PCI_CLASS_STORAGE_RAID)+*weight+=15;+else+*weight+=10;++return0;+}++staticunsignedintpnv_pci_ioda_pe_dma_weight(structpnv_ioda_pe*pe)+{+unsignedintweight=0;++if((pe->flags&PNV_IODA_PE_DEV)&&pe->pdev){+pnv_pci_ioda_dev_dma_weight(pe->pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS)&&pe->pbus){+structpci_dev*pdev;++list_for_each_entry(pdev,&pe->pbus->devices,bus_list)+pnv_pci_ioda_dev_dma_weight(pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS_ALL)&&pe->pbus){+pci_walk_bus(pe->pbus,pnv_pci_ioda_dev_dma_weight,&weight);+}++returnweight;+}+staticvoidpnv_pci_ioda1_setup_dma_pe(structpnv_phb*phb,structpnv_ioda_pe*pe,unsignedintbase,
@@ -1858,17 +1846,12 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,/* XXX FIXME: Provide 64-bit DMA facilities & non-4K TCE tables etc.. *//* XXX FIXME: Allocate multi-level tables on PHB3 */-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-tbl=pnv_pci_table_alloc(phb->hose->node);iommu_register_group(&pe->table_group,phb->hose->global_number,pe->pe_number);pnv_pci_link_table_and_group(phb->hose->node,0,tbl,&pe->table_group);/* Grab a 32-bit TCE table */-pe->tce32_seg=base;pe_info(pe," Setting up 32-bit TCE table at %08x..%08x\n",base*PNV_IODA1_DMA32_SEGSIZE,(base+segs)*PNV_IODA1_DMA32_SEGSIZE-1);
@@ -1932,8 +1915,6 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,return;fail:/* XXX Failure: Try to fallback to 64-bit only ? */-if(pe->tce32_seg>=0)-pe->tce32_seg=-1;if(tce_mem)__free_pages(tce_mem,get_order(tce32_segsz*segs));if(tbl){
@@ -2344,10 +2325,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,{int64_trc;-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-/* TVE #1 is selected by PCI address bit 59 */pe->tce_bypass_base=1ull<<59;
@@ -2355,7 +2332,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,pe->pe_number);/* The PE will reserve all possible 32-bits space */-pe->tce32_seg=0;pe_info(pe,"Setting up 32-bit TCE table at 0..%08x\n",phb->ioda.m32_pci_base);
@@ -2386,24 +2359,34 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,staticvoidpnv_pci_ioda1_setup_dma(structpnv_phb*phb){structpci_controller*hose=phb->hose;-unsignedintresidual,remaining,segs,tw,base;+unsignedintweight,total_weight,dma_pe_count;+unsignedintresidual,remaining,segs,base;structpnv_ioda_pe*pe;+total_weight=0;+dma_pe_count=0;+list_for_each_entry(pe,&phb->ioda.pe_list,list){+weight=pnv_pci_ioda_pe_dma_weight(pe);+if(weight>0)+dma_pe_count++;++total_weight+=weight;+}+/* If we have more PE# than segments available, hand out one*perPEuntilwerunoutandlettherestfail.Ifnot,*thenweassignatleastonesegmentperPE,plusmorebased*ontheamountofdevicesunderthatPE*/-if(phb->ioda.dma_pe_count>phb->ioda.tce32_count)+if(dma_pe_count>phb->ioda.tce32_count)residual=0;else-residual=phb->ioda.tce32_count--phb->ioda.dma_pe_count;+residual=phb->ioda.tce32_count-dma_pe_count;pr_info("PCI: Domain %04x has %ld available 32-bit DMA segments\n",hose->global_number,phb->ioda.tce32_count);pr_info("PCI: %d PE# for a total weight of %d\n",-phb->ioda.dma_pe_count,phb->ioda.dma_weight);+dma_pe_count,total_weight);pnv_pci_ioda_setup_opal_tce_kill(phb);
On Tue, Nov 17, 2015 at 11:29:26AM +1100, Daniel Axtens wrote:
Gavin Shan [off-list ref] writes:
quoted
Each PHB maintains an array helping to translate 2-bytes Request
ID (RID) to PE# with the assumption that PE# takes one byte, meaning
that we can't have more than 256 PEs. However, pci_dn->pe_number
already had 4-bytes for the PE#.
This extends the PE# capacity so that each of them will be 4-bytes
long. Then we can reuse IODA_INVALID_PE to check the PE# stored in
phb->pe_rmap[] is valid or not.
Just for clarity, could you make it clear in the commit message that
you're increasing the PE# capacity _in the PHB_? I just found it a bit
confusing the first time I read it.
With that clarified I'll be happy to add my reviewed-by tag.
Sure, will add it and thanks for your happiness :-)
On Tue, Nov 17, 2015 at 12:54:04PM +1100, Alexey Kardashevskiy wrote:
On 11/05/2015 12:12 AM, Gavin Shan wrote:
quoted
PEs are put into PHB DMA32 list (phb->ioda.pe_dma_list) according
to their DMA32 weight. The PEs on the list are iterated to setup
their TCE32 tables at system booting time. The list is used for
once and there is no good reason for it to survive.
From the above I concluded that you need a list, just do not need to keep
after the configuration is done but in fact you remove the list completely so
just remove "to survive" (s/for it to survive/for keep having it/) :)
Thanks & will change it accordingly in next revision :)
quoted
This moves the logic calculating DMA32 weight of PHB and PE to
pnv_pci_ioda1_setup_dma() to drop PHB's DMA32 list.
Signed-off-by: Gavin Shan <redacted>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 150 ++++++++++++++----------------
arch/powerpc/platforms/powernv/pci.h | 19 ----
2 files changed, 68 insertions(+), 101 deletions(-)
@@ -891,44 +891,6 @@ out:return0;}-staticvoidpnv_ioda_link_pe_by_weight(structpnv_phb*phb,-structpnv_ioda_pe*pe)-{-structpnv_ioda_pe*lpe;--list_for_each_entry(lpe,&phb->ioda.pe_dma_list,dma_link){-if(lpe->dma_weight<pe->dma_weight){-list_add_tail(&pe->dma_link,&lpe->dma_link);-return;-}-}-list_add_tail(&pe->dma_link,&phb->ioda.pe_dma_list);-}--staticunsignedintpnv_ioda_dma_weight(structpci_dev*dev)-{-/* This is quite simplistic. The "base" weight of a device-*is10.0meansnoDMAistobeaccountedforit.-*/--/* If it's a bridge, no DMA */-if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)-return0;--/* Reduce the weight of slow USB controllers */-if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||-dev->class==PCI_CLASS_SERIAL_USB_OHCI||-dev->class==PCI_CLASS_SERIAL_USB_EHCI)-return3;--/* Increase the weight of RAID (includes Obsidian) */-if((dev->class>>8)==PCI_CLASS_STORAGE_RAID)-return15;--/* Default */-return10;-}-#ifdef CONFIG_PCI_IOVstaticintpnv_pci_vf_resource_shift(structpci_dev*dev,intoffset){
@@ -1046,10 +1007,8 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)pe->flags|=(all?PNV_IODA_PE_BUS_ALL:PNV_IODA_PE_BUS);pe->pbus=bus;pe->pdev=NULL;-pe->tce32_seg=-1;pe->mve_number=-1;pe->rid=bus->busn_res.start<<8;-pe->dma_weight=0;if(all)pe_info(pe,"Secondary bus %d..%d associated with PE#%d\n",
@@ -1071,17 +1030,6 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)/* Put PE to the list */list_add_tail(&pe->list,&phb->ioda.pe_list);--/* Account for one DMA PE if at least one DMA capable device exist-*belowthebridge-*/-if(pe->dma_weight!=0){-phb->ioda.dma_weight+=pe->dma_weight;-phb->ioda.dma_pe_count++;-}--/* Link the PE */-pnv_ioda_link_pe_by_weight(phb,pe);}staticvoidpnv_ioda_setup_PEs(structpci_bus*bus)
@@ -1842,6 +1789,47 @@ static struct iommu_table_ops pnv_ioda2_iommu_ops = {.free=pnv_ioda2_table_free,};+staticintpnv_pci_ioda_dev_dma_weight(structpci_dev*dev,void*data)+{+unsignedint*weight=(unsignedint*)data;++/* This is quite simplistic. The "base" weight of a device+*is10.0meansnoDMAistobeaccountedforit.+*/++if(dev->hdr_type!=PCI_HEADER_TYPE_NORMAL)+return0;++if(dev->class==PCI_CLASS_SERIAL_USB_UHCI||+dev->class==PCI_CLASS_SERIAL_USB_OHCI||+dev->class==PCI_CLASS_SERIAL_USB_EHCI)+*weight+=3;+elseif((dev->class>>8)==PCI_CLASS_STORAGE_RAID)+*weight+=15;+else+*weight+=10;++return0;+}++staticunsignedintpnv_pci_ioda_pe_dma_weight(structpnv_ioda_pe*pe)+{+unsignedintweight=0;++if((pe->flags&PNV_IODA_PE_DEV)&&pe->pdev){+pnv_pci_ioda_dev_dma_weight(pe->pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS)&&pe->pbus){+structpci_dev*pdev;++list_for_each_entry(pdev,&pe->pbus->devices,bus_list)+pnv_pci_ioda_dev_dma_weight(pdev,&weight);+}elseif((pe->flags&PNV_IODA_PE_BUS_ALL)&&pe->pbus){+pci_walk_bus(pe->pbus,pnv_pci_ioda_dev_dma_weight,&weight);+}++returnweight;+}+staticvoidpnv_pci_ioda1_setup_dma_pe(structpnv_phb*phb,structpnv_ioda_pe*pe,unsignedintbase,
@@ -1858,17 +1846,12 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,/* XXX FIXME: Provide 64-bit DMA facilities & non-4K TCE tables etc.. *//* XXX FIXME: Allocate multi-level tables on PHB3 */-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-tbl=pnv_pci_table_alloc(phb->hose->node);iommu_register_group(&pe->table_group,phb->hose->global_number,pe->pe_number);pnv_pci_link_table_and_group(phb->hose->node,0,tbl,&pe->table_group);/* Grab a 32-bit TCE table */-pe->tce32_seg=base;pe_info(pe," Setting up 32-bit TCE table at %08x..%08x\n",base*PNV_IODA1_DMA32_SEGSIZE,(base+segs)*PNV_IODA1_DMA32_SEGSIZE-1);
@@ -1932,8 +1915,6 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,return;fail:/* XXX Failure: Try to fallback to 64-bit only ? */-if(pe->tce32_seg>=0)-pe->tce32_seg=-1;if(tce_mem)__free_pages(tce_mem,get_order(tce32_segsz*segs));if(tbl){
@@ -2344,10 +2325,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,{int64_trc;-/* We shouldn't already have a 32-bit DMA associated */-if(WARN_ON(pe->tce32_seg>=0))-return;-/* TVE #1 is selected by PCI address bit 59 */pe->tce_bypass_base=1ull<<59;
@@ -2355,7 +2332,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,pe->pe_number);/* The PE will reserve all possible 32-bits space */-pe->tce32_seg=0;pe_info(pe,"Setting up 32-bit TCE table at 0..%08x\n",phb->ioda.m32_pci_base);
@@ -2386,24 +2359,34 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,staticvoidpnv_pci_ioda1_setup_dma(structpnv_phb*phb){structpci_controller*hose=phb->hose;-unsignedintresidual,remaining,segs,tw,base;+unsignedintweight,total_weight,dma_pe_count;+unsignedintresidual,remaining,segs,base;structpnv_ioda_pe*pe;+total_weight=0;+dma_pe_count=0;+list_for_each_entry(pe,&phb->ioda.pe_list,list){+weight=pnv_pci_ioda_pe_dma_weight(pe);+if(weight>0)+dma_pe_count++;++total_weight+=weight;+}+/* If we have more PE# than segments available, hand out one*perPEuntilwerunoutandlettherestfail.Ifnot,*thenweassignatleastonesegmentperPE,plusmorebased*ontheamountofdevicesunderthatPE*/-if(phb->ioda.dma_pe_count>phb->ioda.tce32_count)+if(dma_pe_count>phb->ioda.tce32_count)residual=0;else-residual=phb->ioda.tce32_count--phb->ioda.dma_pe_count;+residual=phb->ioda.tce32_count-dma_pe_count;pr_info("PCI: Domain %04x has %ld available 32-bit DMA segments\n",hose->global_number,phb->ioda.tce32_count);pr_info("PCI: %d PE# for a total weight of %d\n",-phb->ioda.dma_pe_count,phb->ioda.dma_weight);+dma_pe_count,total_weight);pnv_pci_ioda_setup_opal_tce_kill(phb);