Re: [PATCH kernel 2/2] powerpc/powernv: Define PHB4 type and enable sketchy bypass on POWER9

On Sat, 16 Jun 2018 11:05:19 +1000
Benjamin Herrenschmidt [off-list ref] wrote:

On Fri, 2018-06-01 at 18:10 +1000, Alexey Kardashevskiy wrote:

quoted

These are found in POWER9 chips. Right now these PHBs have unknown type
so changing it to PHB4 won't make much of a difference except enabling
sketchy bypass for POWER9 as this does below.  

And that will break on multi-chip systems since P9 doesn't have the
memory contiguous (it has the chip ID in the top bits).

This did not break mine and it is hard to see why it would break at all
if we use 1G pages and the maximum we need to cover is 48 bits (this
is what we are trying to support here - all these gpus, right?), or is
it more now? If so, I have posted v2 of tce multilevel dynamic
allocation which helps with enormous tce tables.

Russell is working on a different implementation that should be much
more imune to the system physical memory layout.

quoted

Signed-off-by: Alexey Kardashevskiy <redacted>
---
 arch/powerpc/platforms/powernv/pci.h      | 1 +
 arch/powerpc/platforms/powernv/pci-ioda.c | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index eada4b6..1408247 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h

@@ -23,6 +23,7 @@ enum pnv_phb_model {
 	PNV_PHB_MODEL_UNKNOWN,
 	PNV_PHB_MODEL_P7IOC,
 	PNV_PHB_MODEL_PHB3,
+	PNV_PHB_MODEL_PHB4,
 	PNV_PHB_MODEL_NPU,
 	PNV_PHB_MODEL_NPU2,
 };

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 9239142..66c2804 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c

@@ -1882,7 +1882,8 @@ static int pnv_pci_ioda_dma_set_mask(struct pci_dev *pdev, u64 dma_mask)
 		if (dma_mask >> 32 &&
 		    dma_mask > (memory_hotplug_max() + (1ULL << 32)) &&
 		    pnv_pci_ioda_pe_single_vendor(pe) &&
-		    phb->model == PNV_PHB_MODEL_PHB3) {
+		    (phb->model == PNV_PHB_MODEL_PHB3 ||
+		     phb->model == PNV_PHB_MODEL_PHB4)) {
 			/* Configure the bypass mode */
 			rc = pnv_pci_ioda_dma_64bit_bypass(pe);
 			if (rc)

@@ -3930,6 +3931,8 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 		phb->model = PNV_PHB_MODEL_P7IOC;
 	else if (of_device_is_compatible(np, "ibm,power8-pciex"))
 		phb->model = PNV_PHB_MODEL_PHB3;
+	else if (of_device_is_compatible(np, "ibm,power9-pciex"))
+		phb->model = PNV_PHB_MODEL_PHB4;
 	else if (of_device_is_compatible(np, "ibm,power8-npu-pciex"))
 		phb->model = PNV_PHB_MODEL_NPU;
 	else if (of_device_is_compatible(np, "ibm,power9-npu-pciex"))  

--
Alexey