Re: [PATCH 02/10] PCI: designware-ep: Add the doorbell mode of MSI-X in EP mode

Hi,

On 16/08/19 8:28 AM, Xiaowei Bao wrote:

quoted

-----Original Message-----
From: Andrew Murray <redacted>
Sent: 2019年8月15日 19:54
To: Xiaowei Bao <redacted>
Cc: jingoohan1@gmail.com; gustavo.pimentel@synopsys.com;
bhelgaas@google.com; robh+dt@kernel.org; mark.rutland@arm.com;
shawnguo@kernel.org; Leo Li [off-list ref]; kishon@ti.com;
lorenzo.pieralisi@arm.com; arnd@arndb.de; gregkh@linuxfoundation.org;
M.h. Lian [off-list ref]; Mingkai Hu [off-list ref];
Roy Zang [off-list ref]; linux-pci@vger.kernel.org;
devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
linux-arm-kernel@lists.infradead.org; linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH 02/10] PCI: designware-ep: Add the doorbell mode of
MSI-X in EP mode

On Thu, Aug 15, 2019 at 04:37:08PM +0800, Xiaowei Bao wrote:

quoted

Add the doorbell mode of MSI-X in EP mode.

Signed-off-by: Xiaowei Bao <redacted>
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 14 ++++++++++++++
 drivers/pci/controller/dwc/pcie-designware.h    | 14 ++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c

b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 75e2955..e3a7cdf 100644

--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c

@@ -454,6 +454,20 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep

*ep, u8 func_no,

quoted

 	return 0;
 }

+int dw_pcie_ep_raise_msix_irq_doorbell(struct dw_pcie_ep *ep, u8

func_no,

quoted

+				       u16 interrupt_num)
+{
+	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
+	u32 msg_data;
+
+	msg_data = (func_no << PCIE_MSIX_DOORBELL_PF_SHIFT) |
+		   (interrupt_num - 1);
+
+	dw_pcie_writel_dbi(pci, PCIE_MSIX_DOORBELL, msg_data);
+
+	return 0;
+}
+
 int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 			      u16 interrupt_num)

Have I understood correctly that the hardware provides an alternative
mechanism that allows for raising MSI-X interrupts without the bother of
reading the capabilities registers?

Yes, the hardware provide two way to MSI-X, please check the page 492 of 
DWC_pcie_dm_registers_4.30 Menu.
MSIX_DOORBELL_OFF on page 492 0x948 Description: MSI-X Doorbell Register....>

quoted

If so is there any good reason to keep dw_pcie_ep_raise_msix_irq? (And thus
use it in dw_plat_pcie_ep_raise_irq also)?

I am not sure, but I think the dw_pcie_ep_raise_msix_irq function is not correct, 
because I think we can't get the MSIX table from the address ep->phys_base + tbl_addr, 
but I also don't know where I can get the correct MSIX table.

Sometime back when I tried raising MSI-X from EP, it was failing. It's quite
possible dw_pcie_ep_raise_msix_irq function is not correct.

MSI-X table can be obtained from the inbound ATU corresponding to the MSIX bar.
IMO MSI-X support in EP mode needs rework. For instance set_msix should also
take BAR number as input to be configured in the MSI-X capability. The function
driver (pci-epf-test.c) should allocate memory taking into account the MSI-X table.

Thanks
Kishon