On Thu, Nov 24 2022 at 00:50, Frank Li wrote:

quoted

┌────────────┐   ┌───────────────

────────────────────┐   ┌────────
────────┐

quoted

│            │   │                                   │   │                │
│            │   │ PCI Endpoint                      │   │ PCI Host       │
│            │   │                                   │   │                │
│            │◄──┤ 1.platform_msi_domain_alloc_irqs()│   │                │
│            │   │                                   │   │                │
│ MSI        ├──►│ 2.write_msi_msg()                 ├──►├─BAR<n>

│

quoted

│ Controller │   │   update doorbell register address│   │                │
│            │   │   for BAR                         │   │                │
│            │   │                                   │   │ 3. Write BAR<n>│
│            │◄──┼──────────────────────

─────────────┼───┤                │

quoted

│            │   │                                   │   │                │
│            ├──►│ 4.Irq Handle                      │   │                │
│            │   │                                   │   │                │
│            │   │                                   │   │                │
└────────────┘   └───────────────

────────────────────┘   └────────
────────┘

quoted

Using platform MSI interrupt controller as endpoint(EP)'s doorbell.

Can you please explain what the MSI controller is in this picture? MSI
controller is not a term which is common in the interrupt handling
landscape despite the fact that it's pretty wide spread in device tree
bindings presumably through intensive copy & pasta cargo cult.

quoted

Basic working follow as
1. EP function driver call platform_msi_domain_alloc_irqs() alloc a
MSI irq from MSI controller with call back function write_msi_msg();
2. write_msg_msg will config BAR and map to address defined in msi_msg;
3. Host side trigger an IRQ at Endpoint by write to BAR region.

You're explaining what the code does, but fail to explain the underlying
mechanisms.

Platform MSI is definitely the wrong mechanism here. Why?

This is about a PCIe endpoint, which is usually handled by a PCI/MSI
interrupt domain. Obviously this usage does not fit into the way how the
"global" PCI/MSI domains work.

There is upcoming work and at least the generic parts should show up in
6.2 which addresses exactly the problem you are trying to solve:

@tglx

I finally get a platform, ls1028, which use gic its and also support PCIe EP function.
I port my code into linux-next, (next-20230105) and understand your comments.

Its need a device ID to allocate msi irq for each devices.  

Epf device is dynamic created, where are field epc,  epc.parent point to PCIe EP
Controller, which have been probed by device-tree. 

I use a workaround to get msi irq number. 
	dev->of_node=  ntb->efp->epc->dev.parent->of_node;
	Platform_msi_domain_alloc_irqs(dev, ...);

Irq-gic-v3-its-pci-msi use of_node to get a ID. 

You said PCI/IMS can resolve this problem.  But I still not figure out it yet.
Can you give me sample code, which use PCI/IMS or provide a pseudo code?

Frank 

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
ernel.org%2Fall%2F20221124225331.464480443%40linutronix.de&data=0
5%7C01%7CFrank.Li%40nxp.com%7C6a07e33e56af45ffc1ff08dad174d02d%7C
686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638052596904953006%7C
Unknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Q8jreVGG
La2M4yhjGO7Njqwdm59XDC0GyLEwkr0k6B0%3D&reserved=0

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
ernel.org%2Fall%2F20221124230505.073418677%40linutronix.de&data=0
5%7C01%7CFrank.Li%40nxp.com%7C6a07e33e56af45ffc1ff08dad174d02d%7C
686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638052596904953006%7C
Unknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Tc9pXNJ49
9ETFgNWQBNLViFk8D5GbvrrwYDlBW%2Bf2qg%3D&reserved=0

plus the prove that the platform MSI mess can be replaced by this:


https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
ernel.org%2Fall%2F20221121135653.208611233%40linutronix.de&data=0
5%7C01%7CFrank.Li%40nxp.com%7C6a07e33e56af45ffc1ff08dad174d02d%7C
686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638052596904953006%7C
Unknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=R5KNVfcG
qxoCam%2FYhY57ihsloWGhGLM3Kh9IkyME4lk%3D&reserved=0

NTB in it's current form should never have happened, but that's water
down the bridge.

What you really want is:

  1) Convert your platform to the new MSI parent model

  2) Utilize PCI/IMS which is giving you exactly what you need with
     proper PCI semantics

Thanks,

        tglx

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel