On Fri, Mar 07, 2025 at 10:32:39AM -0500, Frank Li wrote:

On Tue, Mar 04, 2025 at 05:25:45PM -0500, Frank Li wrote:

quoted

On Tue, Mar 04, 2025 at 05:11:54PM -0500, Frank Li wrote:

quoted

On Tue, Mar 04, 2025 at 11:50:10AM -0600, Bjorn Helgaas wrote:

quoted

On Mon, Mar 03, 2025 at 04:57:29PM -0500, Frank Li wrote:

quoted

On Wed, Feb 26, 2025 at 06:23:26PM -0600, Bjorn Helgaas wrote:

quoted

On Tue, Jan 28, 2025 at 05:07:36PM -0500, Frank Li wrote:

quoted

Introduce `parent_bus_offset` in `resource_entry` and a new API,
`pci_add_resource_parent_bus_offset()`, to provide necessary information
for PCI controllers with address translation units.

Typical PCI data flow involves:
  CPU (CPU address) -> Bus Fabric (Intermediate address) ->
  PCI Controller (PCI bus address) -> PCI Bus.

While most bus fabrics preserve address consistency, some modify addresses
to intermediate values. The `parent_bus_offset` enables PCI controllers to
translate these intermediate addresses correctly to PCI bus addresses.

Pave the road to remove hardcoded cpu_addr_fixup() and similar patterns in
PCI controller drivers.
...

quoted

+++ b/drivers/pci/of.c

@@ -402,7 +402,17 @@ static int devm_of_pci_get_host_bridge_resources(struct device *dev,
 			res->flags &= ~IORESOURCE_MEM_64;
 		}

-		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
+		/*
+		 * IORESOURCE_IO res->start is io space start address.
+		 * IORESOURCE_MEM res->start is cpu start address, which is the
+		 * same as range.cpu_addr.
+		 *
+		 * Use (range.cpu_addr - range.parent_bus_addr) to align both
+		 * IO and MEM's parent_bus_offset always offset to cpu address.
+		 */
+
+		pci_add_resource_parent_bus_offset(resources, res, res->start - range.pci_addr,
+						   range.cpu_addr - range.parent_bus_addr);

I don't know exactly where it needs to go, but I think we can call
.cpu_addr_fixup() once at startup on the base of the region.  This
will tell us the offset that applies to the entire region, i.e.,
parent_bus_offset.

Then we can remove all the .cpu_addr_fixup() calls in
cdns_pcie_host_init_address_translation(),
cdns_pcie_set_outbound_region(), and dw_pcie_prog_outbound_atu().

Until we can get rid of all the .cpu_addr_fixup() implementations,
We'll still have that single call at startup (I guess once for cadence
and another for designware), but it should simplify the current
callers quite a bit.

I don't think it can simple code. cdns_pcie_set_outbound_region() and
dw_pcie_prog_outbound_atu() are called by EP functions, which have not use
"resource" to manage outbound windows.

Let's ignore cadence for now.  I don't think we need to solve that
until later.

dw_pcie_prog_outbound_atu() is called by:

  - dw_pcie_other_conf_map_bus(): atu.parent_bus_addr = pp->cfg0_base

    I think dw_pcie_host_init() can set pp->cfg0_base with the correct
    intermediate address, either via the the of_property_read_reg() or
    .cpu_addr_fixup().

And chicken and egg problem here for artpec6_pcie_cpu_addr_fixup(), which
need cfg0_base. But try to use .cpu_addr_fixup() to get cfg0_base's
intermediate address.

Bjorn:
	Do you have chance to check my reply? some dwc platform driver
.cpu_addr_fixup() implement have dependence with old initilize sequency.

Yes, I tried to sketch out what I was thinking to make it more
concrete.  I posted it and sent to you, but just for other readers of
this thread, it's at:

https://lore.kernel.org/linux-pci/20250307233744.440476-1-helgaas@kernel.org/T/#t (local)

Bjorn