Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
at Documentation/devicetree/bindings/pci/hisilicon,kirin-pcie.yaml
IMO, the best would be to merge this series via your tree, as it
depends on the patch converting the DT bindings for the PCIe DWC
driver.
v3:
- Fixed a comment on patch 3: The Ethernet controller is at lane 6.
v2:
- removed the DTS file. I'll submit it in separate, once having
everything else merged;
- it now doesn't produce any warnings with:
make DT_SCHEMA_FILES=Documentation/devicetree/bindings/pci/hisilicon,kirin
-pcie.yaml DT_CHECKER_FLAGS=-m dt_binding_check
- added the upstream node;
- the clock enable now uses a new property (hisilicon,clken-gpios);
- the reg for the PCI devices are now properly filled;
- the pcie@x,y nodes now match the port number from table 4-1 from the
datasheet.
Mauro Carvalho Chehab (4):
dt-bindings: PCI: kirin: Fix compatible string
dt-bindings: PCI: kirin: Convert kirin-pcie.txt to yaml
dt-bindings: PCI: kirin: Add support for Kirin970
dt-bindings: phy: Add bindings for HiKey 970 PCIe PHY
.../bindings/pci/hisilicon,kirin-pcie.yaml | 160 ++++++++++++++++++
.../devicetree/bindings/pci/kirin-pcie.txt | 50 ------
.../devicetree/bindings/pci/snps,dw-pcie.yaml | 2 +-
.../phy/hisilicon,phy-hi3670-pcie.yaml | 86 ++++++++++
MAINTAINERS | 2 +-
5 files changed, 248 insertions(+), 52 deletions(-)
create mode 100644 Documentation/devicetree/bindings/pci/hisilicon,kirin-pcie.yaml
delete mode 100644 Documentation/devicetree/bindings/pci/kirin-pcie.txt
create mode 100644 Documentation/devicetree/bindings/phy/hisilicon,phy-hi3670-pcie.yaml
--
2.31.1
The pcie-kirin driver doesn't declare a hisilicon,kirin-pcie.
Also, remove the useless comment after the description, as other
compat will be supported by the same driver in the future.
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/devicetree/bindings/pci/kirin-pcie.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
@@ -9,7 +9,7 @@ Additional properties are described here: Required properties - compatible:- "hisilicon,kirin960-pcie" for PCIe of Kirin960 SoC+ "hisilicon,kirin960-pcie" - reg: Should contain rc_dbi, apb, phy, config registers location and length. - reg-names: Must include the following entries: "dbi": controller configuration registers;
Add a new compatible, plus the new bindings needed by
HiKey970 board.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
.../bindings/pci/hisilicon,kirin-pcie.yaml | 76 ++++++++++++++++++-
1 file changed, 75 insertions(+), 1 deletion(-)
@@ -24,11 +24,12 @@ properties:contains:enum:-hisilicon,kirin960-pcie+-hisilicon,kirin970-pciereg:description:|Should contain dbi, apb, config registers location and length.-For HiKey960, it should also contain phy.+For hisilicon,kirin960-pcie, it should also contain phy.minItems:3maxItems:4
@@ -36,6 +37,11 @@ properties:minItems:3maxItems:4+hisilicon,clken-gpios:+description:|+Clock input enablement GPIOs from PCI devices like Ethernet, M.2 and+mini-PCIe slots.+required:-compatible-reg
From: Rob Herring <robh+dt@kernel.org> Date: 2021-08-03 22:11:59
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
at Documentation/devicetree/bindings/pci/hisilicon,kirin-pcie.yaml
IMO, the best would be to merge this series via your tree, as it
depends on the patch converting the DT bindings for the PCIe DWC
driver.
From: Rob Herring <robh@kernel.org> Date: 2021-08-03 22:22:59
On Tue, 03 Aug 2021 06:38:55 +0200, Mauro Carvalho Chehab wrote:
The pcie-kirin driver doesn't declare a hisilicon,kirin-pcie.
Also, remove the useless comment after the description, as other
compat will be supported by the same driver in the future.
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/devicetree/bindings/pci/kirin-pcie.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
From: Rob Herring <robh@kernel.org> Date: 2021-08-04 16:29:05
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
In the mean time, I applied the series but haven't pushed it out.
Rob
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I added a printk on both pci_set_*of_node() functions:
[ 4.872991] (null): pci_set_bus_of_node: of_node: /soc/pcie@f4000000
[ 4.913806] (null): pci_set_of_node: of_node: /soc/pcie@f4000000
[ 4.978102] pci_bus 0000:01: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0
[ 4.990622] (null): pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0
[ 5.052383] pci_bus 0000:02: pci_set_bus_of_node: of_node: (null)
[ 5.059263] (null): pci_set_of_node: of_node: (null)
[ 5.085552] (null): pci_set_of_node: of_node: (null)
[ 5.112073] (null): pci_set_of_node: of_node: (null)
[ 5.138320] (null): pci_set_of_node: of_node: (null)
[ 5.164673] (null): pci_set_of_node: of_node: (null)
[ 5.233759] pci_bus 0000:03: pci_set_bus_of_node: of_node: (null)
[ 5.240539] (null): pci_set_of_node: of_node: (null)
[ 5.310545] pci_bus 0000:04: pci_set_bus_of_node: of_node: (null)
[ 5.324719] pci_bus 0000:05: pci_set_bus_of_node: of_node: (null)
[ 5.338914] pci_bus 0000:06: pci_set_bus_of_node: of_node: (null)
[ 5.345516] (null): pci_set_of_node: of_node: (null)
[ 5.415795] pci_bus 0000:07: pci_set_bus_of_node: of_node: (null)
It sounds that the parent is missing when pci_set_bus_of_node() is
called on some places. I'll try to identify why.
Thanks,
Mauro
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
It sounds that the parent is missing when pci_set_bus_of_node() is
called on some places. I'll try to identify why.
Thanks,
Mauro
Thanks,
Mauro
[PATCH] pci: setup PCI before setting the OF node
With this change, it is easier to add a debug printk at
pci_set_of_node() in order to address possible issues.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
From: Rob Herring <robh@kernel.org> Date: 2021-08-06 16:23:53
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
Rob
Em Fri, 6 Aug 2021 10:23:35 -0600
Rob Herring [off-list ref] escreveu:
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
From: Rob Herring <robh@kernel.org> Date: 2021-08-10 13:45:07
On Tue, Aug 10, 2021 at 3:42 AM Mauro Carvalho Chehab
[off-list ref] wrote:
Em Fri, 6 Aug 2021 10:23:35 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
Adding a child pcie@0 produces the following output from my debug
patches:
You removed your changes to the PCI code other than the debug print?
Em Tue, 10 Aug 2021 07:44:50 -0600
Rob Herring [off-list ref] escreveu:
On Tue, Aug 10, 2021 at 3:42 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Fri, 6 Aug 2021 10:23:35 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
Adding a child pcie@0 produces the following output from my debug
patches:
You removed your changes to the PCI code other than the debug print?
While technically correct having the bus# in the address, that doesn't
work for FDT since we don't know the bus assignment. So we should just
use 0.
Using 0 causes DTB compilation to produce a warning, due to the
bus-range. Without the bus-range, there will be runtime warnings,
as this will be assigned as bus 1.
These 3 nodes (1, 5, 7) need to be child nodes of the above node.
quoted
reg = <0x010800 0 0 0 0>;
Just 0x800
The same applies here and to all the other nodes: they all need to have
the bus number on it, as otherwise either DTB compilation warnings
are generated, or runtime ones are produced, like:
[ 4.986196] kirin-pcie f4000000.pcie: PCI host bridge to bus 0000:00
[ 4.992572] pci_bus 0000:00: root bus resource [bus 00-01]
...
[ 5.065566] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-01] (conflicts with (null) [bus 00-01])
From: Rob Herring <robh@kernel.org> Date: 2021-08-10 17:14:05
On Tue, Aug 10, 2021 at 8:21 AM Mauro Carvalho Chehab
[off-list ref] wrote:
Em Tue, 10 Aug 2021 07:44:50 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Tue, Aug 10, 2021 at 3:42 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Fri, 6 Aug 2021 10:23:35 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
Adding a child pcie@0 produces the following output from my debug
patches:
You removed your changes to the PCI code other than the debug print?
While technically correct having the bus# in the address, that doesn't
work for FDT since we don't know the bus assignment. So we should just
use 0.
Using 0 causes DTB compilation to produce a warning, due to the
bus-range. Without the bus-range, there will be runtime warnings,
as this will be assigned as bus 1.
From: Rob Herring <robh@kernel.org> Date: 2021-08-10 18:09:55
On Tue, Aug 10, 2021 at 11:13 AM Rob Herring [off-list ref] wrote:
On Tue, Aug 10, 2021 at 8:21 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Tue, 10 Aug 2021 07:44:50 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Tue, Aug 10, 2021 at 3:42 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Fri, 6 Aug 2021 10:23:35 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Thu, Aug 5, 2021 at 1:58 AM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Em Thu, 5 Aug 2021 09:46:12 +0200
Mauro Carvalho Chehab [off-list ref] escreveu:
quoted
Em Wed, 4 Aug 2021 10:28:53 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Wed, Aug 04, 2021 at 08:50:45AM +0200, Mauro Carvalho Chehab wrote:
quoted
Em Tue, 3 Aug 2021 16:11:42 -0600
Rob Herring [off-list ref] escreveu:
quoted
On Mon, Aug 2, 2021 at 10:39 PM Mauro Carvalho Chehab
[off-list ref] wrote:
quoted
Hi Rob,
That's the third version of the DT bindings for Kirin 970 PCIE and its
corresponding PHY.
It is identical to v2, except by:
- pcie@7,0 { // Lane 7: Ethernet
+ pcie@7,0 { // Lane 6: Ethernet
Can you check whether you have DT node links in sysfs for the PCI
devices? If you don't, then something is wrong still in the topology
or the PCI core is failing to set the DT node pointer in struct
device. Though you don't rely on that currently, we want the topology
to match. It's possible this never worked on arm/arm64 as mainly
powerpc relied on this.
I'd like some way to validate the DT matches the PCI topology. We
could have a tool that generates the DT structure based on the PCI
topology.
The of_node node link is on those places:
$ find /sys/devices/platform/soc/f4000000.pcie/ -name of_node
/sys/devices/platform/soc/f4000000.pcie/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/0000:00:00.0/pci_bus/0000:01/of_node
/sys/devices/platform/soc/f4000000.pcie/pci0000:00/pci_bus/0000:00/of_node
Looks like we're missing some...
It's not immediately obvious to me what's wrong here. Only the root
bus is getting it's DT node set. The relevant code is pci_scan_device(),
pci_set_of_node() and pci_set_bus_of_node(). Give me a few days to try
to reproduce and debug it.
I believe the issue is we need another bridge node in the DT
hierarchy. What we have is:
Bus 0 is node /soc/pcie@f4000000
Bus 1 is device 0 on bus 0 is node /soc/pcie@f4000000/pcie@0,0
Bus 2 is device 0 on bus 1 in node ... whoops, there's no device 0
under /soc/pcie@f4000000/pcie@0,0
So we need the hierarchy to be: /soc/pcie@f4000000/pcie@0/pcie@0/pcie@{1,5,7}
Adding a child pcie@0 produces the following output from my debug
patches:
You removed your changes to the PCI code other than the debug print?
While technically correct having the bus# in the address, that doesn't
work for FDT since we don't know the bus assignment. So we should just
use 0.
Using 0 causes DTB compilation to produce a warning, due to the
bus-range.
What's the warning? 'bus-range' should be optional.
quoted
Without the bus-range, there will be runtime warnings,
as this will be assigned as bus 1.
Okay, that might be something we need to fix.
Actually, I don't see how there's a problem. First, the only place we
read 'bus-range' is of_pci_parse_bus_range() and that's only called
for the host bridge. Any other occurrence of 'bus-range' should be
ignored. The only place we read 'reg' is of_pci_get_devfn() and we
mask just the devfn bits.
[ 4.992572] pci_bus 0000:00: root bus resource [bus 00-01]
I don't see any way this can happen other than pcie@f4000000 node
containing 'bus-range = <0 1>;'. It comes from
pci_host_bridge.windows.
Rob
While technically correct having the bus# in the address, that doesn't
work for FDT since we don't know the bus assignment. So we should just
use 0.
Using 0 causes DTB compilation to produce a warning, due to the
bus-range.
What's the warning? 'bus-range' should be optional.
With this DT schema (simplified to show only relevant bits):
pcie@f4000000 {
bus-range = <0x00 0xff>;
pcie@0,0 { // Lane 0: PCIe switch: Bus 1, Device 0
bus-range = <0x01 0xff>;
pcie@0,0 { // Lane 0: upstream
reg = <0x0000 0 0 0 0>;
pcie@1,0 { // Lane 4: M.2
reg = <0x0800 0 0 0 0>;
};
pcie@5,0 { // Lane 5: Mini PCIe
reg = <0x2800 0 0 0 0>;
};
pcie@7,0 { // Lane 6: Ethernet
reg = <0x3800 0 0 0 0>;
};
};
};
};
This is the compilation warning:
$ make dtbs
arch/arm64/boot/dts/hisilicon/hi3670.dtsi:735.5-29: Warning (pci_device_bus_num): /soc/pcie@f4000000/pcie@0,0/pcie@0,0:bus-range: PCI bus number 0 out of range, expected (1 - 1)
This is solved by changing:
pcie@0,0 { // Lane 0: upstream
reg = <0x0000 0 0 0 0>;
to:
pcie@0,0 { // Lane 0: upstream
reg = <0x010000 0 0 0 0>;
quoted
quoted
Without the bus-range, there will be runtime warnings,
as this will be assigned as bus 1.
Okay, that might be something we need to fix.
Actually, I don't see how there's a problem. First, the only place we
read 'bus-range' is of_pci_parse_bus_range() and that's only called
for the host bridge. Any other occurrence of 'bus-range' should be
ignored. The only place we read 'reg' is of_pci_get_devfn() and we
mask just the devfn bits.
quoted
[ 4.992572] pci_bus 0000:00: root bus resource [bus 00-01]
I don't see any way this can happen other than pcie@f4000000 node
containing 'bus-range = <0 1>;'. It comes from
pci_host_bridge.windows.
Yeah, you're right. On the first versions, 'bus-range = <0 1>;' was used,
just like:
arch/arm64/boot/dts/hisilicon/hi3660.dtsi
(I have a fixup patch fixing the warning on Kirin 960 due to that)
Ok, I did another test here: if I drop both dma-range, it will output:
[ 3.778028] kirin-pcie f4000000.pcie: No bus range found for /soc/pcie@f4000000, using [bus 00-ff]
But no conflict warnings. So, I guess dropping bus-range and the explicit
bus 1 setting at "reg" is a clean approach.
As a reference, this is the dmesg log here (with OF debug turned on):
# dmesg|grep -i pci
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.14.0-rc1+ root=UUID=646147e1-d186-44cc-b0d2-60c42f8dcc6d ro drm.debug=0x6 earlycon=pl011,0xfff32000,115200 console=tty0 console=ttyAMA6,115200n8 root=/dev/sdd12 rootwait rw quiet efi=noruntime drm.debug=0x06 "dyndbg=file drivers/pci/of.c +p; drivers/gpu/* +p; file drivers/pci/of.c +p; drivers/spmi/* +p; file drivers/pci/of.c +p; drivers/mfd/* +p; file drivers/pci/of.c +p; drivers/regulator/* +p; file drivers/pci/of.c +p; drivers/usb/core/hub.c +p" no_console_suspend loglevel=9 kernel.panic=5
[ 0.000000] Unknown command line parameters: BOOT_IMAGE=/boot/vmlinuz-5.14.0-rc1+ dyndbg=file drivers/pci/of.c +p; drivers/gpu/* +p; file drivers/pci/of.c +p; drivers/spmi/* +p; file drivers/pci/of.c +p; drivers/mfd/* +p; file drivers/pci/of.c +p; drivers/regulator/* +p; file drivers/pci/of.c +p; drivers/usb/core/hub.c +p
[ 0.725101] PCI: CLS 0 bytes, default 64
[ 1.520828] ehci-pci: EHCI PCI platform driver
[ 1.521022] ohci-pci: OHCI PCI platform driver
[ 2.204793] dyndbg=file drivers/pci/of.c +p; drivers/gpu/* +p; file drivers/pci/of.c +p; drivers/spmi/* +p; file drivers/pci/of.c +p; drivers/mfd/* +p; file drivers/pci/of.c +p; drivers/regulator/* +p; file drivers/pci/of.c +p; drivers/usb/core/hub.c +p
[ 3.767252] kirin-pcie f4000000.pcie: host bridge /soc/pcie@f4000000 ranges:
[ 3.778028] kirin-pcie f4000000.pcie: No bus range found for /soc/pcie@f4000000, using [bus 00-ff]
[ 3.792466] kirin-pcie f4000000.pcie: Parsing ranges property...
[ 3.801919] kirin-pcie f4000000.pcie: MEM 0x00f6000000..0x00f7ffffff -> 0x0000000000
[ 3.813680] kirin-pcie f4000000.pcie: invalid resource
[ 3.822189] kirin-pcie f4000000.pcie: iATU unroll: enabled
[ 3.831159] kirin-pcie f4000000.pcie: Detected iATU regions: 8 outbound, 8 inbound
[ 3.943155] kirin-pcie f4000000.pcie: Link up
[ 3.956821] (null): pci_set_bus_of_node: of_node: /soc/pcie@f4000000
[ 3.979143] kirin-pcie f4000000.pcie: PCI host bridge to bus 0000:00
[ 3.989441] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 3.998050] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf7ffffff] (bus address [0x00000000-0x01ffffff])
[ 4.011965] pci 0000:00:00.0: [19e5:3670] type 01 class 0x060400
[ 4.021177] pci 0000:00:00.0: reg 0x10: [mem 0xf6000000-0xf6ffffff]
[ 4.031041] pci 0000:00:00.0: pci_set_of_node: of_node: /soc/pcie@f4000000
[ 4.041362] pci 0000:00:00.0: supports D1 D2
[ 4.049191] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot
[ 4.059554] pci_bus 0000:01: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0
[ 4.071133] pci 0000:01:00.0: [10b5:8606] type 01 class 0x060400
[ 4.080831] pci 0000:01:00.0: reg 0x10: [mem 0xf6000000-0xf601ffff]
[ 4.103511] pci 0000:01:00.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0
[ 4.115403] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 4.139816] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.157297] pci_bus 0000:02: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.172380] pci 0000:02:01.0: [10b5:8606] type 01 class 0x060400
[ 4.182279] pci 0000:02:01.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.197047] pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
[ 4.207583] pci 0000:02:04.0: [10b5:8606] type 01 class 0x060400
[ 4.217659] pci 0000:02:04.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.234551] pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
[ 4.254284] pci 0000:02:05.0: [10b5:8606] type 01 class 0x060400
[ 4.267668] pci 0000:02:05.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.282530] pci 0000:02:05.0: PME# supported from D0 D3hot D3cold
[ 4.295077] pci 0000:02:07.0: [10b5:8606] type 01 class 0x060400
[ 4.306032] pci 0000:02:07.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.320927] pci 0000:02:07.0: PME# supported from D0 D3hot D3cold
[ 4.333414] pci 0000:02:09.0: [10b5:8606] type 01 class 0x060400
[ 4.340439] pci 0000:02:09.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0
[ 4.350084] pci 0000:02:09.0: PME# supported from D0 D3hot D3cold
[ 4.358150] pci 0000:02:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.366379] pci 0000:02:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.374515] pci 0000:02:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.382682] pci 0000:02:07.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.390834] pci 0000:02:09.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 4.399079] pci_bus 0000:03: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0/pcie@1,0
[ 4.409309] pci 0000:03:00.0: [144d:a809] type 00 class 0x010802
[ 4.415564] pci 0000:03:00.0: reg 0x10: [mem 0xf6000000-0xf6003fff 64bit]
[ 4.422769] pci 0000:03:00.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0/pcie@1,0
[ 4.433782] pci 0000:03:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[ 4.449954] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[ 4.456836] pci_bus 0000:04: pci_set_bus_of_node: of_node: (null)
[ 4.457918] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 4.478895] pci_bus 0000:05: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0/pcie@5,0
[ 4.491772] pci_bus 0000:05: busn_res: [bus 05-ff] end is updated to 05
[ 4.503438] pci_bus 0000:06: pci_set_bus_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0/pcie@7,0
[ 4.529140] pci 0000:06:00.0: [10ec:8168] type 00 class 0x020000
[ 4.535374] pci 0000:06:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 4.541229] pci 0000:06:00.0: reg 0x18: [mem 0xf7000000-0xf7000fff 64bit]
[ 4.548194] pci 0000:06:00.0: reg 0x20: [mem 0xf7200000-0xf7203fff 64bit pref]
[ 4.561063] pci 0000:06:00.0: pci_set_of_node: of_node: /soc/pcie@f4000000/pcie@0,0/pcie@0,0/pcie@7,0
[ 4.582075] pci 0000:06:00.0: supports D1 D2
[ 4.586357] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 4.594727] pci_bus 0000:06: busn_res: [bus 06-ff] end is updated to 06
[ 4.601538] pci_bus 0000:07: pci_set_bus_of_node: of_node: (null)
[ 4.608473] pci_bus 0000:07: busn_res: [bus 07-ff] end is updated to 07
[ 4.615124] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 07
[ 4.621810] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6000000-0xf6ffffff]
[ 4.628606] pci 0000:00:00.0: BAR 14: assigned [mem 0xf7000000-0xf72fffff]
[ 4.628610] pci 0000:00:00.0: BAR 15: assigned [mem 0xf7300000-0xf73fffff pref]
[ 4.642813] pci 0000:00:00.0: BAR 13: no space for [io size 0x1000]
[ 4.649174] pci 0000:00:00.0: BAR 13: failed to assign [io size 0x1000]
[ 4.661518] pci 0000:01:00.0: BAR 14: assigned [mem 0xf7000000-0xf71fffff]
[ 4.669074] pci 0000:01:00.0: BAR 15: assigned [mem 0xf7300000-0xf73fffff 64bit pref]
[ 4.677066] pci 0000:01:00.0: BAR 0: assigned [mem 0xf7200000-0xf721ffff]
[ 4.683891] pci 0000:01:00.0: BAR 13: no space for [io size 0x1000]
[ 4.690252] pci 0000:01:00.0: BAR 13: failed to assign [io size 0x1000]
[ 4.690266] pci 0000:02:01.0: BAR 14: assigned [mem 0xf7000000-0xf70fffff]
[ 4.690270] pci 0000:02:07.0: BAR 14: assigned [mem 0xf7100000-0xf71fffff]
[ 4.710728] pci 0000:02:07.0: BAR 15: assigned [mem 0xf7300000-0xf73fffff 64bit pref]
[ 4.727780] pci 0000:02:07.0: BAR 13: no space for [io size 0x1000]
[ 4.727783] pci 0000:02:07.0: BAR 13: failed to assign [io size 0x1000]
[ 4.727790] pci 0000:03:00.0: BAR 0: assigned [mem 0xf7000000-0xf7003fff 64bit]
[ 4.727904] pci 0000:02:01.0: PCI bridge to [bus 03]
[ 4.753165] pci 0000:02:01.0: bridge window [mem 0xf7000000-0xf70fffff]
[ 4.769738] pci 0000:02:04.0: PCI bridge to [bus 04]
[ 4.774843] pci 0000:02:05.0: PCI bridge to [bus 05]
[ 4.779959] pci 0000:06:00.0: BAR 4: assigned [mem 0xf7300000-0xf7303fff 64bit pref]
[ 4.787839] pci 0000:06:00.0: BAR 2: assigned [mem 0xf7100000-0xf7100fff 64bit]
[ 4.795273] pci 0000:06:00.0: BAR 0: no space for [io size 0x0100]
[ 4.801543] pci 0000:06:00.0: BAR 0: failed to assign [io size 0x0100]
[ 4.808159] pci 0000:02:07.0: PCI bridge to [bus 06]
[ 4.813166] pci 0000:02:07.0: bridge window [mem 0xf7100000-0xf71fffff]
[ 4.819981] pci 0000:02:07.0: bridge window [mem 0xf7300000-0xf73fffff 64bit pref]
[ 4.827782] pci 0000:02:09.0: PCI bridge to [bus 07]
[ 4.832871] pci 0000:01:00.0: PCI bridge to [bus 02-07]
[ 4.838138] pci 0000:01:00.0: bridge window [mem 0xf7000000-0xf71fffff]
[ 4.844952] pci 0000:01:00.0: bridge window [mem 0xf7300000-0xf73fffff 64bit pref]
[ 4.852751] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 4.857980] pci 0000:00:00.0: bridge window [mem 0xf7000000-0xf72fffff]
[ 4.864770] pci 0000:00:00.0: bridge window [mem 0xf7300000-0xf73fffff pref]
[ 4.881547] pcieport 0000:00:00.0: PME: Signaling with IRQ 58
[ 4.888905] pcieport 0000:00:00.0: AER: enabled with IRQ 58
[ 4.895013] pcieport 0000:01:00.0: enabling device (0000 -> 0002)
[ 4.903260] pcieport 0000:02:01.0: enabling device (0000 -> 0002)
[ 4.914761] pcieport 0000:02:07.0: enabling device (0000 -> 0002)
[ 4.928984] nvme nvme0: pci function 0000:03:00.0
Thanks,
Mauro