Hi Guys
I see that you have been hacking Freescale PCI before so I send this to you(and the list)
We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI device that
started from user space (needs advance clock conf) so when linux boots there is
no device at all.
Trying to "hotplug" the device after it is enabled fails, no amount of recan/remove using
either fake or real hotplug makes a difference.
I found the cause eventually but I can't fix it properly as I known almost nothing about PCI.
Cause:
indirect_pci.c:indirect_read_config() tests for if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
and returns PCIBIOS_DEVICE_NOT_FOUND
PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for fsl_pcie_check_link) but is never cleared.
Clearing it as appropriate makes a small difference. If you
remove the RC and do a few of rescan's then the device appears.
Hacking some more, like so:
int fsl_pcie_check_link(struct pci_controller *hose)
{
u32 val;
early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
if (val < PCIE_LTSSM_L0)
return 1;
hose->indirect_type &= ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
return 0;
}
and then using it carefully(it is easy to make linux hang) in indirect_read_config():
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
int len, u32 *val)
{
struct pci_controller *hose = pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type = 0;
u32 bus_no, reg;
if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
if (bus->number != hose->first_busno ||
devfn != 0) {
fsl_pcie_check_link(hose);
return PCIBIOS_DEVICE_NOT_FOUND;
}
}
Now it works, just one rescan and the device appears!
This is a hack, I don't known what other trouble it can cause, I hope you can
sort this out.
BTW,
I see that the controller has some support for hot plug, by setting a bit or two in
PEX_CONFIG, out_be32(&pci->pex_config, 0x02), I see slot caps in lspci:
Capabilities: [4c] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE- FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <2us, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise-
Slot #0, PowerLimit 15.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet+ CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet- LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
That would be good to enable I think(real hotplug needs them), don't know if this bit is
common for all PCIe controllers from Freescale?
Jocke
On Jul 20, 2012, at 2:17 AM, Joakim Tjernlund wrote:
=20
Hi Guys
=20
I see that you have been hacking Freescale PCI before so I send this =
to you(and the list)
=20
We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI =
device that
started from user space (needs advance clock conf) so when linux boots =
there is
no device at all.
Trying to "hotplug" the device after it is enabled fails, no amount of =
recan/remove using
either fake or real hotplug makes a difference.
=20
I found the cause eventually but I can't fix it properly as I known =
almost nothing about PCI.
Cause:
indirect_pci.c:indirect_read_config() tests for if =
(hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
and returns PCIBIOS_DEVICE_NOT_FOUND
=20
PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for =
fsl_pcie_check_link) but is never cleared.
Clearing it as appropriate makes a small difference. If you
remove the RC and do a few of rescan's then the device appears.
=20
Hacking some more, like so:
=20
int fsl_pcie_check_link(struct pci_controller *hose)
{
u32 val;
=20
early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
hose->indirect_type |=3D PPC_INDIRECT_TYPE_NO_PCIE_LINK;
if (val < PCIE_LTSSM_L0)
return 1;
hose->indirect_type &=3D ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
return 0;
}
and then using it carefully(it is easy to make linux hang) in =
indirect_read_config():
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int =
offset,
int len, u32 *val)
{
struct pci_controller *hose =3D pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type =3D 0;
u32 bus_no, reg;
=20
if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
if (bus->number !=3D hose->first_busno ||
devfn !=3D 0) {
fsl_pcie_check_link(hose);
return PCIBIOS_DEVICE_NOT_FOUND;
}
}
=20
Now it works, just one rescan and the device appears!
This is a hack, I don't known what other trouble it can cause, I hope =
you can
sort this out.
How are you forcing the re-scan? We can see if we can add a re-check of =
the link state in that flow somewhere.
Can you do a dump_stack() or something to get a call chain?
- k=
Kumar Gala [off-list ref] wrote on 2012/07/20 20:53:10:
On Jul 20, 2012, at 2:17 AM, Joakim Tjernlund wrote:
quoted
Hi Guys
I see that you have been hacking Freescale PCI before so I send this to you(and the list)
We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI device that
started from user space (needs advance clock conf) so when linux boots there is
no device at all.
Trying to "hotplug" the device after it is enabled fails, no amount of recan/remove using
either fake or real hotplug makes a difference.
I found the cause eventually but I can't fix it properly as I known almost nothing about PCI.
Cause:
indirect_pci.c:indirect_read_config() tests for if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
and returns PCIBIOS_DEVICE_NOT_FOUND
PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for fsl_pcie_check_link) but is never cleared.
Clearing it as appropriate makes a small difference. If you
remove the RC and do a few of rescan's then the device appears.
Hacking some more, like so:
int fsl_pcie_check_link(struct pci_controller *hose)
{
u32 val;
early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
if (val < PCIE_LTSSM_L0)
return 1;
hose->indirect_type &= ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
return 0;
}
and then using it carefully(it is easy to make linux hang) in indirect_read_config():
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
int len, u32 *val)
{
struct pci_controller *hose = pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type = 0;
u32 bus_no, reg;
if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
if (bus->number != hose->first_busno ||
devfn != 0) {
fsl_pcie_check_link(hose);
return PCIBIOS_DEVICE_NOT_FOUND;
}
}
Now it works, just one rescan and the device appears!
This is a hack, I don't known what other trouble it can cause, I hope you can
sort this out.
How are you forcing the re-scan? We can see if we can add a re-check of the link state in that flow somewhere.
echo 1 > /sys/bus/pci/rescan
Why is that check important? Seems like some very ppc specific workaround for something.
Can you do a dump_stack() or something to get a call chain?
here?
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
int len, u32 *val)
{
struct pci_controller *hose = pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type = 0;
u32 bus_no, reg;
static int first_dump;
if (!first_dump) {
dump_stack();
first_dump = 1;
}
...
I am not at work and and my board needs a reset button press to recover :(
Furthermore, my vacation starts next week, not sure I can get it fixed soon enough
Jocke
Kumar Gala [off-list ref] wrote on 2012/07/20 20:53:10:
From: Kumar Gala <redacted>
To: Joakim Tjernlund <redacted>,
Cc: scottwood@freescale.com, linuxppc-dev@ozlabs.org
Date: 2012/07/20 20:53
Subject: Re: mpc8xxx PCIe hotplug needs fixing, some clues ..
On Jul 20, 2012, at 2:17 AM, Joakim Tjernlund wrote:
quoted
Hi Guys
I see that you have been hacking Freescale PCI before so I send this to you(and the list)
We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI device that
started from user space (needs advance clock conf) so when linux boots there is
no device at all.
Trying to "hotplug" the device after it is enabled fails, no amount of recan/remove using
either fake or real hotplug makes a difference.
I found the cause eventually but I can't fix it properly as I known almost nothing about PCI.
Cause:
indirect_pci.c:indirect_read_config() tests for if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
and returns PCIBIOS_DEVICE_NOT_FOUND
PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for fsl_pcie_check_link) but is never cleared.
Clearing it as appropriate makes a small difference. If you
remove the RC and do a few of rescan's then the device appears.
Hacking some more, like so:
int fsl_pcie_check_link(struct pci_controller *hose)
{
u32 val;
early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
if (val < PCIE_LTSSM_L0)
return 1;
hose->indirect_type &= ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
return 0;
}
and then using it carefully(it is easy to make linux hang) in indirect_read_config():
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
int len, u32 *val)
{
struct pci_controller *hose = pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type = 0;
u32 bus_no, reg;
if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
if (bus->number != hose->first_busno ||
devfn != 0) {
fsl_pcie_check_link(hose);
return PCIBIOS_DEVICE_NOT_FOUND;
}
}
Now it works, just one rescan and the device appears!
This is a hack, I don't known what other trouble it can cause, I hope you can
sort this out.
Related, should not all those fsl quirks be __devinit instead of __init?
See this for motivation
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=85a053fa5f2d67ae5b2968305b16e8d2fe4cdf4d
Could quirk_fsl_pcie_header be moved to early fixup time? I recall some code, maybe is was hotplug,
complaining about not recognising the header because it executed before the header fixup.
Jocke