Re: mpc8xxx PCIe hotplug needs fixing, some clues ..

Kumar Gala [off-list ref] wrote on 2012/07/20 20:53:10:

On Jul 20, 2012, at 2:17 AM, Joakim Tjernlund wrote:

quoted

Hi Guys

I see that you have been hacking Freescale PCI before so I send this to you(and the list)

We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI device that
started from user space (needs advance clock conf) so when linux boots there is
no device at all.
Trying to "hotplug" the device after it is enabled fails, no amount of recan/remove using
either fake or real hotplug makes a difference.

I found the cause eventually but I can't fix it properly as I known almost nothing about PCI.
Cause:
indirect_pci.c:indirect_read_config() tests for if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
and returns  PCIBIOS_DEVICE_NOT_FOUND

PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for fsl_pcie_check_link) but is never cleared.
Clearing it as appropriate makes a small difference. If you
remove the RC and do a few of rescan's then the device appears.

Hacking some more, like so:

int fsl_pcie_check_link(struct pci_controller *hose)
{
   u32 val;

   early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
   hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
   if (val < PCIE_LTSSM_L0)
      return 1;
   hose->indirect_type &= ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
   return 0;
}
and then using it carefully(it is easy to make linux hang) in indirect_read_config():
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
           int len, u32 *val)
{
   struct pci_controller *hose = pci_bus_to_host(bus);
   volatile void __iomem *cfg_data;
   u8 cfg_type = 0;
   u32 bus_no, reg;

   if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
      if (bus->number != hose->first_busno ||
          devfn != 0) {
         fsl_pcie_check_link(hose);
         return PCIBIOS_DEVICE_NOT_FOUND;
      }
   }

Now it works, just one rescan and the device appears!
This is a hack, I don't known what other trouble it can cause, I hope you can
sort this out.

How are you forcing the re-scan?  We can see if we can add a re-check of the link state in that flow somewhere.

echo 1 > /sys/bus/pci/rescan

Why is that check important? Seems like some very ppc specific workaround for something.

Can you do a dump_stack() or something to get a call chain?

here?
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
		     int len, u32 *val)
{
	struct pci_controller *hose = pci_bus_to_host(bus);
	volatile void __iomem *cfg_data;
	u8 cfg_type = 0;
	u32 bus_no, reg;
	static int first_dump;

	if (!first_dump) {
		dump_stack();
		first_dump = 1;
	}
...

I am not at work and and my board needs a reset button press to recover :(
Furthermore, my vacation starts next week, not sure I can get it fixed soon enough

 Jocke