dra7xx pci driver broken on dra7xx-evm

The dra7xx-evm box freezes hard at the very first PCI bus access, the
call to in dra7xx_pcie_readl() in dra7xx_pcie_probe().

The out-of-tree 3.14.53 TI driver performs a reset via platform data,
calling omap_device_assert_hardreset() by hook and crook.

Adding the reset into the mainline driver cures the freeze.

Question: How on earth is this driver supposed to work?

Thanks,
Richard

Hi,

On Thursday 07 January 2016 03:15 AM, Richard Cochran wrote:

The dra7xx-evm box freezes hard at the very first PCI bus access, the
call to in dra7xx_pcie_readl() in dra7xx_pcie_probe().

The out-of-tree 3.14.53 TI driver performs a reset via platform data,
calling omap_device_assert_hardreset() by hook and crook.

Adding the reset into the mainline driver cures the freeze.

Question: How on earth is this driver supposed to work?

It doesn't work out of the box in mainline. (Every time I post a PCI patch, I
add the pdata-quirks mechanism to test it). The pdata-quirks mechanism used in
TI tree is not an upstream-able solution. Something like this [1] was done for
performing reset. Not sure the latest state of that. Dan? Suman?

Thanks
Kishon

[1] -> https://patches.linaro.org/33797/

Thanks,
Richard

On Thu, Jan 07, 2016 at 10:47:50AM +0530, Kishon Vijay Abraham I wrote:

It doesn't work out of the box in mainline.

This driver was added in v3.17.  So once again, TI people have merged
non-functioning drivers into mainline.  Super.  Thanks for wasting my
time.

Disappointed, annoyed, but not surprised,

Richard

This patch marks the dra7xx pci host driver as broken.  This driver
was first merged in v3.17 and has never worked.  Although the driver
compiles just fine, it is missing an essential device reset.  If the
driver is included, the kernel locks up hard shortly after booting,
before any console output appears.

Signed-off-by: Richard Cochran <richardcochran@gmail.com>

---
 drivers/pci/host/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index d5e58ba..d83e8f0 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig

@@ -5,6 +5,7 @@ config PCI_DRA7XX
 	bool "TI DRA7xx PCIe controller"
 	select PCIE_DW
 	depends on OF && HAS_IOMEM && TI_PIPE3
+	depends on BROKEN
 	help
 	 Enables support for the PCIe controller in the DRA7xx SoC.  There
 	 are two instances of PCIe controller in DRA7xx.  This controller can

Hi Richard,

On Thursday 07 January 2016 01:41 PM, Richard Cochran wrote:

In my point of view the driver is not broken as such but an independent piece
(reset) which is missing, since this driver compiles and works fine if that
reset piece is added. You are right in that this shouldn't have been probed
since it's known that the reset piece is missing. Maybe we should just set
"status = disabled" in dra7.dtsi and when that reset piece is added enable it
back in dra7-evm.dts?

Thanks
Kishon

 	help
 	 Enables support for the PCIe controller in the DRA7xx SoC.  There
 	 are two instances of PCIe controller in DRA7xx.  This controller can

On Thu, Jan 07, 2016 at 02:14:22PM +0530, Kishon Vijay Abraham I wrote:

In my point of view the driver is not broken as such but an independent piece
(reset) which is missing, since this driver compiles and works fine if that
reset piece is added. You are right in that this shouldn't have been probed
since it's known that the reset piece is missing. Maybe we should just set
"status = disabled" in dra7.dtsi and when that reset piece is added enable it
back in dra7-evm.dts?

That isn't strong enough, IMHO, since status=disabled is the default
for most every DTS item.  Maybe you could comment the PCI block out or
remove it and put a comment in about the missing piece and the lock up
behavior.  It is really a nasty bug to have the machine freeze up with
no output at all.  It took me some time to figure out what was going
on, and it would be a shame if someone else sets status=okay in their
board file and has to repeat the whole exercise.

Thanks,
Richard

On Thu, Jan 7, 2016 at 3:25 AM, Richard Cochran
[off-list ref] wrote:

On Thu, Jan 07, 2016 at 02:14:22PM +0530, Kishon Vijay Abraham I wrote:

quoted

In my point of view the driver is not broken as such but an independent piece
(reset) which is missing, since this driver compiles and works fine if that
reset piece is added. You are right in that this shouldn't have been probed
since it's known that the reset piece is missing. Maybe we should just set
"status = disabled" in dra7.dtsi and when that reset piece is added enable it
back in dra7-evm.dts?

That isn't strong enough, IMHO, since status=disabled is the default
for most every DTS item.  Maybe you could comment the PCI block out or
remove it and put a comment in about the missing piece and the lock up
behavior.  It is really a nasty bug to have the machine freeze up with
no output at all.  It took me some time to figure out what was going
on, and it would be a shame if someone else sets status=okay in their
board file and has to repeat the whole exercise.

It sounds like dra7xx is missing a piece of essential functionality.
If that's the case, what is the benefit of having it in the kernel at
all?  It makes work for everybody else (adapting to changes outside
the driver), but it sounds like nobody gets any benefit.

If somebody posts a patch to remove the driver entirely to
linux-pci at vger.kernel.org, we can discuss whether that's the right
thing to do.

Bjorn

On 01/06/2016 11:17 PM, Kishon Vijay Abraham I wrote:

Hi,

+ Tero, Tony and linux-omap

On Thursday 07 January 2016 03:15 AM, Richard Cochran wrote:

quoted

The dra7xx-evm box freezes hard at the very first PCI bus access, the
call to in dra7xx_pcie_readl() in dra7xx_pcie_probe().

The out-of-tree 3.14.53 TI driver performs a reset via platform data,
calling omap_device_assert_hardreset() by hook and crook.

Yeah, this is the only feasible way and not break multi-arch atm until a
reset framework is in place. We have couple of drivers (omap_iommu and
wkup_m3_rproc) following this approach.

quoted

Adding the reset into the mainline driver cures the freeze.

quoted

Question: How on earth is this driver supposed to work?

It doesn't work out of the box in mainline. (Every time I post a PCI patch, I
add the pdata-quirks mechanism to test it). The pdata-quirks mechanism used in
TI tree is not an upstream-able solution. Something like this [1] was done for
performing reset. Not sure the latest state of that. Dan? Suman?

Yeah, [1] is very old, and it doesn't deal with the hwmod integration
for pm_runtime API usage. Tero has posted another revamped series [2]
with hwmod integration, and it needs to be refreshed with few additional
patches and fixes. It will probably take couple of releases before it
hits mainline. So, I guess it is then a question of picking up the pdata
quirks solution for interim to get the driver functional until the reset
framework hits mainline, or mark the driver as broken.

regards
Suman

[1] -> https://patches.linaro.org/33797/

[2] http://marc.info/?l=linux-omap&m=144310496514782&w=2

* Suman Anna [off-list ref] [160107 13:26]:

On 01/06/2016 11:17 PM, Kishon Vijay Abraham I wrote:

quoted

Hi,

+ Tero, Tony and linux-omap

quoted

On Thursday 07 January 2016 03:15 AM, Richard Cochran wrote:

quoted

The dra7xx-evm box freezes hard at the very first PCI bus access, the
call to in dra7xx_pcie_readl() in dra7xx_pcie_probe().

The out-of-tree 3.14.53 TI driver performs a reset via platform data,
calling omap_device_assert_hardreset() by hook and crook.

Yeah, this is the only feasible way and not break multi-arch atm until a
reset framework is in place. We have couple of drivers (omap_iommu and
wkup_m3_rproc) following this approach.

Sounds like doing reset with platform data is the way to go until the
reset controller works.

Regards,

Tony

Hi,

On Friday 08 January 2016 03:29 AM, Tony Lindgren wrote:

* Suman Anna [off-list ref] [160107 13:26]:

quoted

On 01/06/2016 11:17 PM, Kishon Vijay Abraham I wrote:

quoted

Hi,

+ Tero, Tony and linux-omap

quoted

On Thursday 07 January 2016 03:15 AM, Richard Cochran wrote:

quoted

The dra7xx-evm box freezes hard at the very first PCI bus access, the
call to in dra7xx_pcie_readl() in dra7xx_pcie_probe().

The out-of-tree 3.14.53 TI driver performs a reset via platform data,
calling omap_device_assert_hardreset() by hook and crook.

Yeah, this is the only feasible way and not break multi-arch atm until a
reset framework is in place. We have couple of drivers (omap_iommu and
wkup_m3_rproc) following this approach.

Sounds like doing reset with platform data is the way to go until the
reset controller works.

All right. I'll post the patch to do reset using platform data.

Thanks
Kishon

Hi Bjorn,

On Thursday 07 January 2016 08:12 PM, Bjorn Helgaas wrote:

On Thu, Jan 7, 2016 at 3:25 AM, Richard Cochran
[off-list ref] wrote:

quoted

On Thu, Jan 07, 2016 at 02:14:22PM +0530, Kishon Vijay Abraham I wrote:

quoted

In my point of view the driver is not broken as such but an independent piece
(reset) which is missing, since this driver compiles and works fine if that
reset piece is added. You are right in that this shouldn't have been probed
since it's known that the reset piece is missing. Maybe we should just set
"status = disabled" in dra7.dtsi and when that reset piece is added enable it
back in dra7-evm.dts?

That isn't strong enough, IMHO, since status=disabled is the default
for most every DTS item.  Maybe you could comment the PCI block out or
remove it and put a comment in about the missing piece and the lock up
behavior.  It is really a nasty bug to have the machine freeze up with
no output at all.  It took me some time to figure out what was going
on, and it would be a shame if someone else sets status=okay in their
board file and has to repeat the whole exercise.

It sounds like dra7xx is missing a piece of essential functionality.
If that's the case, what is the benefit of having it in the kernel at
all?  It makes work for everybody else (adapting to changes outside
the driver), but it sounds like nobody gets any benefit.

During the time pci-dra7xx driver was merged, the reset controller driver was
already being discussed in the list. And I didn't think it'll take this long
for it to be merged. And I didn't want pci driver development to be hampered by
these external pieces. That's why I sent the pci-dra7xx patches for merging.

IMHO merging pci-dra7xx helped to keep active development of the driver, though
I agree we should have kept mechanisms to keep the driver disabled either from
dt or in Kconfig and a way to tell that reset piece is missing if someone has
to enable the driver.

If somebody posts a patch to remove the driver entirely to
linux-pci at vger.kernel.org, we can discuss whether that's the right
thing to do.

I think that's not required now since Tony agreed to have pdata approach to
perform reset till reset controller is merged. I'll be posting patch for it soon.

Thanks
Kishon

On Fri, Jan 08, 2016 at 11:37:09AM +0530, Kishon Vijay Abraham I wrote:

All right. I'll post the patch to do reset using platform data.

Thanks for taking care of this.

BTW, I am trying to get a new board working.  Are there any *other*
dra7xx peripherals I should know about, or is the mainline stuff all
working?

Thanks,
Richard

Hi,

On Friday 08 January 2016 02:52 PM, Richard Cochran wrote:

On Fri, Jan 08, 2016 at 11:37:09AM +0530, Kishon Vijay Abraham I wrote:

quoted

All right. I'll post the patch to do reset using platform data.

Thanks for taking care of this.

BTW, I am trying to get a new board working.  Are there any *other*
dra7xx peripherals I should know about, or is the mainline stuff all
working?

Nothing that I'm aware of. I've tested Ethernet, USB, SATA, MMC and PCIe (with
reset changes) myself. PCI was the only device that needed out of tree patches.

Thanks
Kishon