On Thu, 2011-04-28 at 13:29 -0400, Steven A. Falco wrote:

On 04/27/2011 03:51 PM, Steven A. Falco wrote:

quoted

On 04/26/2011 07:39 PM, Benjamin Herrenschmidt wrote:

quoted

On Tue, 2011-04-26 at 09:38 -0400, Steven A. Falco wrote:

quoted

On 04/25/2011 08:01 PM, Benjamin Herrenschmidt wrote:

quoted

On Mon, 2011-04-25 at 16:10 -0400, Steven A. Falco wrote:

quoted

I'm getting an error message when trying to talk to some custom
hardware:

dx83xx 0001:43:00.0: device not available because of BAR 0
[0xa1000000-0xa1ffffff] collisions

I believe I've gotten to the root cause of this issue.  Turns out the
ASIC is reporting a class code of 0000, in violation of the PCI spec.

As a result, Linux is refusing to set up the ASIC, and so it remains
with the settings made in U-Boot, which are in conflict with the
allocation Linux makes for the bridge the ASIC is connected to.

So Ben, you are absolutely right that there was some left-over
configuration.

Hrm. that's odd still, do you know where in linux is the code that tests
for the class code and decides to not setup the device resources ? I
can't see that...

My choices appear to be:

1) Fix the ASIC (yeah, right)

2) Force Linux to use the U-Boot settings

3) Hack Linux to set up a device with a bogus class.

I'm not sure why this hardware works in x86 - I guess it is less
fussy.

x86 probably just re-uses whatever setting the BIOS does, but I'm still
a bit surprised by your class code story.

I supose you can do a PCI header quirk that overrides the class code in
struct pci_dev. Something like:

static void __devinit quirk_your_asic_class(struct pci_dev *dev)
{
	dev->class = foobar;
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_xxx, PCI_DEVICE_ID_yyy, quirk_your_asic_class);

But I'd like to figure out where that is tested bcs I haven't found so
far...

	Steve

quoted

quoted

quoted

quoted

quoted

I see in setup-res.c that this message comes out when there is no
parent for
a device resource.

 .../...

It mostly happens in arch/powerpc/kernel/pci-common.c and the generic
setup-res.c

Try #define DEBUG at the top (before the #includes) of pci-common.c and
pci_32.c (remove the exiting #undef in the last one) and send us the
full dmesg log, along with the output of cat /proc/iomem

Have you set any specific flags ? IE. Modified the value of
ppc_pci_flags from what the 4xx code sets originally ?

For fun, I just tried changing:

ppc_pci_set_flags(PPC_PCI_REASSIGN_ALL_RSRC);

to:

ppc_pci_set_flags(PPC_PCI_PROBE_ONLY);

I realize that is the exact opposite of what you were suggesting, but
please bear with me for a bit.

I also changed the PCIE 0 ranges from:

ranges = <0x02000000 0x00000000 0x80000000 0x90000000 0x00000000 0x10000000
          0x01000000 0x00000000 0x00000000 0xe8010000 0x00000000 0x00010000>;

ranges = <0x02000000 0x00000000 0x90000000 0x90000000 0x00000000 0x10000000
          0x01000000 0x00000000 0x00000000 0xe8010000 0x00000000 0x00010000>;

I changed the ranges not because I wanted a 1:1 map, but because 90000000 is
what U-Boot chooses when it scans PCIe 1.

At this point, everything is working.  Here is /proc/iomap:

90000000-9fffffff : /plb/pciex@0c0000000
  90000000-94ffffff : PCI Bus 0001:41
    90000000-9001ffff : 0001:41:00.0
    90100000-94ffffff : PCI Bus 0001:42
      90100000-92ffffff : PCI Bus 0001:43
        91000000-91ffffff : 0001:43:00.0  //<--- was missing before
        92000000-92ffffff : 0001:43:00.0  //<--- was missing before
      93000000-94ffffff : PCI Bus 0001:44
        93000000-93ffffff : 0001:44:00.0  //<--- was missing before
        94000000-94ffffff : 0001:44:00.0  //<--- was missing before
e0000000-e7ffffff : /plb/pciex@0a0000000
  e0000000-e7ffffff : PCI Bus 0000:01
    e0000000-e00fffff : 0000:01:00.0
    e0100000-e01fffff : 0000:01:00.0
    e4000000-e7ffffff : 0000:01:00.0
ef600200-ef600207 : serial
ef600300-ef600307 : serial
ef600600-ef600606 : spi_ppc4xx_of
ef6c0000-ef6cffff : dwc_otg.0
  ef6c0000-ef6cffff : dwc_otg
fc000000-ffffffff : fc000000.nor_flash

Now I see the bars for the ASICs (flagged above).  I could stop here,
and declare success, but I don't really like this solution, because it
requires me to be sure the dts has the same bus addresses that U-Boot
will choose.  Seems risky.

Tentative conclusion:  Either I still have something set wrong in my dts
or there is a bug in the Linux PCI bus mapping code.

	Steve

quoted

It does look to me like some of your device BARs have been setup already
by the firmware in a way that conflict with the way you configure your
ranges, and the kernel doesn't appear to detect nor try to remap that
which would happen if you have the "probe only" flag set.

IE. On your c0000000 bus, you have memory at 90000000 CPU space mapped
to 80000000 PCI space. However, when probing, the kernel finds:

pci 0001:41:00.0: reg 10 32bit mmio: [0x90000000-0x9001ffff]

IE. A BAR was already set with a value of 90000000 PCI-side which is out
of the bounds you have for your bus.

Maybe you really want to configure that second bus to have CPU 90000000
mapped to 90000000 PCI-side ? (IE. a 1:1 mapping). That would be
something to fix in your "ranges" property.

Cheers,
Ben.