pci-mvebu driver on km_kirkwood
From: Gerlando Falauto <hidden>
Date: 2014-02-19 09:39:07
Hi Thomas, spoiler first: SUCCESS!!!! On 02/19/2014 10:26 AM, Thomas Petazzoni wrote:
Dear Gerlando Falauto,
[...]
quoted
# hexdump -C /proc/device-tree/ocp at f1000000/pcie-controller/ranges | cut -c1-58 00000000 82 00 00 00 00 00 00 00 00 04 00 00 00 04 00 00 00000010 00 00 00 00 00 00 20 00 82 00 00 00 00 00 00 00 00000020 e0 00 00 00 e0 00 00 00 00 00 00 00 10 00 00 00 ^^^^^^^^^^^Wow, that's an old DT representation that you have here :)
Indeed... ;-)
quoted hunk ↗ jump to hunk
But ok, let me try to explain. The 256 MB value that you define in the DT is the global PCIe memory aperture: it is the maximum amount of memory that we allow the PCIe driver to allocate for PCIe windows. But depending on which PCIe devices you have plugged in, and how large their BARs are, not necessarily all of these 256 MB will be used. So, you can very well have a 256 MB global PCIe memory aperture, and still have only one 1 MB PCIe memory window for PCIe 0.0 and a 256 KB PCIe memory window for PCIe 1.0, and that's it. Now, the 192 MB comes from the enumeration of your device. Linux enumerates the BAR of your device: pci 0000:01:00.0: BAR 1: assigned [mem 0xe0000000-0xe7ffffff] pci 0000:01:00.0: BAR 3: assigned [mem 0xe8000000-0xe87fffff] pci 0000:01:00.0: BAR 4: assigned [mem 0xe8800000-0xe8801fff] pci 0000:01:00.0: BAR 0: assigned [mem 0xe8802000-0xe8802fff] pci 0000:01:00.0: BAR 2: assigned [mem 0xe8803000-0xe8803fff] pci 0000:01:00.0: BAR 5: assigned [mem 0xe8804000-0xe8804fff] and then concludes that at the emulated bridge level, the memory region to be created is: pci 0000:00:01.0: bridge window [mem 0xe0000000-0xebffffff] which corresponds to the 192 MB window that we see created. But I believe a 192 MB memory window cannot work with MBus, it should be rounded up to the next power of 2. Can you try the below patch (not tested, not even compiled, might need some tweaks to apply to your 3.10 kernel) :diff --git a/drivers/pci/host/pci-mvebu.c b/drivers/pci/host/pci-mvebu.c index 13478ec..002229a 100644 --- a/drivers/pci/host/pci-mvebu.c +++ b/drivers/pci/host/pci-mvebu.c@@ -372,6 +372,11 @@ static void mvebu_pcie_handle_membase_change(struct mvebu_pcie_port *port) (((port->bridge.memlimit & 0xFFF0) << 16) | 0xFFFFF) - port->memwin_base; + pr_info("PCIE %d.%d: creating window at 0x%x, size 0x%x rounded up to 0x%x\n", + port->port, port->lane, port->memwin_base, + port->memwin_size, roundup_pow_of_two(port->memwin_size)); + port->memwin_size = roundup_pow_of_two(port->memwin_size); + mvebu_mbus_add_window_by_id(port->mem_target, port->mem_attr, port->memwin_base, port->memwin_size); }I'm obviously interested in seeing the message that gets shown, as well as the new mvebu-mbus debugfs output.
---------- pci 0000:00:01.0: bridge window [mem 0xe0000000-0xebffffff] PCIE 0.0: creating window at 0xe0000000, size 0xbffffff rounded up to 0x10000000 ---------- cat /sys/kernel/debug/mvebu-mbus/devices [00] disabled [01] disabled [02] disabled [03] disabled [04] 00000000ff000000 - 00000000ff010000 : nand [05] 00000000f4000000 - 00000000f8000000 : vpcie [06] 00000000fe000000 - 00000000fe010000 : dragonite [07] 00000000e0000000 - 00000000f0000000 : pcie0.0
For good measure, if you could also dump the registers of the PCIe window. In your case, it was window 7, so dumping 0xf1020070 and 0xf1020074 would be useful.
Isn't that where the output of debugfs comes from?
quoted
But apart from that, what I still don't understand is how that could have anything to do with my problem. The memory area I'm not able to access starts at 0xe4000000. BAR0, on the other hand, spawns 0xe8802000-0xe8802fff and seems to work fine.I am not sure, but since we are configuring an invalid memory size, maybe the MBus behavior is undefined, and we get some completely funky behavior, where parts of the 192 MB window are actually work, but parts of it are not.
And... Ladies and gentlemen... it turns out YOU'RE RIGHT!!! With your patch now everything works fine!!! No words (or quads, for that matter) can express how grateful I am! ;-) Thank you so much!!! Gerlando
Thomas