Re: Help needed to describe a custom bus in the device tree

On Tue, Mar 11, 2008 at 03:27:26PM +0100, Laurent Pinchart wrote:

Hi everybody,

the migration process from ARCH=ppc to ARCH=powerpc is easier than I
thought in some parts, but a few devices are still giving me
headaches. This should hopefully be one of my last major requests
for help (I'm sure most of you will be happy to see traffic on this
list going down when I'll be done :-))

I'm having trouble describing a custom bus named MS bus (completely
unrelated to a well-known software company) in the device tree. The
hardware is MPC8248-based and has the following hardware topology.

MPC8248 <-- localbus --> FPGA <-- ms bus --> Custom peripherals

The bus interrupt controller, serial access (SPI) controller and
status registers are accessed through memory-mapped registers in the
FPGA. Parallel access to the MS bus is handled transparently by the
FPGA which handles address mapping.

The FPGA is mapped on the locabus at address 0xf4000000. Bus control
registers are at 0xf4002000 - 0xf4003000. The parallel bus memory
window on the localbus is located at 0xf5000000.

My current dts draft describes that topology as follows (unrelated
devices on the local bus such as flash memory are removed for
clarity).

localbus@f0010100 {
	compatible = "fsl,pq2-localbus";
	#address-cells = <2>;
	#size-cells = <1>;
	reg = <f0010100 40>;

	ranges = <0 0 40000000 01000000
		  2 0 f2000000 00100000
		  3 0 f3000000 00100000
		  4 0 f4000000 00100000
		  5 0 f5000000 00100000>;

	fpga@4,0 {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges = <4 0 0 00010000>;

		msbus-arbitrer@2000 {
			compatible = "tbox,cp11-msbus-arbitrer";
			reg = <2000 4>;
		};

		msbus_pic: interrupt-controller@2100 {
			compatible = "tbox,cp11-msbus-pic";
			reg = <2100 8>;
			interrupts = <17 2>;
			interrupt-parent = <&cpm_pic>;
			#interrupt-cells = <1>;
			interrupt-controller;
		};

		msbus-spi@2200 {
			compatible = "tbox,cp11-msbus-spi";
			reg = <2200 100>;
			interrupts = <18 8>;
			interrupt-parent = <&cpm_pic>;
		};

		sdhc@5000 {
			compatible = "tbox,sdhci";
			reg = <5000 1000>;
			interrupts = <16 8>;
			interrupt-parent = <&cpm_pic>;
		};
	};

	msbus@5,0 {
		compatible = "tbox,cp11-msbus";
		#address-cells = <1>;
		#size-cells = <1>;
		#interrupt-cells = <1>;
		reg = <5 0 0 00000400>;
		interrupt-parent = <&msbus_pic>;
	};
};

The device tree reflects the physical topology but makes driver
access to the bus quite complex. An OF platform device driver
matching on compatible = "tbox,cp11-msbus" will not have the bus
FPGA registers described in its device node.

Having a look at the various device trees included in the kernel
sources, it seems platforms with a PCI bus experience a similar
problem. To solve it the PCI bus node address and registers describe
the configuration registers, and the memory window to access PCI
devices is described by the ranges property.  Applying that to my
custom bus would lead to the following tree.

localbus@f0010100 {
	compatible = "fsl,pq2-localbus";
	#address-cells = <2>;
	#size-cells = <1>;
	reg = <f0010100 40>;

	ranges = <0 0 40000000 01000000
		  2 0 f2000000 00100000
		  3 0 f3000000 00100000
		  4 0 f4000000 00100000
		  4 1 f4002000 00000100
		  5 0 f5000000 00100000>;

	fpga@4,0 {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges = <4 0 0 00010000>;

		msbus_pic: interrupt-controller@2100 {
			compatible = "tbox,cp11-msbus-pic";
			reg = <2100 8>;
			interrupts = <17 2>;
			interrupt-parent = <&cpm_pic>;
			#interrupt-cells = <1>;
			interrupt-controller;
		};

		msbus-spi@2200 {
			compatible = "tbox,cp11-msbus-spi";
			reg = <2200 100>;
			interrupts = <18 8>;
			interrupt-parent = <&cpm_pic>;
		};

		sdhc@5000 {
			compatible = "tbox,sdhci";
			reg = <5000 1000>;
			interrupts = <16 8>;
			interrupt-parent = <&cpm_pic>;
		};
	};

	msbus@4,1 {
		compatible = "tbox,cp11-msbus";
		#address-cells = <1>;
		#size-cells = <1>;
		#interrupt-cells = <1>;
		reg = <4 1 4>;
		interrupt-parent = <&msbus_pic>;
		ranges = <5 0 0 00000400>;
	};
};

Is this correct ? Is that the best way to describe my custom bus in
the device tree ?

Your second example looks closer to right.  Certainly you should use
'reg' only for bus control registers, and 'ranges' for windows into
the bus address space itself.

The device tree describes hardware from a functional point of view, so
I don't know that it's relevant that all the bus control functions are
implemented in an FPGA.  Each of the subnodes are more-or-less
independent devices, so they could just have separate nodes.

Or, if this seems more sensible, you could decide that they're
sufficiently closely related to put them all as one node, with
multiple register blocks listed in the 'reg' property.  That would
probably get messy for your PIC at the very least though.

How would the relationships between the bus and
its PIC and SPI controller be handled in the drivers ?

If the msbus driver needs to work with the associated PIC and SPI
controllers, then you should put properties in the msbus node giving
their phandles.

I also don't
understand how interrupt mappings are supposed to be handled. PCI
busses have two CPM interrupt lines, one for the PCI PIC and one for
the PCI bus, with the PCI bus having the CPM PIC as its interrupt
controller. My bus PIC uses a single interrupt line. Is there some
documentation explaining how PICs and interrupt mappings should be
described ?

Are interrupts from devices on the msbus routed over the msbus, or are
they routed independently to the mspic or the cpm PIC?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson