MSI support on Linux PCI implementation for Ocotea

Hi,

I'm looking at the linux PCI implementation, especially MSI support
for Ocotea. And I have some observations and questions about it. Maybe
somebody here can shed some light on them. Thanks.

1. Obviously MSI is supported in Linux 2.6.x, maybe even in 2.4.x. But
MSI implementation seems to support only IOxAPIC on x86 or IA64
architectures, though the implemenation is in generic drivers/pci
tree.

2. Why is the message data defined in a such way shown as follows? Or
does it just follow IOxAPIC's design? Further question is who defines
the format of the message data. The PCI/PCIe specs don't say anything
about the content of the message data register.

struct msg_data {
	__u32	vector		:  8;
	__u32	delivery_mode	:  3;	/* 000b: FIXED | 001b: lowest prior */
	__u32	reserved_1	:  3;
	__u32	level		:  1;	/* 0: deassert | 1: assert */
	__u32	trigger		:  1;	/* 0: edge | 1: level */
	__u32	reserved_2	: 16;
} __attribute__ ((packed));

3. How is the pci subsystem initialized on a PPC platform? Is there
any existing document discussing PCI subsystem? Of course I've been
doing a lot internet search and couldn't find too much useful infor.

Thanks a lot,
-Shawn.

On Thu, 2006-06-15 at 14:14 -0700, Shawn Jin wrote:

Hi,

I'm looking at the linux PCI implementation, especially MSI support
for Ocotea. And I have some observations and questions about it. Maybe
somebody here can shed some light on them. Thanks.

Hi ! What is Ocotea ! :)

1. Obviously MSI is supported in Linux 2.6.x, maybe even in 2.4.x. But
MSI implementation seems to support only IOxAPIC on x86 or IA64
architectures, though the implemenation is in generic drivers/pci
tree.

Exact. And it's also a pile of crap.

2. Why is the message data defined in a such way shown as follows? Or
does it just follow IOxAPIC's design? Further question is who defines
the format of the message data. The PCI/PCIe specs don't say anything
about the content of the message data register.

 .../...

It's all intel specific junk.

3. How is the pci subsystem initialized on a PPC platform? Is there
any existing document discussing PCI subsystem? Of course I've been
doing a lot internet search and couldn't find too much useful infor.

pci host bridges are initialized early, generally in setup_arch() and
the PCI bus is probed later. You can see the code in
arch/powerpc/kernel/pci_{32,64}.c

Now, regarding MSI, as you have noticed, the situation isn't great. I'm
currently in the middle of reworking our interrupt management (and
interrupt numbers allocation layer) and Michael Ellermann is looking at
the MSI issue in parallel.

We don't have a solution yet but are working on it.

Ben.

    Benjamin> Hi ! What is Ocotea ! :)

It's an eval board for some PowerPC 440 ... 440GX I think (which has
the HW capability to support MSI on PCI-X iirc)

 - R.

Hi ! What is Ocotea ! :)

Did I spell it wrong?? I don't believe Ben doesn't know Ocotea.
Hmmm...Ocotea again? Let me check if I really spelled it wrong. Hey,
it's really Ocotea. ;)

Of course, there is no such MSI support specific for Ocotea.

quoted

1. Obviously MSI is supported in Linux 2.6.x, maybe even in 2.4.x. But
MSI implementation seems to support only IOxAPIC on x86 or IA64
architectures, though the implemenation is in generic drivers/pci
tree.

Exact. And it's also a pile of crap.

;) I'm not in a position where I can either agree or disagree with
you. :D But may I ask if ppc platform can make use of this crap in our
PCI express testing? That is, are the functionalities of this crap
working?

pci host bridges are initialized early, generally in setup_arch() and
the PCI bus is probed later. You can see the code in
arch/powerpc/kernel/pci_{32,64}.c

Now, regarding MSI, as you have noticed, the situation isn't great. I'm
currently in the middle of reworking our interrupt management (and
interrupt numbers allocation layer) and Michael Ellermann is looking at
the MSI issue in parallel.

We don't have a solution yet but are working on it.

It's good to know the development status. What's the relationship
between generic pci driver layer in drivers/pci and architecture
specific implementation? One thing I notice is that architectures
provide pcibios_init() and do resource allocation while the generic
driver provides PCI generic functions such as pci_scan_device().

Thanks,
-Shawn.

Hi Ben,

Now, regarding MSI, as you have noticed, the situation isn't great. I'm
currently in the middle of reworking our interrupt management (and
interrupt numbers allocation layer) and Michael Ellermann is looking at
the MSI issue in parallel.

Just one point about MSI(-X) message address and data format. I think
it should be the platform that defines the content. Current
implementation doesn't have this flexibility. We have a design where
MSI mode uses message data to carry vector number info while MSI-X
mode uses message address to carry this info. Although it seems
strange, but I think it's allowed by the spec.

Thanks,
-Shawn.

On Fri, 2006-06-16 at 14:45 -0700, Shawn Jin wrote:

Hi Ben,

quoted

Now, regarding MSI, as you have noticed, the situation isn't great. I'm
currently in the middle of reworking our interrupt management (and
interrupt numbers allocation layer) and Michael Ellermann is looking at
the MSI issue in parallel.

Just one point about MSI(-X) message address and data format. I think
it should be the platform that defines the content.

That's an obvious requirement :)

Current
implementation doesn't have this flexibility. We have a design where
MSI mode uses message data to carry vector number info while MSI-X
mode uses message address to carry this info. Although it seems
strange, but I think it's allowed by the spec.

The current implementation is bad :) There are some patches for altix
that are in -mm that add some platform hooks that make the situation a
little bit better, but it's still far from perfect.

Ben.