Here are the two base patches for PCIe on pSeries for 2.6.17.
This is enough to probe, find, configure, and run basic I/O to PCIe
enabled devices.
The patches do not include PCIe hotplug, nor multiple MSI vectors per
adapter.
These patches have had limited testing, as firmware still is working on
a few MSI bugs of their own.
Added recognition of device_type: "pciex" and made PCIe portdrv
buildable on PPC64.
Signed-off-by: Jake Moilanen <redacted>
---
Index: 2.6.16/arch/powerpc/Kconfig
===================================================================
@@ -395,7 +395,7 @@dev->current_state=4;/* unknown power state */-if(!strcmp(type,"pci")){+if(!strcmp(type,"pci")||!strcmp(type,"pciex")){/* a PCI-PCI bridge */dev->hdr_type=PCI_HEADER_TYPE_BRIDGE;dev->rom_base_reg=PCI_ROM_ADDRESS1;
This patch hooks our current interrupt subsystem and sets up a single
vector MSI as if it was a LSI. Multiple MSI vectors is coming in the
future.
The NR_IRQS got bumped up to 1024, as vectors can go much higher.
Unfortunately, this number was arbitrarily picked as there is no claim
at what the max number really is by either the firmware team, or the
PAPR+.
Signed-off-by: Jake Moilanen <redacted>
---
Index: 2.6.16/arch/powerpc/kernel/prom.c
===================================================================
@@ -503,6 +529,80 @@DBG(" <- finish_device_tree\n");}+void__initfinish_msi_node(structdevice_node*dn,unsignedlong*mem_start,intmeasure_only)+{+staticintseq_num=1;+inti;+intrc;+intquery_token=rtas_token("ibm,query-interrupt-source-number");+intdevfn;+intbusno;+u32*reg;+intreglen;+intret[3];+unsignedintvirq;+unsignedintaddr;+unsignedlongbuid=-1;+unsignedlongwait_time;++reg=(u32*)get_property(dn,"reg",®len);+if(reg==NULL||reglen<20)+return;++devfn=(reg[0]>>8)&0xff;+busno=(reg[0]>>16)&0xff;++buid=get_phb_buid(dn->parent);+addr=(busno<<16)|(devfn<<8);++while(1){+rc=rtas_call(rtas_token("ibm,change-msi"),6,3,ret,addr,+buid>>32,buid&0xffffffff,+1,1,seq_num);++if(!rc)+break;+elseif(rc==RTAS_BUSY)+udelay(1);+elseif(rtas_is_extended_busy(rc)){+wait_time=rtas_extended_busy_delay_time(rc);+udelay(wait_time*1000);+}else{+printk(KERN_WARNING"error[%d]: getting the number of"+"MSI interrupts for %s\n",rc,dn->name);+return;+}++seq_num=ret[1];+}++dn->n_intrs=ret[0];++dn->intrs=prom_alloc(dn->n_intrs*sizeof(*(dn->intrs)),mem_start);+if(!dn->intrs){+printk(KERN_EMERG"finish_msi_node: can't allocate space\n");+return;+}++if(measure_only)+return;++for(i=0;i<dn->n_intrs;i++){+rc=rtas_call(query_token,4,3,ret,addr,buid>>32,buid&0xffffffff,i);++if(!rc){+virq=virt_irq_create_mapping(ret[0]);++dn->intrs[i].line=irq_offset_up(virq);+dn->intrs[i].sense=ret[1];+}else{+printk(KERN_WARNING"error[%d]: query-interrupt-source-number for %s\n",+rc,dn->name);+}+}++}+staticinlinechar*find_flat_dt_string(u32offset){return((char*)initial_boot_params)+
@@ -47,7 +47,7 @@/**Maximumnumberofinterruptsourcesthatwecanhandle.*/-#define NR_IRQS 512+#define NR_IRQS 1024/* Interrupt numbers are virtual in case they are sparsely*distributedbythehardware.
That one looks to have gone a wee bit over 80 columns.
I need to stop developing on my widescreen. :)
Signed-off-by: Jake Moilanen <redacted>
---
Index: 2.6.16/arch/powerpc/Kconfig
===================================================================
@@ -395,7 +395,7 @@dev->current_state=4;/* unknown power state */-if(!strcmp(type,"pci")){+if(!strcmp(type,"pci")||!strcmp(type,"pciex")){/* a PCI-PCI bridge */dev->hdr_type=PCI_HEADER_TYPE_BRIDGE;dev->rom_base_reg=PCI_ROM_ADDRESS1;
Am Saturday 01 April 2006 00:02 schrieb Jake Moilanen:
Here are the two base patches for PCIe on pSeries for 2.6.17.
This is enough to probe, find, configure, and run basic I/O to PCIe
enabled devices.
Does this use the draft PCIe bindings from Sun, or something else?
Is it compatible with the way that Apple represents PCIe devices
as plain PCI?
I'm just curious.
Arnd <><
Am Saturday 01 April 2006 00:02 schrieb Jake Moilanen:
quoted
Here are the two base patches for PCIe on pSeries for 2.6.17.
This is enough to probe, find, configure, and run basic I/O to PCIe
enabled devices.
Does this use the draft PCIe bindings from Sun, or something else?
Is it compatible with the way that Apple represents PCIe devices
as plain PCI?
I'm just curious.
From: Olof Johansson <hidden> Date: 2006-03-31 22:50:04
On Fri, Mar 31, 2006 at 04:13:30PM -0600, Jake Moilanen wrote:
This patch hooks our current interrupt subsystem and sets up a single
vector MSI as if it was a LSI. Multiple MSI vectors is coming in the
future.
The NR_IRQS got bumped up to 1024, as vectors can go much higher.
Unfortunately, this number was arbitrarily picked as there is no claim
at what the max number really is by either the firmware team, or the
PAPR+.
Please use of_find_property() instead of reimplementing it, i.e. do what
get_property() does now, just don't return the value.
I assume you can't use get_property() because the properties you look
for lack values, right?
@@ -47,7 +47,7 @@/**Maximumnumberofinterruptsourcesthatwecanhandle.*/-#define NR_IRQS 512+#define NR_IRQS 1024/* Interrupt numbers are virtual in case they are sparsely*distributedbythehardware.
I think the get_phb_buid() call from prom.c might break 32-bit, since
rtas_pci.c isn't always built there. An empty declaration here for
#ifndef CONFIG_<x> would take care of it.
From: Paul Mackerras <hidden> Date: 2006-04-01 08:36:07
Jake Moilanen writes:
The NR_IRQS got bumped up to 1024, as vectors can go much higher.
Unfortunately, this number was arbitrarily picked as there is no claim
at what the max number really is by either the firmware team, or the
PAPR+.
What matters is the number of different vectors, not the actual value
of the vectors, because we remap interrupt numbers that the firmware
gives us to logical Linux irq numbers between 0 and NR_IRQS-1. We had
to do that when the POWER5 systems came out, because the interrupt
numbers there occupy 24 bits.
Paul.
On Sat, 1 Apr 2006 19:36:07 +1100
Paul Mackerras [off-list ref] wrote:
Jake Moilanen writes:
quoted
The NR_IRQS got bumped up to 1024, as vectors can go much higher.
Unfortunately, this number was arbitrarily picked as there is no claim
at what the max number really is by either the firmware team, or the
PAPR+.
What matters is the number of different vectors, not the actual value
of the vectors, because we remap interrupt numbers that the firmware
gives us to logical Linux irq numbers between 0 and NR_IRQS-1. We had
to do that when the POWER5 systems came out, because the interrupt
numbers there occupy 24 bits.
Ah. That sounds right. I haven't had a chance to test this version of
the patch. Firmware is currently broken on my machine.
Here's try #2
I ended up putting #ifdef CONFIG_PPC_RTAS around this, as non-rtas
kernels won't like all the rtas calls.
Signed-off-by: Jake Moilanen <redacted>
---
Index: 2.6.16/arch/powerpc/kernel/prom.c
===================================================================
On Sat, 1 Apr 2006 19:36:07 +1100
Paul Mackerras [off-list ref] wrote:
quoted
Jake Moilanen writes:
quoted
The NR_IRQS got bumped up to 1024, as vectors can go much higher.
Unfortunately, this number was arbitrarily picked as there is no claim
at what the max number really is by either the firmware team, or the
PAPR+.
What matters is the number of different vectors, not the actual value
of the vectors, because we remap interrupt numbers that the firmware
gives us to logical Linux irq numbers between 0 and NR_IRQS-1. We had
to do that when the POWER5 systems came out, because the interrupt
numbers there occupy 24 bits.
Ah. That sounds right. I haven't had a chance to test this version of
the patch. Firmware is currently broken on my machine.
I was able to validate that these patches do work, and we are receiving
MSI interrupts correctly.
Is it too late to get into 2.6.17?
Jake
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2006-04-11 21:31:35
I was able to validate that these patches do work, and we are receiving
MSI interrupts correctly.
Is it too late to get into 2.6.17?
Yes and it's totally broken anyway. The whole idea of "hiding" MSIs as
if they were LSIs is totally irrealistic in practice. Our firmware might
do it but we have to undo it.
Linux defines the semantics of a driver probe() routine to be called
with MSIs disabled by default.
This is very important as for a lot of devices (I'd say the vast
majority of the devices that claim to be MSI capable), enabling MSI is
either broken (various HW bugs related to using them, mostly ordering
issues) or requires all sort of additional tweaking on the device side
in addition to the standard config space stuffs.
Thus, we need to help whoever is currently ripping off the
drivers/pci/msi.c code which is way too Intel-centric (there's an SGI
guy working on it already) and implement an arch specific implementation
that undoes what the firmware did, and then re-assign MSIs as requested
by drviers.
Some of the Boeblingen folks have already been looking into it as part
of some bare-metal/bringup effort, but that doesn't include PAPR
implementation.
Ben.
From: Christoph Hellwig <hch@lst.de> Date: 2006-04-12 04:05:16
On Fri, Mar 31, 2006 at 04:13:30PM -0600, Jake Moilanen wrote:
This patch hooks our current interrupt subsystem and sets up a single
vector MSI as if it was a LSI. Multiple MSI vectors is coming in the
future.
This is broken. Linux drivers expect MSI to be disabled on ->probe.
There's at least two reasons for that:
(1) Many devices that claim to implement MSI are actually broken in
more or less subtile ways. and thus must use traditition INTx pins.
(2) MSI defines relaxed semantics for dma synchronization. Silently
enabling MSI could cause subtile data corruption.