Legacy features in PCI Express devices

7 messages, 5 authors, 2017-03-13 · open the first message on its own page

Legacy features in PCI Express devices

From: Mason <hidden>
Date: 2017-03-13 16:10:57

Hello,

There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?

Can someone provide examples of such cards, so that I may test them
on both revisions?

I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)

Regards.

Legacy features in PCI Express devices

From: Greg <hidden>
Date: 2017-03-13 11:08:03

On Mon, 2017-03-13 at 17:10 +0100, Mason wrote:
Hello,

There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
I'm not sure about this...
  2) I/O address space
But yes, definitely some support this.

We're working on a new type of network controller that uses I/O for some
types of low latency feature support.

- Greg
Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?

Can someone provide examples of such cards, so that I may test them
on both revisions?

I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)

Regards.

Legacy features in PCI Express devices

From: robin.murphy@arm.com (Robin Murphy)
Date: 2017-03-13 17:12:41

On 13/03/17 16:10, Mason wrote:
Hello,

There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?

Can someone provide examples of such cards, so that I may test them
on both revisions?

I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)
Secondary to that category is endpoints which nominally support MSI, but
in a way which is unreliable or otherwise broken. My experience shows
that the Silicon Image SiI 3132 (as integrated on ARM Juno boards, but
seemingly also relatively common on 'generic' 2-port SATA cards) falls
into that category - using the command-line parameter to force MSIs
instead of legacy interrupts leads to the the machine barely reaching
userspace before something goes horribly wrong:

...
         Activating swap Swap Partition...
[   10.817806] Adding 524284k swap on /dev/sda2.  Priority:-1 extents:1
across:524284k SS
         Starting File System Check on /dev/disk/by-uuid/1000-9346...
[  OK  ] Activated swap Swap Partition.
[  OK  ] Activated swap
/dev/disk/by-partuui…6cda4-9f73-4e26-bf19-e71f9319b5ce.
[  OK  ] Reached target Swap.
         Mounting Temporary Directory...
[  OK  ] Mounted Temporary Directory.
[  OK  ] Started File System Check on /dev/disk/by-uuid/1000-9346.
[   46.036065] Unhandled fault: synchronous external abort (0x96000210)
at 0xffff0000092f5000
[   46.044273] Internal error: : 96000210 [#1] PREEMPT SMP
[   46.049445] Modules linked in:
[   46.052474] CPU: 0 PID: 122 Comm: scsi_eh_0 Not tainted 4.11.0-rc1+ #1753
[   46.059192] Hardware name: ARM Juno development board (r1) (DT)
[   46.065052] task: ffff8009763b8000 task.stack: ffff800975f70000
[   46.070918] PC is at ata_wait_register+0x2c/0xa0
[   46.075491] LR is at sil24_init_port+0x60/0x110
[   46.079974] pc : [<ffff0000085b342c>] lr : [<ffff0000085d2b78>]
pstate: 40000145
...

Robin.

Legacy features in PCI Express devices

From: Mason <hidden>
Date: 2017-03-13 17:39:55

On 13/03/2017 18:12, Robin Murphy wrote:
On 13/03/17 16:10, Mason wrote:
quoted
There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?

Can someone provide examples of such cards, so that I may test them
on both revisions?

I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)
Secondary to that category is endpoints which nominally support MSI, but
in a way which is unreliable or otherwise broken. My experience shows
that the Silicon Image SiI 3132 (as integrated on ARM Juno boards, but
seemingly also relatively common on 'generic' 2-port SATA cards) falls
into that category - using the command-line parameter to force MSIs
instead of legacy interrupts leads to the the machine barely reaching
userspace before something goes horribly wrong:
Do drivers typically support *both* MSI and INTx?

Specifically, would the xhci driver support both?

If I remove MSI support from my kernel, I might be able to test
legacy interrupt support that way, right?

Regards.

Legacy features in PCI Express devices

From: robin.murphy@arm.com (Robin Murphy)
Date: 2017-03-13 17:55:46

On 13/03/17 17:39, Mason wrote:
On 13/03/2017 18:12, Robin Murphy wrote:
quoted
On 13/03/17 16:10, Mason wrote:
quoted
There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?

Can someone provide examples of such cards, so that I may test them
on both revisions?

I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)
Secondary to that category is endpoints which nominally support MSI, but
in a way which is unreliable or otherwise broken. My experience shows
that the Silicon Image SiI 3132 (as integrated on ARM Juno boards, but
seemingly also relatively common on 'generic' 2-port SATA cards) falls
into that category - using the command-line parameter to force MSIs
instead of legacy interrupts leads to the the machine barely reaching
userspace before something goes horribly wrong:
Do drivers typically support *both* MSI and INTx?
I'm not sure about "typically", but it certainly happens. For example,
the Intel e1000e NIC driver is one I know of which can fall back from
MSI-X to MSI to legacy dynamically.
Specifically, would the xhci driver support both?
Line 415 of xhci.c would appear to imply so.
If I remove MSI support from my kernel, I might be able to test
legacy interrupt support that way, right?
Indeed, disabling CONFIG_PCI_MSI should leave drivers with no other choice.

Robin.
Regards.

Legacy features in PCI Express devices

From: David Daney <hidden>
Date: 2017-03-13 17:24:53

On 03/13/2017 09:10 AM, Mason wrote:
Hello,

There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?
It depends on your definition of "modern".  I have some JMicron AHCI 
SATA controllers that support legacy interrupts only.  These are cheap 
$20 PCIe devices I picked up at Fry's a couple of years ago.  Do they 
count as modern?

I/O address space is probably less important I would say.

David.
Can someone provide examples of such cards, so that I may test them
on both revisions?

[root@localhost ddaney]# lspci -s 0005:90:00.0 -vvv
0005:90:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE 
Controller (rev 03) (prog-if 01 [AHCI 1.0])
	Subsystem: JMicron Technology Corp. JMB363 SATA/IDE Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 26
	NUMA node: 0
	Region 5: Memory at 899010010000 (32-bit, non-prefetchable) [size=8K]
	Expansion ROM at 899010000000 [disabled] [size=64K]
	Capabilities: [68] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] Express (v1) Legacy Endpoint, MSI 01
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr+ UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s 
unlimited, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- 
BWMgmt- ABWMgmt-
	Kernel driver in use: ahci

[root@localhost ddaney]# lspci -s 0005:90:00.0 -vvv -n
0005:90:00.0 0106: 197b:2363 (rev 03) (prog-if 01 [AHCI 1.0])
	Subsystem: 197b:2363
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 26
	NUMA node: 0
	Region 5: Memory at 899010010000 (32-bit, non-prefetchable) [size=8K]
	Expansion ROM at 899010000000 [disabled] [size=64K]
	Capabilities: [68] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] Express (v1) Legacy Endpoint, MSI 01
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr+ UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s 
unlimited, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- 
BWMgmt- ABWMgmt-
	Kernel driver in use: ahci



Look!  No MSI.
I was told to check ath9k-based cards. Any other examples?

Looking around, I came across this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/418254.html
"i.MX6 PCIe: Fix imx6_pcie_deassert_core_reset() polarity"

IIUC, although some PCIe boards do support MSI, the driver might not
put in the work to use that infrastructure, and instead reverts to
legacy interrupts. (So it is a SW issue, in a sense.)

Regards.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Legacy features in PCI Express devices

From: helgaas@kernel.org (Bjorn Helgaas)
Date: 2017-03-13 18:55:04

On Mon, Mar 13, 2017 at 05:10:57PM +0100, Mason wrote:
Hello,

There are two revisions of our PCI Express controller.

Rev 1 did not support the following features:

  1) legacy PCI interrupt delivery (INTx signals)
  2) I/O address space

Internally, someone stated that such missing support would prevent
some PCIe cards from working with our controller.

Are there really modern PCIe cards that require 1) and/or 2)
to function?
From a spec point of view, all endpoints, including Legacy, PCI
Express, and Root Complex Integrated Endpoints, are "required to
support MSI or MSI-X or both if an interrupt resource is requested"
(PCIe r3.0, sec 1.3.2).

The same section says Legacy Endpoints are permitted to use I/O
Requests, but PCI Express and Root Complex Integrated Endpoints are
not.  There's a little wiggle room in the I/O BAR description; I would
interpret it to mean the latter two varieties are permitted to have
I/O BARs, but they must make the resources described by those BARs
available via a memory BAR as well, so they can operate with I/O
address space.

But that's only in theory; David has already given examples of devices
that don't support MSI, and Greg hinted at new devices that might
require I/O space.  I'm particularly curious about that last one,
because there are several host bridges that don't support I/O space at
all.

Bjorn
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help