[PATCH v4 15/15] iommu/arm-smmu-v3: Enable PRI for PCI device in arm_smmu_probe_device()
From: Nicolin Chen <hidden>
Date: 2026-09-10 23:18:11
Also in:
linux-iommu, linux-pci, lkml
Subsystem:
arm smmu drivers, iommu subsystem, the rest · Maintainers:
Will Deacon, Joerg Roedel, Linus Torvalds
PRI handling is now ready. Enable PRI when probing a PCI device, and drain the PRIQ with the EVTQ when an IOPF-enabled attachment ends. Queued fault work may still refer to the old domain when the detach call returns. Note that PRI is enabled at the probe time, while ATS gets toggled by the attach/detach routines, so a master could have PRI enabled when its ATS is disabled. PCIe (Base 6.3, Table 10-14) sets no ATS precondition on the PRI Enable bit. Its only ordering rule is that the interface must have gotten successfully Stopped prior to an enabling, which pci_enable_pri() already checks using PCI_PRI_STATUS_STOPPED. Also, a PRI-enabled device would not issue a page request until it starts to use ATS. Set the per-device outstanding request budget to the full priq depth, same as intel-iommu's per-device PRQ_DEPTH choice. A fixed per-device cap won't prevent multiple PRI-capable devices from potentially exceeding the priq's capacity; priq overflow is recoverable per the SMMUv3 spec, and it is rare in practice. Select PCI_PRI in Kconfig like other IOMMUs, gated on PCI so the build can stay clean for non-PCI ARM SMMUv3 configurations. A stalling master resolves its faults synchronously, so it has no reason at all to issue a page request. Refuse to enable PRI on such a master, so that page_response() can keep dispatching on the master state unambiguously. Refuse to enable PRI as well on any master reporting num_streams != 1, as arm_smmu_enable_iopf() rejects multi-stream masters, so IOPF cannot be set up for them; keeping PRI enabled would let a PRI request arrive on an alias StreamID and get a PRI_RESP_DENY issued against streams[0] by the driver's error-response path. Signed-off-by: Nicolin Chen <redacted> --- drivers/iommu/arm/Kconfig | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 52 +++++++++++++++++++++ 2 files changed, 53 insertions(+)
diff --git a/drivers/iommu/arm/Kconfig b/drivers/iommu/arm/Kconfig
index b848a4253677d..a31d04f5b031a 100644
--- a/drivers/iommu/arm/Kconfig
+++ b/drivers/iommu/arm/Kconfig@@ -80,6 +80,7 @@ config ARM_SMMU_V3 select IOMMU_IO_PGTABLE_LPAE select IOMMU_IOPF select GENERIC_MSI_IRQ + select PCI_PRI if PCI select IOMMUFD_DRIVER if IOMMUFD help Support for implementations of the ARM System MMU architecture
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 3cc2d365d2d66..104b0e008eb7f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c@@ -3513,6 +3513,20 @@ void arm_smmu_attach_release(struct arm_smmu_attach_state *state) } } + /* Same as the eventq drain above, for the hardware priq */ + if (master_domain->using_iopf && master->pri_enabled) { + timed_out |= arm_smmu_wait_for_queue_drained( + smmu, &smmu->priq.q, false); + /* Ensure pending requests have reached the IOPF queue */ + if (!timed_out) { + if (smmu->priq.q.irq) + synchronize_irq(smmu->priq.q.irq); + /* Pending requests might be in the combined_irq handler */ + if (smmu->combined_irq) + synchronize_irq(smmu->combined_irq); + } + } + /* Lastly, flush the fault work that the drained events queued */ if (master_domain->using_iopf) { iopf_queue_flush_dev(master->dev);
@@ -4445,6 +4459,40 @@ static int arm_smmu_master_prepare_ats(struct arm_smmu_master *master) return arm_smmu_alloc_cd_tables(master); } +static void arm_smmu_master_enable_pri(struct arm_smmu_master *master) +{ + struct arm_smmu_device *smmu = master->smmu; + struct pci_dev *pdev; + unsigned int reqs; + + if (!(smmu->features & ARM_SMMU_FEAT_PRI) || !smmu->evtq.iopf) + return; + if (!dev_is_pci(master->dev)) + return; + pdev = to_pci_dev(master->dev); + + if (!pci_pri_supported(pdev)) + return; + + /* A stalling master resolves its faults without any page request */ + if (master->stall_enabled) { + pci_warn(pdev, "stall enabled, skip PRI\n"); + return; + } + + if (master->num_streams != 1) { + pci_warn(pdev, "multi-SID master, skip PRI\n"); + return; + } + + reqs = 1 << smmu->priq.q.llq.max_n_shift; + + if (!pci_reset_pri(pdev) && !pci_enable_pri(pdev, reqs)) + master->pri_enabled = true; + else + pci_warn(pdev, "failed to enable PRI\n"); +} + static struct iommu_device *arm_smmu_probe_device(struct device *dev) { int ret;
@@ -4497,6 +4545,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) if (ret) goto err_disable_pasid; + arm_smmu_master_enable_pri(master); + return &smmu->iommu; err_disable_pasid:
@@ -4522,6 +4572,8 @@ static void arm_smmu_release_device(struct device *dev) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); } + if (master->pri_enabled) + pci_disable_pri(to_pci_dev(master->dev)); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); if (arm_smmu_cdtab_allocated(&master->cd_table))
--
2.43.0