[PATCH v4 07/15] iommu/arm-smmu-v3: Submit CMDQ_OP_PRI_RESP for IOPF event
From: Nicolin Chen <hidden>
Date: 2026-09-10 23:17:56
Also in:
linux-iommu, linux-pci, lkml
Subsystem:
arm smmu drivers, iommu subsystem, the rest · Maintainers:
Will Deacon, Joerg Roedel, Linus Torvalds
From: Malak Marrid <redacted> To handle IOMMU_FAULT_PAGE_REQ from the PRI queue, arm_smmu_page_response() must issue a CMDQ_OP_PRI_RESP back to the SMMU. Both EVTQ stall events and PRIQ requests reach the IOPF layer with the same IOMMU_FAULT_PAGE_REQ, but these two fault models are mutually exclusive. A stalling master holds a transaction until CMDQ_OP_RESUME and has no reason to send a page request. A later patch rejects PRI for such a master, so the response command can be selected from stall_enabled or pri_enabled. Note that a CMD_PRI_RESP.Resp encodes 0b00 as ResponseFailure (a permanent non-paging error), 0b01 as InvalidRequest (page-in unsuccessful), and 0b10 as Success. So IOMMU_PAGE_RESP_FAILURE maps to PRI_RESP_DENY (0b00) while IOMMU_PAGE_RESP_INVALID maps to PRI_RESP_FAIL (0b01), following the codes rather than the similarity of the enum names. Extend arm_smmu_enable_iopf() to also proceed for a PRI-enabled master, so that attaching a fault-capable domain would set up IOPF for it. Note that a later change will set master->pri_enabled, once all PRI paths are ready. Note: streams[0].id remains the RID because arm_smmu_enable_iopf() rejects num_streams != 1. Co-developed-by: Barak Biber <redacted> Signed-off-by: Barak Biber <redacted> Co-developed-by: Stefan Kaestle <redacted> Signed-off-by: Stefan Kaestle <redacted> Signed-off-by: Malak Marrid <redacted> Reviewed-by: Jonathan Cameron <redacted> Signed-off-by: Nicolin Chen <redacted> --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 74 +++++++++++++++------ 2 files changed, 56 insertions(+), 19 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 21b00b9296b31..558ba40f2f456 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h@@ -1017,6 +1017,7 @@ struct arm_smmu_master { /* Locked by the iommu core using the group mutex */ struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; + bool pri_enabled : 1; bool ats_enabled : 1; bool ste_ats_enabled : 1; bool stall_enabled;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 8b7d88d469198..815847c2d7b43 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c@@ -1031,32 +1031,68 @@ static int arm_smmu_wait_for_queue_drained(struct arm_smmu_device *smmu, return -ETIMEDOUT; } -static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused, +static void arm_smmu_page_response(struct device *dev, struct iopf_fault *evt, struct iommu_page_response *resp) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); - u8 resume_resp; + struct arm_smmu_cmd cmd; + int sid; - if (WARN_ON(!master->stall_enabled)) + if (WARN_ON_ONCE(evt->fault.type != IOMMU_FAULT_PAGE_REQ)) return; - switch (resp->code) { - case IOMMU_PAGE_RESP_INVALID: - case IOMMU_PAGE_RESP_FAILURE: - resume_resp = CMDQ_RESUME_0_RESP_ABORT; - break; - case IOMMU_PAGE_RESP_SUCCESS: - resume_resp = CMDQ_RESUME_0_RESP_RETRY; - break; - default: - resume_resp = CMDQ_RESUME_0_RESP_TERM; - break; + /* IOPF is gated to num_streams == 1 in arm_smmu_enable_iopf() */ + sid = master->streams[0].id; + + if (master->stall_enabled) { + u8 resume_resp; + + switch (resp->code) { + case IOMMU_PAGE_RESP_INVALID: + case IOMMU_PAGE_RESP_FAILURE: + resume_resp = CMDQ_RESUME_0_RESP_ABORT; + break; + case IOMMU_PAGE_RESP_SUCCESS: + resume_resp = CMDQ_RESUME_0_RESP_RETRY; + break; + default: + resume_resp = CMDQ_RESUME_0_RESP_TERM; + break; + } + cmd = arm_smmu_make_cmd_resume(sid, resp->grpid, resume_resp); + } else if (master->pri_enabled) { + struct iommu_fault_page_request *prm = &evt->fault.prm; + enum pri_resp pri_resp; + bool ssv; + + /* PCIe allows only one PRG Response per group */ + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) + return; + switch (resp->code) { + case IOMMU_PAGE_RESP_SUCCESS: + pri_resp = PRI_RESP_SUCC; + break; + case IOMMU_PAGE_RESP_FAILURE: + /* 0b00 ResponseFailure: a permanent non-paging error */ + pri_resp = PRI_RESP_DENY; + break; + case IOMMU_PAGE_RESP_INVALID: + /* 0b01 InvalidRequest: page-in unsuccessful */ + pri_resp = PRI_RESP_FAIL; + break; + default: + WARN_ON(true); + return; + } + ssv = !!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID); + cmd = arm_smmu_make_cmd_pri_resp(sid, resp->pasid, ssv, + resp->grpid, pri_resp); + } else { + WARN_ON_ONCE(1); + return; } - arm_smmu_cmdq_issue_cmd(master->smmu, - arm_smmu_make_cmd_resume(master->streams[0].id, - resp->grpid, - resume_resp)); + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); /* * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP. * RESUME consumption guarantees that the stalled transaction will be
@@ -3212,7 +3248,7 @@ static int arm_smmu_enable_iopf(struct arm_smmu_master *master, * device-specific fault handlers and don't need IOPF, so this is not a * failure. */ - if (!master->stall_enabled) + if (!master->stall_enabled && !master->pri_enabled) return 0; /* We're not keeping track of SIDs in fault events */
--
2.43.0