-----Original Message-----
From: Jason Gunthorpe <jgg@nvidia.com>
Sent: Wednesday, August 7, 2024 12:41 AM
To: acpica-devel@lists.linux.dev; Alex Williamson
[off-list ref]; Guohanjun (Hanjun Guo)
[off-list ref]; iommu@lists.linux.dev; Joerg Roedel
[off-list ref]; Kevin Tian [off-list ref]; kvm@vger.kernel.org;
Len Brown [off-list ref]; linux-acpi@vger.kernel.org; linux-arm-
kernel@lists.infradead.org; Lorenzo Pieralisi [off-list ref]; Rafael J.
Wysocki [off-list ref]; Robert Moore [off-list ref]; Robin
Murphy [off-list ref]; Sudeep Holla [off-list ref];
Will Deacon [off-list ref]
Cc: Eric Auger <eric.auger@redhat.com>; Jean-Philippe Brucker <jean-
philippe@linaro.org>; Moritz Fischer [off-list ref]; Michael Shavit
[off-list ref]; Nicolin Chen [off-list ref];
patches@lists.linux.dev; Shameerali Kolothum Thodi
[off-list ref]
Subject: [PATCH 2/8] iommu/arm-smmu-v3: Use S2FWB when available

Force Write Back (FWB) changes how the S2 IOPTE's MemAttr field
works. When S2FWB is supported and enabled the IOPTE will force cachable
access to IOMMU_CACHE memory and deny cachable access otherwise.

This is not especially meaningful for simple S2 domains, it apparently
doesn't even force PCI no-snoop access to be coherent.

However, when used with a nested S1, FWB has the effect of preventing the
guest from choosing a MemAttr that would cause ordinary DMA to bypass the
cache. Consistent with KVM we wish to deny the guest the ability to become
incoherent with cached memory the hypervisor believes is cachable so we
don't have to flush it.

Turn on S2FWB whenever the SMMU supports it and use it for all S2
mappings.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  6 ++++++
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  3 +++
 drivers/iommu/io-pgtable-arm.c              | 24 +++++++++++++++++----
 include/linux/io-pgtable.h                  |  2 ++
 4 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c

b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 531125f231b662..7fe1e27d11586c 100644

--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c

@@ -1612,6 +1612,8 @@ void arm_smmu_make_s2_domain_ste(struct

arm_smmu_ste *target,
 		FIELD_PREP(STRTAB_STE_1_EATS,
 			   ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0));

+	if (smmu->features & ARM_SMMU_FEAT_S2FWB)
+		target->data[1] |= cpu_to_le64(STRTAB_STE_1_S2FWB);
 	if (smmu->features & ARM_SMMU_FEAT_ATTR_TYPES_OVR)
 		target->data[1] |=
cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG,

STRTAB_STE_1_SHCFG_INCOMING));

@@ -2400,6 +2402,8 @@ static int arm_smmu_domain_finalise(struct

arm_smmu_domain *smmu_domain,
 		pgtbl_cfg.oas = smmu->oas;
 		fmt = ARM_64_LPAE_S2;
 		finalise_stage_fn = arm_smmu_domain_finalise_s2;
+		if (smmu->features & ARM_SMMU_FEAT_S2FWB)
+			pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_S2FWB;