On Fri, Aug 30, 2024 at 07:44:35AM +0000, Tian, Kevin wrote:

quoted

From: Jason Gunthorpe <jgg@nvidia.com>
Sent: Tuesday, August 27, 2024 11:52 PM

@@ -4189,6 +4193,13 @@ static int arm_smmu_device_hw_probe(struct

arm_smmu_device *smmu)

      /* IDR3 */
      reg = readl_relaxed(smmu->base + ARM_SMMU_IDR3);
+     /*
+      * If for some reason the HW does not support DMA coherency then
using
+      * S2FWB won't work. This will also disable nesting support.
+      */
+     if (FIELD_GET(IDR3_FWB, reg) &&
+         (smmu->features & ARM_SMMU_FEAT_COHERENCY))
+             smmu->features |= ARM_SMMU_FEAT_S2FWB;
      if (FIELD_GET(IDR3_RIL, reg))
              smmu->features |= ARM_SMMU_FEAT_RANGE_INV;

then also clear ARM_SMMU_FEAT_NESTING?

S2FWB isn't the only HW option for nesting. Pls refer to PATCH-8:
https://lore.kernel.org/linux-iommu/8-v2-621370057090+91fec-smmuv3_nesting_jgg@nvidia.com/ (local)

+static struct iommu_domain *
+arm_smmu_domain_alloc_nesting(struct device *dev, u32 flags,
[...]
+	/*
+	 * Must support some way to prevent the VM from bypassing the cache
+	 * because VFIO currently does not do any cache maintenance.
+	 */
+	if (!(fwspec->flags & IOMMU_FWSPEC_PCI_RC_CANWBS) &&
+	    !(master->smmu->features & ARM_SMMU_FEAT_S2FWB))
+		return ERR_PTR(-EOPNOTSUPP);

Thanks
Nicolin