Hi Jean,

-----Original Message-----
From: Jean-Philippe Brucker [mailto:jean-philippe@linaro.org]
Sent: 04 March 2021 17:11
To: Shameerali Kolothum Thodi <redacted>
Cc: linux-arm-kernel@lists.infradead.org; iommu@lists.linux-foundation.org;
kvmarm@lists.cs.columbia.edu; maz@kernel.org;
alex.williamson@redhat.com; eric.auger@redhat.com;
zhangfei.gao@linaro.org; Jonathan Cameron
[off-list ref]; Zengtao (B) [off-list ref];
linuxarm@openeuler.org
Subject: Re: [RFC PATCH 4/5] iommu/arm-smmu-v3: Use pinned VMID for
NESTED stage with BTM

[...]

quoted

 	kfree(smmu_domain);

@@ -3199,6 +3230,17 @@ static int arm_smmu_attach_pasid_table(struct

iommu_domain *domain,

quoted

 				!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB))
 			goto out;

+		if (smmu->features & ARM_SMMU_FEAT_BTM) {
+			ret = arm_smmu_pinned_vmid_get(smmu_domain);
+			if (ret < 0)
+				goto out;
+
+			if (smmu_domain->s2_cfg.vmid)
+				arm_smmu_bitmap_free(smmu->vmid_map,

smmu_domain->s2_cfg.vmid);

quoted

+
+			smmu_domain->s2_cfg.vmid = (u16)ret;

That will require a TLB invalidation on the old VMID, once the STE is
rewritten.

More generally I think this pinned VMID set conflicts with that of
stage-2-only domains (which is the default state until a guest attaches a
PASID table). Say you have one guest using DOMAIN_NESTED without PASID
table, just DMA to IPA using VMID 0x8000. Now another guest attaches a
PASID table and obtains the same VMID from KVM. The stage-2 translation
might use TLB entries from the other guest, no?  They'll both create
stage-2 TLB entries with {StreamWorld=NS-EL1, VMID=0x8000}

Now that we are trying to align the KVM VMID allocation algorithm similar to
that of the ASID allocator [1], I attempted to use that for the SMMU pinned 
VMID allocation. But the issue you have mentioned above is still valid. 

And as a solution what I have tried now is follow what pinned ASID is doing 
in SVA,
 -Use xarray for private VMIDs
 -Get pinned VMID from KVM for DOMAIN_NESTED with PASID table
 -If the new pinned VMID is in use by private, then update the private
  VMID(VMID update to a live STE).

This seems to work, but still need to run more tests with this though.  

It's tempting to allocate all VMIDs through KVM instead, but that will
force a dependency on KVM to use VFIO_TYPE1_NESTING_IOMMU and might
break
existing users of that extension (though I'm not sure there are any).
Instead we might need to restrict the SMMU VMID bitmap to match the
private VMID set in KVM.

Another solution I have in mind is, make the new KVM VMID allocator common
between SMMUv3 and KVM. This will help to avoid all the private and shared
VMID splitting, also no need for live updates to STE VMID. One possible drawback
is less number of available KVM VMIDs but with 16 bit VMID space I am not sure
how much that is a concern.

Please let me know your thoughts.

Thanks,
Shameer

[1]. https://lore.kernel.org/kvmarm/20210616155606.2806-1-shameerali.kolothum.thodi@huawei.com/ (local)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel