RE: [RFC PATCH v4 03/16] iommu/arm-smmu-v3: Add initial pSMMU realm viommu plumbing

From: Jason Gunthorpe <jgg@ziepe.ca>
Sent: Thursday, September 10, 2026 8:47 PM

On Thu, Sep 10, 2026 at 09:52:12AM +0000, Tian, Kevin wrote:

quoted

quoted

The mmio is owned by vfio, there should be a handshake in VFIO to
remove mmio when it becomes private, otherwise I don't think we need
to do anything more?

for TDX a callback in VFIO is invoked to revoke the dmabuf.

btw I didn't see any dmabuf related in this series (even in a hackish
way as Yilun sent out last year). Is it not required by CCA or just
skipped now waiting for the framework to be settled?

I think just one step at a time :\ I think CCA needs it as well, you
also cannot safely leave private MMIO floating around the host
userspace on CCA.

Good to know

quoted

So viommu will become the abstraction point for tsm operations and
directly talks to the tsm driver instead of going through the merged
tsm_ops (bind/unbind/guest_req).

More precisely for what was being called the TDI - ie it is the
abstraction point for the host side of a guest virtual PCI device.

iommufd vdevice already represents this, so we use it again.

quoted

currently what tsm_ops provides additionally is more about synchronization
with connect/disconnect, otherwise just calls into underlying tsm driver. As
long as viommu creation holds a reference to tsm then disconnection is
blocked then it's safe.

Yes, this already has to be true or our lifecylce model is
nonsense. Once the iommufd creates the vdevice the guest is running
and we cannot disconnect it without unplugging it from the guest. Thus
the locking can rely on that. Within an iommufd ioctl context the vdev
and underlying connection must be stable.

btw looks a related suggestion is having the tsm driver (arm-cca-host
here) implement the viommu ops while leaving the original smmu driver
largely intact, and "obtain the viommu through tsm_ops not through
iommu_ops".

It's not super clear to me about the implication behind after reading
related discussions and what the "obtain" part means. 

Could you help elaborate this part?

The old way is kind of:

vdevice_tsm_ops -> tsm_ops -> tsm driver -> iommu driver

then it will become:

viommu_ops (tsm related) -> tsm_driver

where tsm driver will fully handle tsm related viommu_ops?

quoted

and IOMMU_VDEVICE_TSM_BIND will be removed.

Replaced, the same logical operations should flow through the viommu
command ioctl

quoted

Seems originally this series
does SMC_RMI_VDEV_CREATE and SMC_RMI_VDEV_LOCK both in tsm
bind op. Then the proposal is moving them all to vdevice creation time so
no separate bind step is required.

Hmm, lock still should be guest visible on CCA, and the device has to
start up in the guest as TDISP unlocked. I'm not 100% on the RMI side
of the flow, but this does not look right.

VDEV create should be done before the realm is started. We need this
setup right because Linux VM is going to validate the vdev during boot
if it is affiliated to a vSMMU. lock should be done only when
requested by the guest. The guest must start with a normal unlocked
T=0 PCI device.

Okay then this part is aligned cross all vendors. Thanks for clarification!

quoted

Does it mean that ARM CCA is essentially an early-bind model i.e. the TDI
starts in locked state from guest p.o.v.?

No

quoted

but https://lore.kernel.org/all/yq5aecfbzjyk.fsf@kernel.org/ (local) seems to

indicate

quoted

that guest_req() still handles the bind request from the guest:

"
- cca_tsm_guest_req() now handles TSM_REQ_SET_TDI_STATE requests for
   the unlocked, locked, and running states.
"

Right the guest side request to lock should come that way.

quoted

a bit confusing, but maybe due to stale info cited from different timing...

for TDX we're currently pursuing a late-bind model (i.e. TDI starts in shared
state in guest), so a separate bind interface is still necessary. We could look
at whether it should be an explicit @bind viommu op or carried by
@guest_req (say, if most preparation works can be moved to vdevice

creation).

It should go over the viommu command ioctl. It looked like there was
alway some iommu specific format to the command in the tsm version of
this, so that is the appropriate way.

quoted

btw per past discussions VFIO should block some operations (reset, etc.)
while a TDI is locked. Do we expect a callback into VFIO to notify the locked
state, or require VFIO's own interface to place a cdev into a special state
which prevents sensitive operations even before binding it to iommufd
(and allow creating TDI vdevice only on such idevice)?

At a minimum I think we have to synchronize with vfio, somehow, that
when the device enforces its private MMIO so it can unmap it. Thay may
be an argument we need a few standard viommu ops so the core code can
capture lock/unlocks and do this prep work. Or maybe a special cdev
state is simpler.

This should be figured out before any uapi is settled.

Agree

I don't know about reset, that sounds like something the pci core
should deal with?

yes the pci core sounds a better place to handle it.

btw I held the impression of blocked operations from past discussion [1].
Initial attempt tried to proactively unbind the TDI upon any operations 
which may transit the TDI to the ERROR state. Then the suggestion at
the moment was:

" But now the suggestion is never let VFIO do unbind, instead VFIO
should block these operations when device is bound. "

[1] https://lore.kernel.org/all/aEFmPaYorqaYCKBY@yilunxu-OptiPlex-7050/ (local)