Hi Jason,

On Wed, 31 Mar 2021 14:31:48 -0300, Jason Gunthorpe [off-list ref] wrote:

quoted

quoted

We should try to avoid hidden behind the scenes kernel
interconnections between subsystems.
  

Can we? in case of exception. Since all these IOCTLs are coming from the
unreliable user space, we must deal all exceptions.

For example, when user closes /dev/ioasid FD before (or w/o) unbind
IOCTL for VFIO, KVM, kernel must do cleanup and coordinate among
subsystems. In this patchset, we have a per mm(ioasid_set) notifier to
inform mdev, KVM to clean up and drop its refcount. Do you have any
suggestion on this?  

The ioasid should be a reference counted object.

yes, this is done in this patchset.

When the FD is closed, or the ioasid is "destroyed" it just blocks DMA
and parks the PASID until *all* places release it. Upon a zero
refcount the PASID is recycled for future use.

Just to clarify, you are saying (when FREE happens before proper
teardown) there is no need to proactively notify all users of the IOASID to
drop their reference. Instead, just wait for the other parties to naturally
close and drop their references. Am I understanding you correctly?

I feel having the notifications can add two values:
1. Shorten the duration of errors (as you mentioned below), FD close can
take a long and unpredictable time. e.g. FD shared.
2. Provide teardown ordering among PASID users. i.e. vCPU, IOMMU, mdev.

The duration between unmapping the ioasid and releasing all HW access
will have HW see PCIE TLP errors due to the blocked access. If
userspace messes up the order it is fine to cause this. We already had
this dicussion when talking about how to deal with process exit in the
simple SVA case.

Yes, we have disabled fault reporting during this period. The slight
differences vs. the simple SVA case is that KVM is also involved and there
might be an ordering requirement to stop vCPU first.

Thanks,

Jacob