On Mon, Jan 13, 2025 at 03:54:33PM -0400, Jason Gunthorpe wrote:

On Mon, Jan 13, 2025 at 11:47:52AM -0800, Nicolin Chen wrote:

quoted

quoted

The other approach would be to add a sequence number to each event and
let userspace detect the non-montonicity. It would require adding a
header to the native ARM evt.

Yea, I thought about that. The tricky thing is that the header will
be a core-level header pairing with a driver-level vEVENTQ type and
can never change its length, though we can define a 64-bit flag that
can reserve the other 63 bits for future use?

The header format could be revised by changing the driver specific
format tag.

Yea, we need a header format (or "header type") when the vEVENTQ
is allocated.

You'd want to push a special event when the first overflow happens and
probably also report a counter so userspace can know how many events
got lost.

How about this:

enum iommufd_veventq_header_type {
	IOMMU_VEVENTQ_HEADER_TYPE_V1,
};

enum iommu_hwpt_pgfault_flags {
	IOMMU_VEVENT_HEADER_FLAGS_OVERFLOW = (1 << 0),
};

struct iommufd_vevent_header_v1 {
	__u64 flags;
	__u32 num_events;
	__u32 num_overflows; // valid if flag_overflow is set
};

This seems most robust and simplest to implement..

I think I'd implement it by having a static overflow list entry so no
memory allocation is needed and just keep moving that entry to the
back of the list every time an event is lost. This way it will cover
lost events due to memory outages too

Could double-adding the same static node to the list happen and
corrupt the list?

I think the vevent_header doesn't need to be exactly match with
the driver event. So, maybe a vEVENTQ object could hold a header
structure during iommu_veventq_alloc?

struct iommufd_veventq {
	...
	atomic_t num_events;
	atomic_t num_overflows;
	DECLARE_BITMAP(errors, 32);
	struct iommufd_vevent_header_v1 *header;
};

The header is a bit redundant to its upper three members though.

For old formats like the fault queue you could return EOVERFLOW
whenever the sequence number becomes discontiguous or it sees the
overflow event..

So, IOMMU_OBJ_FAULT is safe to return EOVERFLOW via read(), as
you mentioned that it is self-limited right?

Thanks
Nicolin