Re: MMIO/PIO dispatch file descriptors (ioregionfd) design discussion

On Tue, Oct 26, 2021 at 08:01:39PM +0100, John Levon wrote:

On Mon, Oct 25, 2021 at 08:21:22AM -0700, Elena wrote:

quoted

quoted

I'm curious what approach you want to propose for QEMU integration. A
while back I thought about the QEMU API. It's possible to implement it
along the lines of the memory_region_add_eventfd() API where each
ioregionfd is explicitly added by device emulation code. An advantage of
this approach is that a MemoryRegion can have multiple ioregionfds, but
I'm not sure if that is a useful feature.

This is the approach that is currently in the works. Agree, I dont see
much of the application here at this point to have multiple ioregions
per MemoryRegion.
I added Memory API/eventfd approach to the vfio-user as well to try
things out.

quoted

An alternative is to cover the entire MemoryRegion with one ioregionfd.
That way the device emulation code can use ioregionfd without much fuss
since there is a 1:1 mapping between MemoryRegions, which are already
there in existing devices. There is no need to think deeply about which
ioregionfds to create for a device.

A new API called memory_region_set_aio_context(MemoryRegion *mr,
AioContext *ctx) would cause ioregionfd (or a userspace fallback for
non-KVM cases) to execute the MemoryRegion->read/write() accessors from
the given AioContext. The details of ioregionfd are hidden behind the
memory_region_set_aio_context() API, so the device emulation code
doesn't need to know the capabilities of ioregionfd.

quoted

The second approach seems promising if we want more devices to use
ioregionfd inside QEMU because it requires less ioregionfd-specific
code.

I like this approach as well.
As you have mentioned, the device emulation code with first approach
does have to how to handle the region accesses. The second approach will
make things more transparent. Let me see how can I modify what there is
there now and may ask further questions.

Sorry I'm a bit late to this discussion, I'm not clear on the above WRT
vfio-user. If an ioregionfd has to cover a whole BAR0 (?), how would this
interact with partly-mmap()able regions like we do with SPDK/vfio-user/NVMe?

The ioregionfd doesn't need to cover an entire BAR. QEMU's MemoryRegions
form a hierarchy, so it's possible to sub-divide the BAR into several
MemoryRegions.

This means it's still possible to have mmap() sub-regions or even
ioeventfds sprinkled in between.

Stefan