On 6/7/2021 12:11 PM, Jason Gunthorpe wrote:

On Mon, Jun 07, 2021 at 11:13:04AM -0700, Dave Jiang wrote:

quoted

So in step 1, we 'tag' the wq to be dedicated to guest usage and put the
hardware wq into enable state. For a dedicated mode wq, we can definitely
just register directly and skip the mdev step. For a shared wq mode, we can
have multiple mdev running on top of a single wq. So we need some way to
create more mdevs. We can either go with the existing established creation
path by mdev, or invent something custom for the driver as Jason suggested
to accomodate additional virtual devices for guests. We implemented the mdev
path originally with consideration of mdev is established and has a known
interface already.

It sounds like you could just as easially have a 'create new vfio'
file under the idxd sysfs.. Especially since you already have a bus
and dynamic vfio specific things being created on this bus.

Will explore this and using of 'struct vfio_device' without mdev.

Have you gone over this with Dan?

quoted

I think things become more complicated when we go from a dedicated wq to
shared wq where the relationship of wq : mdev is 1 : 1 goes to 1 : N. Also
needing to keep a consistent user config experience is desired, especially
we already have such behavior since kernel 5.6 for host usages. So we really
need try to avoid doing wq configuration differently just for "mdev" wqs. In
the case suggested above, we basically just flipped the configuration steps.
Mdev is first created through mdev sysfs interface. And then the device
paramters are configured. Where for us, we configure the device parameter
first, and then create the mdev. But in the end, it's still the hybrid mdev
setup right?

So you don't even use mdev to configure anything? Yuk.

Jason