Re: kvm PCI assignment & VFIO ramblings

On 26.08.2011, at 04:33, Roedel, Joerg wrote:

On Fri, Aug 26, 2011 at 12:20:00AM -0400, David Gibson wrote:

quoted

On Wed, Aug 24, 2011 at 01:03:32PM +0200, Roedel, Joerg wrote:

quoted

On Wed, Aug 24, 2011 at 05:33:00AM -0400, David Gibson wrote:

quoted

On Wed, Aug 24, 2011 at 11:14:26AM +0200, Roedel, Joerg wrote:

=20

quoted

quoted

I don't see a reason to make this meta-grouping static. It would =

harm

quoted

quoted

quoted

quoted

flexibility on x86. I think it makes things easier on power but =

there

quoted

quoted

quoted

quoted

are options on that platform to get the dynamic solution too.

=20
I think several people are misreading what Ben means by "static".  =

I

quoted

quoted

quoted

would prefer to say 'persistent', in that the meta-groups lifetime =

is

quoted

quoted

quoted

not tied to an fd, but they can be freely created, altered and =

removed

quoted

quoted

quoted

during runtime.

=20
Even if it can be altered at runtime, from a usability perspective =

it is

quoted

quoted

certainly the best to handle these groups directly in qemu. Or are =

there

quoted

quoted

strong reasons to do it somewhere else?

=20
Funny, Ben and I think usability demands it be the other way around.

=20
The reason is that you mean the usability for the programmer and I =

mean

it for the actual user of qemu :)

No, we mean the actual user of qemu. The reason being that making a =
device available for any user space application is an administrative =
task.

Forget the KVM case for a moment and think of a user space device =
driver. I as a user am not root. But I as a user when having access to =
/dev/vfioX want to be able to access the device and manage it - and only =
it. The admin of that box needs to set it up properly for me to be able =
to access it.

So having two steps is really the correct way to go:

  * create VFIO group
  * use VFIO group

because the two are done by completely different users. It's similar to =
how tun/tap works in Linux too. Of course nothing keeps you from also =
creating a group on the fly, but it shouldn't be the only interface =
available. The persistent setup is definitely more useful.

=20

quoted

If the meta-groups are transient - that is lifetime tied to an fd -
then any program that wants to use meta-groups *must* know the
interfaces for creating one, whatever they are.
=20
But if they're persistent, the admin can use other tools to create =

the

quoted

meta-group then just hand it to a program to use, since the =

interfaces

quoted

for _using_ a meta-group are identical to those for an atomic group.
=20
This doesn't preclude a program from being meta-group aware, and
creating its own if it wants to, of course.  My guess is that qemu
would not want to build its own meta-groups, but libvirt probably
would.

=20
Doing it in libvirt makes it really hard for a plain user of qemu to
assign more than one device to a guest. What I want it that a user =

just

types
=20
	qemu -device vfio,host=3D00:01.0 -device vfio,host=3D00:02.0 ...
=20
and it just works. Qemu creates the meta-groups and they are
automatically destroyed when qemu exits. That the programs are not =

aware

of meta-groups is not a big problem because all software using vfio
needs still to be written :)
=20
Btw, with this concept the programmer can still decide to not use
meta-groups and just multiplex the mappings to all open device-fds it
uses.

What I want to see is:

  # vfio-create 00:01.0
    /dev/vfio0
  # vftio-create -a /dev/vfio0 00:02.0
    /dev/vfio0

  $ qemu -vfio dev=3D/dev/vfio0,id=3Dvfio0 -device vfio,vfio=3Dvfio0.0 =
-device vfio,vfio=3Dvfio0.1


Alex