On 2020/12/18 上午6:28, Michael S. Tsirkin wrote:

On Thu, Dec 17, 2020 at 05:02:49PM +0800, Jason Wang wrote:

quoted

On 2020/12/17 下午3:58, Michael S. Tsirkin wrote:

quoted

On Thu, Dec 17, 2020 at 11:30:18AM +0800, Jason Wang wrote:

quoted

On 2020/12/16 下午5:47, Michael S. Tsirkin wrote:

quoted

On Wed, Dec 16, 2020 at 02:47:57PM +0800, Jason Wang wrote:

quoted

Hi All:

This series tries to add the support for control virtqueue in vDPA.

Control virtqueue is used by networking device for accepting various
commands from the driver. It's a must to support multiqueue and other
configurations.

When used by vhost-vDPA bus driver for VM, the control virtqueue
should be shadowed via userspace VMM (Qemu) instead of being assigned
directly to Guest. This is because Qemu needs to know the device state
in order to start and stop device correctly (e.g for Live Migration).

This requies to isolate the memory mapping for control virtqueue
presented by vhost-vDPA to prevent guest from accesing it directly.
To achieve this, vDPA introduce two new abstractions:

- address space: identified through address space id (ASID) and a set
                    of memory mapping in maintained
- virtqueue group: the minimal set of virtqueues that must share an
                    address space

How will this support the pretty common case where control vq
is programmed by the kernel through the PF, and others by the VFs?

In this case, the VF parent need to provide a software control vq and decode
the command then send them to VF.

But how does that tie to the address space infrastructure?

In this case, address space is not a must.

That's ok, problem is I don't see how address space is going
to work in this case at all.

There's no address space there that userspace/guest can control.

The virtqueue group is mandated by parent but the association between 
virtqueue group and address space is under the control of userspace (Qemu).

A simple but common case is that:

1) Device advertise two virtqueue groups: group 0 contains RX and TX, 
group 1 contains CVQ.
2) Device advertise two address spaces

Then, for vhost-vDPA using by VM:

1) associate group 0 with as 0, group 1 with as 1 (via vhost-vDPA 
VHOST_VDPA_SET_GROUP_ASID)
2) Publish guest memory mapping via IOTLB asid 0
3) Publish control virtqueue mapping via IOTLB asid 1

Then the DMA is totally isolated in this case.

For vhost-vDPA using by DPDK or virtio-vDPA

1) associate group 0 and group 1 with as 0

since we don't need DMA isolation in this case.

In order to let it be controlled by Guest, we need extend virtio spec to 
support those concepts.

Thanks