On Wed, Jun 26, 2024 at 10:43:11AM +0200, Jiri Pirko wrote:

Wed, Jun 26, 2024 at 10:08:14AM CEST, mst@redhat.com wrote:

quoted

On Wed, Jun 26, 2024 at 09:52:58AM +0200, Jiri Pirko wrote:

quoted

Thu, Jun 20, 2024 at 12:31:34PM CEST, hengqi@linux.alibaba.com wrote:

quoted

On Thu, 20 Jun 2024 06:11:40 -0400, "Michael S. Tsirkin" [off-list ref] wrote:

quoted

On Thu, Jun 20, 2024 at 06:10:51AM -0400, Michael S. Tsirkin wrote:

quoted

On Thu, Jun 20, 2024 at 05:53:15PM +0800, Heng Qi wrote:

quoted

On Thu, 20 Jun 2024 16:26:05 +0800, Jason Wang [off-list ref] wrote:

quoted

On Thu, Jun 20, 2024 at 4:21 PM Jason Wang [off-list ref] wrote:

quoted

On Thu, Jun 20, 2024 at 3:35 PM Heng Qi [off-list ref] wrote:

quoted

On Wed, 19 Jun 2024 17:19:12 -0400, "Michael S. Tsirkin" [off-list ref] wrote:

quoted

On Thu, Jun 20, 2024 at 12:19:05AM +0800, Heng Qi wrote:

quoted

@@ -5312,7 +5315,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)

    /* Parameters for control virtqueue, if any */
    if (vi->has_cvq) {
-           callbacks[total_vqs - 1] = NULL;
+           callbacks[total_vqs - 1] = virtnet_cvq_done;
            names[total_vqs - 1] = "control";
    }

If the # of MSIX vectors is exactly for data path VQs,
this will cause irq sharing between VQs which will degrade
performance significantly.

Why do we need to care about buggy management? I think libvirt has
been teached to use 2N+2 since the introduction of the multiqueue[1].

And Qemu can calculate it correctly automatically since:

commit 51a81a2118df0c70988f00d61647da9e298483a4
Author: Jason Wang [off-list ref]
Date:   Mon Mar 8 12:49:19 2021 +0800

    virtio-net: calculating proper msix vectors on init

    Currently, the default msix vectors for virtio-net-pci is 3 which is
    obvious not suitable for multiqueue guest, so we depends on the user
    or management tools to pass a correct vectors parameter. In fact, we
    can simplifying this by calculating the number of vectors on realize.

    Consider we have N queues, the number of vectors needed is 2*N + 2
    (#queue pairs + plus one config interrupt and control vq). We didn't
    check whether or not host support control vq because it was added
    unconditionally by qemu to avoid breaking legacy guests such as Minix.

    Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com
    Reviewed-by: Stefano Garzarella [off-list ref]
    Reviewed-by: Stefan Hajnoczi [off-list ref]
    Signed-off-by: Jason Wang [off-list ref]

Yes, devices designed according to the spec need to reserve an interrupt
vector for ctrlq. So, Michael, do we want to be compatible with buggy devices?

Thanks.

These aren't buggy, the spec allows this. So don't fail, but
I'm fine with using polling if not enough vectors.

sharing with config interrupt is easier code-wise though, FWIW -
we don't need to maintain two code-paths.

Yes, it works well - config change irq is used less before - and will not fail.

Please note I'm working on such fallback for admin queue. I would Like
to send the patchset by the end of this week. You can then use it easily
for cvq.

Something like:
/* the config->find_vqs() implementation */
int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
                struct virtqueue *vqs[], vq_callback_t *callbacks[],
                const char * const names[], const bool *ctx,
                struct irq_affinity *desc)
{
        int err;

        /* Try MSI-X with one vector per queue. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_EACH, ctx, desc);
        if (!err)
                return 0;
        /* Fallback: MSI-X with one shared vector for config and
         * slow path queues, one vector per queue for the rest. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_SHARED_SLOW, ctx, desc);
        if (!err)
                return 0;
        /* Fallback: MSI-X with one vector for config, one shared for queues. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_SHARED, ctx, desc);
        if (!err)
                return 0;
        /* Is there an interrupt? If not give up. */
        if (!(to_vp_device(vdev)->pci_dev->irq))
                return err;
        /* Finally fall back to regular interrupts. */
        return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
}

Well for cvq, we'll need to adjust the API so core
knows cvq interrupts are be shared with config not
datapath.

Agreed. I was thinking about introducing some info struct and pass array
of it instead of callbacks[] and names[]. Then the struct can contain
flag indication. Something like:

struct vq_info {
	vq_callback_t *callback;
	const char *name;
	bool slow_path;
};

Yes. Add ctx too? There were attempts at it already btw.

quoted

quoted

quoted

Thanks.

quoted

quoted

quoted

quoted

Thanks

quoted

quoted

quoted

So no, you can not just do it unconditionally.

The correct fix probably requires virtio core/API extensions.

If the introduction of cvq irq causes interrupts to become shared, then
ctrlq need to fall back to polling mode and keep the status quo.

Having to path sounds a burden.

quoted

Thanks.

Thanks

[1] https://www.linux-kvm.org/page/Multiqueue

quoted

quoted

--
MST