Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors
From: Sasha Levin <hidden>
Date: 2011-12-04 18:24:19
Also in:
kvm, lkml
Subsystem:
the rest, virtio core · Maintainers:
Linus Torvalds, "Michael S. Tsirkin", Jason Wang
Possibly related (same subject, not in this thread)
- 2011-12-09 · Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors · Rusty Russell <hidden>
- 2011-12-08 · Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors · Sasha Levin <hidden>
- 2011-12-07 · Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors · Avi Kivity <hidden>
- 2011-11-29 · Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors · "Michael S. Tsirkin" <mst@redhat.com>
- 2011-11-29 · Re: [PATCH] virtio-ring: Use threshold for switching to indirect descriptors · Sasha Levin <hidden>
On Sun, 2011-12-04 at 19:39 +0200, Sasha Levin wrote:
On Sun, 2011-12-04 at 19:37 +0200, Avi Kivity wrote:quoted
On 12/04/2011 07:34 PM, Sasha Levin wrote:quoted
quoted
I'm confused. didn't you see a bigger benefit for guest->host by switching indirect off?The 5% improvement is over the 'regular' indirect on, not over indirect off. Sorry for the confusion there. I suggested this change regardless of the outcome of indirect descriptor threshold discussion, since it would help anyways.For net, this makes sense. For block, it reduces the effective queue depth, so it's not a trivial change. It probably makes sense there too, though.It doesn't have to be limited at that number, anything above that can go through the regular kmalloc() path.
Something like the following patch:
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index c7a2c20..3166ca0 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c@@ -82,6 +82,7 @@ struct vring_virtqueue /* Host supports indirect buffers */ bool indirect; + struct kmem_cache *indirect_cache; /* Host publishes avail event idx */ bool event;
@@ -110,6 +111,9 @@ struct vring_virtqueue #define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq) +static unsigned int ind_alloc_thresh = 0; +module_param(ind_alloc_thresh, uint, S_IRUGO); + /* Set up an indirect table of descriptors and add it to the queue. */ static int vring_add_indirect(struct vring_virtqueue *vq, struct scatterlist sg[],
@@ -121,7 +125,10 @@ static int vring_add_indirect(struct vring_virtqueue *vq, unsigned head; int i; - desc = kmalloc((out + in) * sizeof(struct vring_desc), gfp); + if ((out + in) <= ind_alloc_thresh) + desc = kmem_cache_alloc(vq->indirect_cache, gfp); + else + desc = kmalloc((out + in) * sizeof(struct vring_desc), gfp); if (!desc) return -ENOMEM;
@@ -479,6 +486,9 @@ struct virtqueue *vring_new_virtqueue(unsigned int num, vq->broken = false; vq->last_used_idx = 0; vq->num_added = 0; + if (ind_alloc_thresh) + vq->indirect_cache = KMEM_CACHE(vring_desc[ind_alloc_thresh], 0); list_add_tail(&vq->vq.list, &vdev->vqs); #ifdef DEBUG vq->in_use = false;
--
Sasha.