[dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

7 messages, 3 authors, 2021-05-13 · open the first message on its own page

[dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Wan Junjie <hidden>
Date: 2021-02-02 08:24:54

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
 lib/librte_vhost/vhost.c      | 4 ++--
 lib/librte_vhost/vhost_user.c | 6 +++++-
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c
index b83cf639e..e5341ab98 100644
--- a/lib/librte_vhost/vhost.c
+++ b/lib/librte_vhost/vhost.c

@@ -352,7 +352,8 @@ free_vq(struct virtio_net *dev, struct vhost_virtqueue *vq)
 		vhost_free_async_mem(vq);
 	}
 	rte_free(vq->batch_copy_elems);
-	rte_mempool_free(vq->iotlb_pool);
+	if (vq->iotlb_pool)
+		rte_mempool_free(vq->iotlb_pool);
 	rte_free(vq);
 }

@@ -556,7 +557,6 @@ init_vring_queue(struct virtio_net *dev, uint32_t vring_idx)
 	vq->callfd = VIRTIO_UNINITIALIZED_EVENTFD;
 	vq->notif_enable = VIRTIO_UNINITIALIZED_NOTIF;
 
-	vhost_user_iotlb_init(dev, vring_idx);
 	/* Backends are set to -1 indicating an inactive device. */
 	vq->backend = -1;
 }

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 45c8ac09d..81cbb9797 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c

@@ -470,6 +470,10 @@ vhost_user_set_vring_num(struct virtio_net **pdev,
 		return RTE_VHOST_MSG_RESULT_ERR;
 	}
 
+	if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) {
+		if (vhost_user_iotlb_init(dev, msg->payload.state.index))
+			return RTE_VHOST_MSG_RESULT_ERR;
+	}
 	return RTE_VHOST_MSG_RESULT_OK;
 }

@@ -575,7 +579,7 @@ numa_realloc(struct virtio_net *dev, int index)
 	dev->virtqueue[index] = vq;
 	vhost_devices[dev->vid] = dev;
 
-	if (old_vq != vq)
+	if (old_vq != vq && (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)))
 		vhost_user_iotlb_init(dev, index);
 
 	return dev;

-- 
2.11.0

Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Xia, Chenbo <hidden>
Date: 2021-04-07 07:10:30

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: David Marchand <hidden>
Date: 2021-05-13 08:39:19

On Wed, Apr 7, 2021 at 9:10 AM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

This patch triggered a regression reported by Red Hat QE.
https://bugs.dpdk.org/show_bug.cgi?id=703

Reverting the patch is enough to fix the regression.

I'll let you guys decide on the next step for 21.05.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Xia, Chenbo <hidden>
Date: 2021-05-13 10:24:11

-----Original Message-----
From: David Marchand <redacted>
Sent: Thursday, May 13, 2021 4:39 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: Wan Junjie <redacted>; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while
IOMMU disabled

On Wed, Apr 7, 2021 at 9:10 AM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU

disabled

quoted

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

This patch triggered a regression reported by Red Hat QE.
https://bugs.dpdk.org/show_bug.cgi?id=703

Reverting the patch is enough to fix the regression.

I'll let you guys decide on the next step for 21.05.


--
David Marchand

I cannot reproduce this issue on my machine.

@Wan Junjie Could you reproduce this? I prefer to revert the patch if things
are not very clear as it is only a minor optimization.

Thanks,
Chenbo

Re: [dpdk-dev] [External] RE: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Wan Junjie <hidden>
Date: 2021-05-13 10:52:06

On Thu, May 13, 2021 at 6:24 PM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: David Marchand <redacted>
Sent: Thursday, May 13, 2021 4:39 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: Wan Junjie <redacted>; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation while
IOMMU disabled

On Wed, Apr 7, 2021 at 9:10 AM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU

disabled

quoted

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

This patch triggered a regression reported by Red Hat QE.
https://bugs.dpdk.org/show_bug.cgi?id=703

quoted

Reverting the patch is enough to fix the regression.

I'll let you guys decide on the next step for 21.05.


--
David Marchand

I cannot reproduce this issue on my machine.

@Wan Junjie Could you reproduce this? I prefer to revert the patch if things
are not very clear as it is only a minor optimization.

Thanks,
Chenbo

Hi Chenbo,

I am not available for this now. Please revert it.
Will dig it later.

Thanks,
Junjie

Re: [dpdk-dev] [External] RE: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Xia, Chenbo <hidden>
Date: 2021-05-13 10:58:34

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Thursday, May 13, 2021 6:52 PM
To: Xia, Chenbo <redacted>
Cc: David Marchand <redacted>; Maxime Coquelin
[off-list ref]; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref];
Thomas Monjalon [off-list ref]
Subject: Re: [External] RE: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool
allocation while IOMMU disabled

On Thu, May 13, 2021 at 6:24 PM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: David Marchand <redacted>
Sent: Thursday, May 13, 2021 4:39 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: Wan Junjie <redacted>; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation

while

quoted

IOMMU disabled

On Wed, Apr 7, 2021 at 9:10 AM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang

[off-list ref]

quoted

Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU

disabled

quoted

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

This patch triggered a regression reported by Red Hat QE.
https://bugs.dpdk.org/show_bug.cgi?id=703

quoted

Reverting the patch is enough to fix the regression.

I'll let you guys decide on the next step for 21.05.


--
David Marchand

I cannot reproduce this issue on my machine.

@Wan Junjie Could you reproduce this? I prefer to revert the patch if things
are not very clear as it is only a minor optimization.

Thanks,
Chenbo

Hi Chenbo,

I am not available for this now. Please revert it.
Will dig it later.

Thanks,
Junjie

Luckily I can reproduce now. I will take a look.

Chenbo

Re: [dpdk-dev] [External] RE: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU disabled

From: Xia, Chenbo <hidden>
Date: 2021-05-13 11:41:57

Hi,

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Thursday, May 13, 2021 6:52 PM
To: Xia, Chenbo <redacted>
Cc: David Marchand <redacted>; Maxime Coquelin
[off-list ref]; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref];
Thomas Monjalon [off-list ref]
Subject: Re: [External] RE: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool
allocation while IOMMU disabled

On Thu, May 13, 2021 at 6:24 PM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: David Marchand <redacted>
Sent: Thursday, May 13, 2021 4:39 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: Wan Junjie <redacted>; dev@dpdk.org; Peng He
[off-list ref]; Zhihong Wang [off-list ref]
Subject: Re: [dpdk-dev] [PATCH v2] vhost: avoid iotlb mempool allocation

while

quoted

IOMMU disabled

On Wed, Apr 7, 2021 at 9:10 AM Xia, Chenbo [off-list ref] wrote:

quoted

-----Original Message-----
From: Wan Junjie <redacted>
Sent: Tuesday, February 2, 2021 4:15 PM
To: Xia, Chenbo <redacted>; Maxime Coquelin
[off-list ref]
Cc: dev@dpdk.org; Wan Junjie <redacted>; Peng He
[off-list ref]; Zhihong Wang

[off-list ref]

quoted

Subject: [PATCH v2] vhost: avoid iotlb mempool allocation while IOMMU

disabled

quoted

If vhost device's IOMMU feature is disabled, iotlb mempool allocation
is unnecessary.

Reported-by: Peng He <redacted>
Signed-off-by: Wan Junjie <redacted>
Reviewed-by: Zhihong Wang <redacted>
---
2.11.0

Patch applied to next-virtio/main with conflict resolved, Thanks

This patch triggered a regression reported by Red Hat QE.
https://bugs.dpdk.org/show_bug.cgi?id=703

quoted

Reverting the patch is enough to fix the regression.

I'll let you guys decide on the next step for 21.05.


--
David Marchand

I cannot reproduce this issue on my machine.

@Wan Junjie Could you reproduce this? I prefer to revert the patch if things
are not very clear as it is only a minor optimization.

Thanks,
Chenbo

Hi Chenbo,

I am not available for this now. Please revert it.
Will dig it later.

Thanks,
Junjie

I know the root cause, it is because when multi-queue is on, qemu only enable the
first queue pair and does not 'VHOST_USER_SET_VRING_NUM' for other queue pairs, which leads
to disabled queue pairs' iotlb_pool not init-ed. Then comes the iotlb message that operate
on all queue pairs iotlb pool.

A fix is coming.

Thanks,
Chenbo

`h`	back out one level
`j`	next message in thread
`k`	previous message in thread
`l`	drill in
`Esc`	close help / fold thread tree
`?`	toggle this help