Re: [dpdk-dev] [PATCH] net/iavf: fix vector id assignment
From: Yu, DapengX <hidden>
Date: 2021-01-12 06:46:21
Hi Ting,
The following test case is for i40e. if you want to test the case on ice, just ignore step 1.
Step1: switch i40evf to iavf
# sed -i '/{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },/a { RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_VF) },' drivers/net/iavf/iavf_ethdev.c
# sed -i -e '/I40E_DEV_ID_VF/s/0x154C/0x164C/g' drivers/net/i40e/base/i40e_devids.h
# sed -i -e '/DEV_RX_OFFLOAD_CHECKSUM,/d' ./examples/l3fwd-power/main.c
Step2. build dpdk and l3fwd-power
Step3. create 1 vf
# echo 1 > /sys/bus/pci/devices/0000\:81\:00.0/sriov_numvfs
Step4. bind vf to vfio-pci
# usertools/dpdk-devbind.py --force --bind=vfio-pci 81:02.0
Step5. launch l3fwd-power
# ./x86_64-native-linuxapp-gcc/examples/dpdk-l3fwd-power -l 0-1 -n 4 -- -P -p 0x1 --config='(0,0,0),(0,1,1)'
Expected result:
start l3fwd-power successfully
Actual result:
fail to start l3fwd-power
-----Original Message-----
From: Xu, Ting
Sent: Tuesday, January 12, 2021 2:27 PM
To: Yu, DapengX <redacted>; Zhang, Qi Z <redacted>; Wu, Jingjing <redacted>; Xing, Beilei <redacted>
Cc: dev@dpdk.org; Yu, DapengX <redacted>; stable@dpdk.org
Subject: RE: [PATCH] net/iavf: fix vector id assignment
-----Original Message----- From: dapengx.yu@intel.com <redacted> Sent: Friday, January 8, 2021 6:21 PM To: Zhang, Qi Z <redacted>; Wu, Jingjing [off-list ref]; Xing, Beilei [off-list ref]; Xu, Ting [off-list ref] Cc: dev@dpdk.org; Yu, DapengX <redacted>; stable@dpdk.org Subject: [PATCH] net/iavf: fix vector id assignment From: YU DAPENG <redacted> The number of MSI-X interrupts on Rx shall be the minimal value of the number of available MSI-X interrupts per VF - 1 (the 1 is for miscellaneous interrupt) and the number of configured Rx queues. The current code break the rule because the number of available MSI-X interrupts is used as the first value, but code does not subtract 1 from it. In normal situation, the first value is larger than the second value. So each queue can be assigned a unique vector_id. For example: 17 available MSI-X interrupts, and 16 available Rx queues per VF; but only 4 Rx queues are configured when device is started. vector_id:0 is for misc interrupt, vector_id:1 for Rx queue0, vector_id:2 for Rx queue1, vector_id:3 for Rx queue2, vector_id:4 for Rx queue3. Current code breaks the rule in this normal situation, because when assign vector_ids to interrupt handle, for example, it does not assign vector_id:4 to the queue3, but assign vector_id:1 to it, because the condition used causes vector_id wrap around too early.
Hi, Dapeng, Could you please further explain in which condition will this error happen? Seems it requires vf->nb_msix = 3 to make it happen, but I do not notice such situation. I know it may be an example, is there any more specific case? Thanks.
quoted hunk ↗ jump to hunk
In iavf_config_irq_map(), the current code does not write data into the last element of vecmap[], because of the previous code break. Which cause wrong data is sent to PF with opcode VIRTCHNL_OP_CONFIG_IRQ_MAP and cause error: VIRTCHNL_STATUS_ERR_PARAM(-5). If kernel driver supports large VFs (up to 256 queues), different queues can be assigned same vector_id. In order to adapt to large VFs and avoid wrapping early, the condition is replaced from vec >= vf->nb_msix to vec >= vf->vf_res->max_vectors. Fixes: d6bde6b5eae9 ("net/avf: enable Rx interrupt") Cc: stable@dpdk.org Signed-off-by: YU DAPENG <redacted> --- drivers/net/iavf/iavf_ethdev.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)diff --git a/drivers/net/iavf/iavf_ethdev.cb/drivers/net/iavf/iavf_ethdev.c index 7e3c26a94..d730bb156 100644--- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c@@ -483,6 +483,7 @@ static int iavf_config_rx_queues_irqs(structrte_eth_dev *dev, struct iavf_qv_map *qv_map; uint16_t interval, i; int vec; +uint16_t max_vectors; if (rte_intr_cap_multiple(intr_handle) && dev->data->dev_conf.intr_conf.rxq) { @@ -570,15 +571,16 @@ static int iavf_config_rx_queues_irqs(struct rte_eth_dev *dev, /* If Rx interrupt is reuquired, and we can use * multi interrupts, then the vec is from 1 */ -vf->nb_msix = RTE_MIN(vf->vf_res->max_vectors, - intr_handle->nb_efd); +max_vectors = +vf->vf_res->max_vectors - IAVF_RX_VEC_START; +vf->nb_msix = RTE_MIN(max_vectors, intr_handle-quoted
nb_efd);vf->msix_base = IAVF_RX_VEC_START; vec = IAVF_RX_VEC_START; for (i = 0; i < dev->data->nb_rx_queues; i++) { qv_map[i].queue_id = i; qv_map[i].vector_id = vec; intr_handle->intr_vec[i] = vec++; -if (vec >= vf->nb_msix) +if (vec >= vf->vf_res->max_vectors) vec = IAVF_RX_VEC_START; } vf->qv_map = qv_map; -- 2.27.0