Re: [dpdk-dev] [PATCH v2 01/22] ethdev: fix MTU size exceeds max rx packet length

On 1/6/2021 3:36 AM, Yang, SteveX wrote:

Hi Lijun,

Thanks for your review. Please check my comments inline.
Regards,
Steve Yang.

quoted

-----Original Message-----
From: oulijun <redacted>
Sent: Wednesday, December 30, 2020 6:20 PM
To: Yang, SteveX <redacted>; dev@dpdk.org
Cc: Lu, Wenzhuo <redacted>; Xing, Beilei
[off-list ref]; Iremonger, Bernard
[off-list ref]; asomalap@amd.com;
rahul.lakkireddy@chelsio.com; hemant.agrawal@nxp.com;
sachin.saxena@oss.nxp.com; Guo, Jia [off-list ref]; Wang, Haiyue
[off-list ref]; g.singh@nxp.com; xuanziyang2@huawei.com;
cloud.wangxiaoyun@huawei.com; zhouguoyang@huawei.com;
xavier.huwei@huawei.com; humin29@huawei.com;
yisen.zhuang@huawei.com; Wu, Jingjing [off-list ref]; Yang,
Qiming [off-list ref]; Zhang, Qi Z [off-list ref]; Xu,
Rosen [off-list ref]; sthotton@marvell.com;
srinivasan@marvell.com; heinrich.kuhn@netronome.com;
hkalra@marvell.com; jerinj@marvell.com; ndabilpuram@marvell.com;
kirankumark@marvell.com; rmody@marvell.com; shshaikh@marvell.com;
andrew.rybchenko@oktetlabs.ru; mczekaj@marvell.com;
thomas@monjalon.net; Yigit, Ferruh [off-list ref];
ivan.boule@6wind.com; Ananyev, Konstantin
[off-list ref]; samuel.gauthier@6wind.com;
david.marchand@6wind.com; shahafs@mellanox.com;
stephen@networkplumber.org; maxime.coquelin@redhat.com;
olivier.matz@6wind.com; lihuisong@huawei.com; shreyansh.jain@nxp.com;
wei.dai@intel.com; fengchunsong@huawei.com; chenhao164@huawei.com;
tangchengchang@hisilicon.com; Zhang, Helin [off-list ref];
yanglong.wu@intel.com; xiaolong.ye@intel.com; Xu, Ting
[off-list ref]; Li, Xiaoyun [off-list ref]; Wei, Dan
[off-list ref]; Pei, Andy [off-list ref];
vattunuru@marvell.com; skori@marvell.com; sony.chacko@qlogic.com;
Richardson, Bruce [off-list ref]; ivan.malov@oktetlabs.ru;
rad@semihalf.com; slawomir.rosek@semihalf.com;
kamil.rytarowski@caviumnetworks.com; Zhao1, Wei [off-list ref];
Jiang, JunyuX [off-list ref]; kumaras@chelsio.com;
girish.nandibasappa@amd.com; rolf.neugebauer@netronome.com;
alejandro.lucero@netronome.com
Subject: Re: [PATCH v2 01/22] ethdev: fix MTU size exceeds max rx packet
length



在 2020/12/17 17:22, Steve Yang 写道:

quoted

If max rx packet length is smaller then MTU + Ether overhead, that
will drop all MTU size packets.

Update the MTU size according to the max rx packet and Ether overhead.

Fixes: 59d0ecdbf0e1 ("ethdev: MTU accessors")

Signed-off-by: Steve Yang <redacted>
---
   lib/librte_ethdev/rte_ethdev.c | 21 ++++++++++++++++++---
   1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/lib/librte_ethdev/rte_ethdev.c

b/lib/librte_ethdev/rte_ethdev.c index 17ddacc78d..ff6a1e675f 100644

--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c

@@ -1292,6 +1292,7 @@ rte_eth_dev_configure(uint16_t port_id,

uint16_t nb_rx_q, uint16_t nb_tx_q,

quoted

   struct rte_eth_dev *dev;
   struct rte_eth_dev_info dev_info;
   struct rte_eth_conf orig_conf;
+uint16_t overhead_len;
   int diag;
   int ret;

@@ -1323,6 +1324,15 @@ rte_eth_dev_configure(uint16_t port_id,

uint16_t nb_rx_q, uint16_t nb_tx_q,

quoted

   if (ret != 0)
   goto rollback;

+/* Get the real Ethernet overhead length */
+if (dev_info.max_mtu &&
+    dev_info.max_mtu != UINT16_MAX &&
+    dev_info.max_rx_pktlen &&
+    dev_info.max_rx_pktlen > dev_info.max_mtu)
+overhead_len = dev_info.max_rx_pktlen -

dev_info.max_mtu;

quoted

+else
+overhead_len = RTE_ETHER_HDR_LEN +

RTE_ETHER_CRC_LEN;

quoted

+
   /* If number of queues specified by application for both Rx and Tx is
    * zero, use driver preferred values. This cannot be done individually
    * as it is valid for either Tx or Rx (but not both) to be zero.

@@ -1410,13 +1420,18 @@ rte_eth_dev_configure(uint16_t port_id,

uint16_t nb_rx_q, uint16_t nb_tx_q,

quoted

   goto rollback;
   }
   } else {
-if (dev_conf->rxmode.max_rx_pkt_len <

RTE_ETHER_MIN_LEN ||

quoted

-dev_conf->rxmode.max_rx_pkt_len >

RTE_ETHER_MAX_LEN)

quoted

+uint16_t pktlen = dev_conf->rxmode.max_rx_pkt_len;
+if (pktlen < RTE_ETHER_MIN_MTU + overhead_len ||
+pktlen > RTE_ETHER_MTU + overhead_len)
   /* Use default value */
   dev->data->dev_conf.rxmode.max_rx_pkt_len =
-

RTE_ETHER_MAX_LEN;

quoted

+RTE_ETHER_MTU +

overhead_len;

quoted

   }

+/* Scale the MTU size to adapt max_rx_pkt_len */
+dev->data->mtu = dev->data->dev_conf.rxmode.max_rx_pkt_len -
+overhead_len;
+

Hi
    I think the dev->data->mtu should be updated after configured success. So
the update in this position seems unreasonable.

Good suggestion, I've checked some PMDs, and the corresponding assignments are existed within their 'dev_configure_ops'.
For example: [bnxt_ethdev.c # 1100]

if (rx_offloads & DEV_RX_OFFLOAD_JUMBO_FRAME) {
eth_dev->data->mtu =
eth_dev->data->dev_conf.rxmode.max_rx_pkt_len -
RTE_ETHER_HDR_LEN - RTE_ETHER_CRC_LEN - VLAN_TAG_SIZE *
BNXT_NUM_VLANS;
bnxt_mtu_set_op(eth_dev, eth_dev->data->mtu);
}

Hence, I will move this 'mtu' assignment to line after '(*dev->dev_ops->dev_configure)(dev)'.

There is a 'rollback' already for the similar reason.

What do you think to store the old MTU and restore it in the rollback if needed? 
So you don't need to change where MTU set.

Actually, it doesn't matter if it is the JUMBO frame, the 'mtu' must keep pace with 'max_rx_pkt_len' anytime.
E.g.: if 'mtu' is 1500, and 'max_rx_pkt_len' is set to 1510 by command line, that 'mtu' must be reduced to '1510 - 18 = 1492' in 'dev_configure' phase, even though it is not a Jumbo frame.

According to the API definition:
  max_rx_pkt_len;  /**< Only used if JUMBO_FRAME enabled. */

The concern was removing this check from the ethdev may break some PMDs that 
does not follow the API and use the 'max_rx_pkt_len' even if JUMBO frame offload 
set.

For this release, we can afford to break the PMDs implementing wrong and fix 
them after testing.

quoted

'max_rx_pkt_len' is only used when jumbo_frame enabled, as follows:
struct rte_eth_rxmode {
.....
uint32_t max_rx_pkt_len;  /**< Only used if JUMBO_FRAME
enabled. */
/** Maximum allowed size of LRO aggregated packet. */
....};

So If DEV_RX_OFFLOAD_JUMBO_FRAME is set to rxmode.offload, driver
should configure mtu to hardware according to 'max_rx_pkt_len' and update
dev->data->mtu. This seems more reasonable. And some PMD drivers are
already doing this.

quoted

In addition, validity check for 'max_rx_pkt_len' in
rte_eth_dev_configure API may be error. It should be greater than
'RTE_ETHER_MTU + overhead_len'.
Because driver must enable DEV_RX_OFFLOAD_JUMBO_FRAME when user
set mtu
with greater than 1500 by 'rte_eth_dev_set_mtu' API.

I'm not sure if it is following validity check you mentioned.

quoted

quoted

+if (pktlen < RTE_ETHER_MIN_MTU + overhead_len ||
+pktlen > RTE_ETHER_MTU + overhead_len)

If yes, no issue here, because the 'rte_eth_dev_configure' is only used for initializing device,
and just gives out an initial 'max_rx_pkt_len' value by default. Once user tries to change mtu via 'set mtu',
the 'max_rx_pkt_len' will be updated to expected value in the 'mtu_set' ops.
Another aspect, that is the 'disable DEV_RX_OFFLOAD_JUMBO' branch, 'max_rx_pkt_len' must be limited to
effective max value for non-Jumbo frame.

In the 'disable DEV_RX_OFFLOAD_JUMBO' branch, 'max_rx_pkt_len' should be invalid 
according API doc as mentioned above, I guess Lijun refers to this.

Overall what about updating as following, in 'rte_eth_dev_configure()':

if (offloads & DEV_RX_OFFLOAD_JUMBO_FRAME) {
   // max_rx_pkt_len checks
   old_mtu = mtu
   mtu = max_rx_pkt_len - overhead_len
}

...

rollback:
  mtu = old_mtu;

quoted

What do you think?

Thanks
Lijun Ou

quoted

   /*
    * If LRO is enabled, check that the maximum aggregated packet
    * size is supported by the configured device.