RE: [PATCH v2 05/23] ice: Add devlink params support

Subject: RE: [PATCH v2 05/23] ice: Add devlink params support

Hi Shiraz,

quoted

From: Shiraz Saleem <redacted>
Sent: Wednesday, March 24, 2021 5:30 AM

Add two new runtime RDMA related devlink parameters to ice driver.
'rdma_resource_limits_sel' is driver-specific while 'rdma_protocol' is generic.
Configuration changes result in unplugging the auxiliary RDMA device
and re- plugging it with updated values for irdma auxiiary driver to
consume at
drv.probe()

Signed-off-by: Shiraz Saleem <redacted>
---
 .../networking/devlink/devlink-params.rst          |   6 +
 Documentation/networking/devlink/ice.rst           |  35 +++++
 drivers/net/ethernet/intel/ice/ice_devlink.c       | 146
++++++++++++++++++++-
 drivers/net/ethernet/intel/ice/ice_devlink.h       |   6 +
 drivers/net/ethernet/intel/ice/ice_main.c          |   2 +
 include/net/devlink.h                              |   4 +
 net/core/devlink.c                                 |   5 +
 7 files changed, 202 insertions(+), 2 deletions(-)

[..]

quoted

+.. list-table:: Driver-specific parameters implemented
+   :widths: 5 5 5 85
+
+   * - Name
+     - Type
+     - Mode
+     - Description
+   * - ``rdma_resource_limits_sel``
+     - string
+     - runtime
+     - Selector to limit the RDMA resources configured for the
+ device. The
range
+       is between 0 and 7 with a default value equal to 3. Each
+ selector
supports
+       up to the value specified in the table.
+          - 0: 128 QPs
+          - 1: 1K QPs
+          - 2: 2K QPs
+          - 3: 4K QPs
+          - 4: 16K QPs
+          - 5: 64K QPs
+          - 6: 128K QPs
+          - 7: 256K QPs

Resources are better represented as devlink resource.
Such as,

$ devlink resource set pci/0000:06:00.0 /rdma/max_qps 16384 $ devlink resource
set pci/0000:06:00.0 /rdma/max_cqs 8192 $ devlink resource set pci/0000:06:00.0
/rdma/max_mrs 16384

Hi Parav - Thank you for the feedback.

Maybe I am missing something but I see that a devlink hot reload is required to enforce the update?
There isn't really a de-init required of PCI driver entities in this case for this rdma param.
But only an unplug, plug of the auxdev with new value. Intuitively it feels more runtime-ish.

There is also a device powerof2 requirement on the maxqp which I don't see enforceable as it stands.

This is not super-critical for the initial submission but a nice to have. But I do want to brainstorm options.. 

Shiraz