RE: [EXTERNAL] Re: [net-next PATCH v10 00/11] Introduce RVU representors

-----Original Message-----
From: Jiri Pirko <jiri@resnulli.us>
Sent: Friday, August 16, 2024 7:47 PM
To: Geethasowjanya Akula <gakula@marvell.com>
Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org; kuba@kernel.org;
davem@davemloft.net; pabeni@redhat.com; edumazet@google.com; Sunil
Kovvuri Goutham [off-list ref]; Subbaraya Sundeep Bhatta
[off-list ref]; Hariprasad Kelam [off-list ref]
Subject: Re: [EXTERNAL] Re: [net-next PATCH v10 00/11] Introduce RVU
representors

Fri, Aug 16, 2024 at 03: 36: 25PM CEST, gakula@ marvell. com wrote: > > >>-----
Original Message----- >>From: Jiri Pirko <jiri@ resnulli. us> >>Sent: Thursday,
August 8, 2024 9: 17 PM >>To: Geethasowjanya Akula <gakula@ marvell. com>

quoted

quoted

-----Original Message-----
From: Jiri Pirko <jiri@resnulli.us>
Sent: Thursday, August 8, 2024 9:17 PM
To: Geethasowjanya Akula <gakula@marvell.com>
Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
kuba@kernel.org; davem@davemloft.net; pabeni@redhat.com;
edumazet@google.com; Sunil Kovvuri Goutham [off-list ref];
Subbaraya Sundeep Bhatta [off-list ref]; Hariprasad Kelam
[off-list ref]
Subject: [EXTERNAL] Re: [net-next PATCH v10 00/11] Introduce RVU
representors

Mon, Aug 05, 2024 at 03: 18: 04PM CEST, gakula@ marvell. com wrote:

quoted

This series adds representor support for each rvu devices. >When

switchdev mode is enabled, representor netdev is registered >for each
rvu device. In implementation of Mon, Aug 05, 2024 at 03:18:04PM CEST,
gakula@marvell.com wrote:

quoted

This series adds representor support for each rvu devices.
When switchdev mode is enabled, representor netdev is registered for
each rvu device. In implementation of representor model, one NIX HW
LF with multiple SQ and RQ is reserved, where each RQ and SQ of the
LF are mapped to a representor. A loopback channel is reserved to
support packet path between representors and VFs.
CN10K silicon supports 2 types of MACs, RPM and SDP. This patch set
adds representor support for both RPM and SDP MAC interfaces.

- Patch 1: Refactors and exports the shared service functions.
- Patch 2: Implements basic representor driver.
- Patch 3: Add devlink support to create representor netdevs that
 can be used to manage VFs.
- Patch 4: Implements basec netdev_ndo_ops.
- Patch 5: Installs tcam rules to route packets between representor and
   VFs.
- Patch 6: Enables fetching VF stats via representor interface
- Patch 7: Adds support to sync link state between representors and VFs .
- Patch 8: Enables configuring VF MTU via representor netdevs.
- Patch 9: Add representors for sdp MAC.
- Patch 10: Add devlink port support.


Command to create PF/VF representor
#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev VF
representors are created for each VF when switch mode is set
switchdev on representor PCI device

#devlink dev
pci/0002:01:00.0
pci/0002:02:00.0
pci/0002:1c:00.0

What are these 3 instances representing? How many PFs do you have? 3?
How many physical ports you have?

The test setup has 3 PFs one for each physical port.

Below example is for the device pci/0002:1c:00.0.

3 port nic, that sounds odd. Is this something shipped already? Care to paste a
link?

https://www.marvell.com/content/dam/marvell/en/public-collateral/embedded-processors/marvell-infrastructure-processors-octeon-tx2-cn913x-product-brief.pdf
Please refer to page2,  Key features -> Ethernet Ports.
Based on the silicon version number of physical ports varies. 

quoted

quoted

quoted

#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev

# ip link show
eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode

quoted

DEFAULT group default qlen 1000 link/ether 7e:58:2d:b6:97:51 brd
ff:ff:ff:ff:ff:ff

What is this eth0? Why isn't it connected to any devlink port?

quoted

r0p1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode DEFAULT group default qlen 1000 link/ether 7e:5a:66:ea:fe:d6 brd
ff:ff:ff:ff:ff:ff

quoted

r1p1v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode DEFAULT group default qlen 1000 link/ether de:29:be:10:9e:bf brd
ff:ff:ff:ff:ff:ff

quoted

r2p1v2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode DEFAULT group default qlen 1000 link/ether 4a:12:c7:a2:66:ad brd
ff:ff:ff:ff:ff:ff

quoted

r3p1v3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode

quoted

DEFAULT group default qlen 1000 link/ether c2:b8:a8:0e:73:fd brd
ff:ff:ff:ff:ff:ff


~# devlink port
pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller 0
pfnum 1 vfnum 0 external false splittable false
pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf controller 0
pfnum 1 vfnum 1 external false splittable false
pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf controller 0
pfnum 1 vfnum 2 external false splittable false
pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf controller 0
pfnum 1 vfnum 3 external false splittable false

You are missing physical port devlink instance here? Where is it?

pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller 0
pfnum 1 vfnum 0 external false splittable false This is for the PF.

That is not a physical port. The flavour should be "physical" for them.

I was using PCI_PF flavour. Will fix it in the next version.

quoted

Below is the example on a setup with one PF before  3VFs are created.

#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev

# ip link show
eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

mode

quoted

DEFAULT group default qlen 1000 link/ether 7e:58:2d:b6:97:51 brd
ff:ff:ff:ff:ff:ff
r0p1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT group default qlen 1000 link/ether 7e:5a:66:ea:fe:d6 brd
ff:ff:ff:ff:ff:ff
r1p1v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT group default qlen 1000 link/ether de:29:be:10:9e:bf brd
ff:ff:ff:ff:ff:ff
r2p1v2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
DEFAULT group default qlen 1000 link/ether 4a:12:c7:a2:66:ad brd
ff:ff:ff:ff:ff:ff
r3p1v3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode

How you setup the netdevice names? Do you have custom udev rule? Would
be great to give here the out-of-box udev names instead.

Ok will fix it in the next version.

quoted

DEFAULT group default qlen 1000 link/ether c2:b8:a8:0e:73:fd brd
ff:ff:ff:ff:ff:ff

Above shows the PF physical port and 4 representors(1 for PF and 3 for VFs).
# devlink port
pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller  0
pfnum 1 vfnum 0 external false splittable false
pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf controller 0
pfnum 1 vfnum 1 external false splittable false
pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf controller 0
pfnum 1 vfnum 2 external false splittable false
pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf controller 0
pfnum 1 vfnum 3 external false splittable false

quoted

quoted

-----------
v1-v2:
-Fixed build warnings.
-Address review comments provided by "Kalesh Anakkur Purayil".

v2-v3:
- Used extack for error messages.
- As suggested reworked commit messages.
- Fixed sparse warning.

v3-v4:
- Patch 2 & 3: Fixed coccinelle reported warnings.
- Patch 10: Added devlink port support.

When someone reviews your patchset at some version, you put him to cc
list from that point. Why didn't you put me to cc list?

quoted

v4-v5:
 - Patch 3: Removed devm_* usage in rvu_rep_create()
 - Patch 3: Fixed build warnings.

v5-v6:
 - Addressed review comments provided by "Simon Horman".
 - Added review tag.

v6-v7:
 - Rebased on top net-next branch.

v7-v8:
  - Implmented offload stats ndo.
  - Added documentation.

v8-v9:
  - Updated the documentation.

v9-v10:
 - Fixed build warning w.r.t documentation.

Geetha sowjanya (11):
 octeontx2-pf: Refactoring RVU driver
 octeontx2-pf: RVU representor driver
 octeontx2-pf: Create representor netdev
 octeontx2-pf: Add basic net_device_ops
 octeontx2-af: Add packet path between representor and VF
 octeontx2-pf: Get VF stats via representor
 octeontx2-pf: Add support to sync link state between representor and
   VFs
 octeontx2-pf: Configure VF mtu via representor
 octeontx2-pf: Add representors for sdp MAC
 octeontx2-pf: Add devlink port support
 octeontx2-pf: Implement offload stats ndo for representors

.../ethernet/marvell/octeontx2.rst            |  85 ++
.../net/ethernet/marvell/octeontx2/Kconfig    |   8 +
.../ethernet/marvell/octeontx2/af/Makefile    |   3 +-
.../ethernet/marvell/octeontx2/af/common.h    |   2 +
.../net/ethernet/marvell/octeontx2/af/mbox.h  |  74 ++
.../net/ethernet/marvell/octeontx2/af/npc.h   |   1 +
.../net/ethernet/marvell/octeontx2/af/rvu.c   |  11 +
.../net/ethernet/marvell/octeontx2/af/rvu.h   |  30 +-
.../marvell/octeontx2/af/rvu_debugfs.c        |  27 -
.../marvell/octeontx2/af/rvu_devlink.c        |   6 +
.../ethernet/marvell/octeontx2/af/rvu_nix.c   |  81 +-
.../marvell/octeontx2/af/rvu_npc_fs.c         |   5 +
.../ethernet/marvell/octeontx2/af/rvu_reg.h   |   4 +
.../ethernet/marvell/octeontx2/af/rvu_rep.c   | 464 +++++++++++
.../marvell/octeontx2/af/rvu_struct.h         |  26 +
.../marvell/octeontx2/af/rvu_switch.c         |  20 +-
.../ethernet/marvell/octeontx2/nic/Makefile   |   2 +
.../ethernet/marvell/octeontx2/nic/cn10k.c    |   4 +-
.../ethernet/marvell/octeontx2/nic/cn10k.h    |   2 +-
.../marvell/octeontx2/nic/otx2_common.c       |  58 +-
.../marvell/octeontx2/nic/otx2_common.h       |  84 +-
.../marvell/octeontx2/nic/otx2_devlink.c      |  49 ++
.../ethernet/marvell/octeontx2/nic/otx2_pf.c  | 305 +++++---
.../marvell/octeontx2/nic/otx2_txrx.c         |  38 +-
.../marvell/octeontx2/nic/otx2_txrx.h         |   3 +-
.../ethernet/marvell/octeontx2/nic/otx2_vf.c  |  19 +-
.../net/ethernet/marvell/octeontx2/nic/rep.c  | 725
++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/rep.h  |
53 ++
28 files changed, 1962 insertions(+), 227 deletions(-) create mode
100644 drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c
create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/rep.c
create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/rep.h

--
2.25.1