-----Original Message-----
From: Wang, Haiyue <redacted>
Sent: Tuesday, February 23, 2021 7:46 PM
To: Stephen Hemminger <stephen@networkplumber.org>; Xueming(Steven) Li <redacted>
Cc: dev@dpdk.org; Slava Ovsiienko <redacted>; Asaf Penso <redacted>
Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor

quoted

-----Original Message-----
From: dev <redacted> On Behalf Of Stephen Hemminger
Sent: Tuesday, February 23, 2021 09:55
To: Xueming Li <redacted>
Cc: dev@dpdk.org; Viacheslav Ovsiienko <redacted>; Asaf
Penso [off-list ref]
Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction
representor

On Sun, 14 Feb 2021 03:21:30 +0000
Xueming Li [off-list ref] wrote:

quoted

SubFunction [1] is a portion of the PCI device, a SF netdev has its
own dedicated queues(txq, rxq). A SF netdev supports E-Switch
representation offload similar to existing PF and VF representors. A
SF shares PCI level resources with other SFs and/or with its parent PCI function.

From SmartNIC perspective, when PCI device is shared for multi-host,
representors for host controller and host PF is required.

This patch set introduces new representor types in addtion to
existing VF representor. Syntax:

[[c#]pf#]vf#: VF port representor/s from controller/pf
[[c#]pf#]sf#: SF port representor/s from controller/pf
#: VF representor - for backwards compatibility

"#" is number instance, list or range, valid examples:
  1, [1,3,5], [0-3], [0,2-4,6]

For backward compatibility, this patch also introduces new netdev
capability to indicate the capability of supportting SF representor.

Version history:
 RFC:
 	initial version [2]
 V2:
    - separate patch for represnetor infrastructure, controller, pf and
      sf.
    - replace representor ID macro with functions:
      rte_eth_representor_id_encode()
      rte_eth_representor_id_parse()
    - new patch to allow devargs with same PCI BDF but different
      representors.
    - other minor code updates according to comments, thanks Andrew!
    - update document
 V3:
    - improve probing of allowed devargs with same name.
    - parse single word of kvargs as key.
    - update kvargs test cases.
 V4:
    - split first representor refactor patch into
      1: add representor type
      2: refector representor list parsing
    - push the patch supporting multi-devargs for same device.
 V5:
    - add comments for parsing functions
    - update switch_representation.rst - Thanks Ajit
 V6:
    - split representor types into different patches, move to
      rte_ethdev.h
    - improvements of rte_eth_devargs_process_list() according to
      Andrew's suggestion
    - fixed PF probe failure for Intel i40e
    - replace ethdev SF capability with rte_eth_representor_info_get()
    - add new ethdev ops api to get representor info from PMD
    - replace representor ID encode/decode with conversion from
      representor info
    - change ethdev representor iterator to use new ID encoding


Xueming Li (9):
  ethdev: introduce representor type
  ethdev: support representor port list
  ethdev: support new VF representor syntax
  ethdev: support sub function representor
  ethdev: support PF index in representor
  ethdev: support multi-host in representor
  ethdev: new API to get representor info
  ethdev: representor iterator compare complete info
  kvargs: update parser to support lists

 app/test/test_kvargs.c                        |  46 ++++-
 config/rte_config.h                           |   1 +
 doc/guides/prog_guide/poll_mode_drv.rst       |  13 +-
 .../prog_guide/switch_representation.rst      |  35 +++-
 drivers/net/bnxt/bnxt_ethdev.c                |   7 +
 drivers/net/enic/enic_ethdev.c                |   6 +
 drivers/net/i40e/i40e_ethdev.c                |   7 +
 drivers/net/ixgbe/ixgbe_ethdev.c              |   7 +
 drivers/net/mlx5/linux/mlx5_os.c              |  11 ++
 lib/librte_ethdev/ethdev_driver.h             |  49 ++++-
 lib/librte_ethdev/ethdev_private.c            | 173 ++++++++++++------
 lib/librte_ethdev/ethdev_private.h            |   3 -
 lib/librte_ethdev/rte_class_eth.c             |  40 ++--
 lib/librte_ethdev/rte_ethdev.c                | 102 ++++++++++-
 lib/librte_ethdev/rte_ethdev.h                |  53 ++++++
 lib/librte_ethdev/version.map                 |   4 +
 lib/librte_kvargs/rte_kvargs.c                | 101 +++++++---
 17 files changed, 535 insertions(+), 123 deletions(-)

A couple of higher level design questions:
1. How does Linux and other OS handle this in their API?
2. This solution seems quite tied into a specific implementation on your hardware.
   Is there a PCI standard for this?

quoted

3. Maybe a more general solution where these were represented as buses would
   allow for other connection methods in the future?

It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c

mlx5 subfunction support
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=24a790da0ac4d9bcce2a9becc8799241716672f6

Hi,

The Auxiliary driver for DPDK posted, welcome to have a look :)
http://patchwork.dpdk.org/project/dpdk/patch/20210311130113.31382-1-xuemingl@nvidia.com/

Thanks