RE: [RFC PATCH v12 17/17] dlb: add basic sysfs interfaces

-----Original Message-----
From: Stephen Hemminger <stephen@networkplumber.org>
Sent: Tuesday, December 21, 2021 6:35 PM
To: Chen, Mike Ximing <redacted>
Cc: linux-kernel@vger.kernel.org; arnd@arndb.de; gregkh@linuxfoundation.org; Williams, Dan J
[off-list ref]; pierre-louis.bossart@linux.intel.com; netdev@vger.kernel.org;
davem@davemloft.net; kuba@kernel.org
Subject: Re: [RFC PATCH v12 17/17] dlb: add basic sysfs interfaces

On Tue, 21 Dec 2021 00:50:47 -0600
Mike Ximing Chen [off-list ref] wrote:

quoted

The dlb sysfs interfaces include files for reading the total and
available device resources, and reading the device ID and version. The
interfaces are used for device level configurations and resource
inquiries.

Signed-off-by: Mike Ximing Chen <redacted>
---
 Documentation/ABI/testing/sysfs-driver-dlb | 116 ++++++++++++
 drivers/misc/dlb/dlb_args.h                |  34 ++++
 drivers/misc/dlb/dlb_main.c                |   5 +
 drivers/misc/dlb/dlb_main.h                |   3 +
 drivers/misc/dlb/dlb_pf_ops.c              | 195 +++++++++++++++++++++
 drivers/misc/dlb/dlb_resource.c            |  50 ++++++
 6 files changed, 403 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-dlb

diff --git a/Documentation/ABI/testing/sysfs-driver-dlb

b/Documentation/ABI/testing/sysfs-driver-dlb
new file mode 100644
index 000000000000..bf09ef6f8a3a

--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-dlb

@@ -0,0 +1,116 @@
+What:		/sys/bus/pci/devices/.../total_resources/num_atomic_inflights
+What:		/sys/bus/pci/devices/.../total_resources/num_dir_credits
+What:		/sys/bus/pci/devices/.../total_resources/num_dir_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_hist_list_entries
+What:		/sys/bus/pci/devices/.../total_resources/num_ldb_credits
+What:		/sys/bus/pci/devices/.../total_resources/num_ldb_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_cos0_ldb_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_cos1_ldb_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_cos2_ldb_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_cos3_ldb_ports
+What:		/sys/bus/pci/devices/.../total_resources/num_ldb_queues
+What:		/sys/bus/pci/devices/.../total_resources/num_sched_domains
+Date:		Oct 15, 2021
+KernelVersion:	5.15
+Contact:	mike.ximing.chen@intel.com
+Description:
+		The total_resources subdirectory contains read-only files that
+		indicate the total number of resources in the device.
+
+		num_atomic_inflights:  Total number of atomic inflights in the
+				       device. Atomic inflights refers to the
+				       on-device storage used by the atomic
+				       scheduler.
+
+		num_dir_credits:       Total number of directed credits in the
+				       device.
+
+		num_dir_ports:	       Total number of directed ports (and
+				       queues) in the device.
+
+		num_hist_list_entries: Total number of history list entries in
+				       the device.
+
+		num_ldb_credits:       Total number of load-balanced credits in
+				       the device.
+
+		num_ldb_ports:	       Total number of load-balanced ports in
+				       the device.
+
+		num_cos<M>_ldb_ports:  Total number of load-balanced ports
+				       belonging to class-of-service M in the
+				       device.
+
+		num_ldb_queues:	       Total number of load-balanced queues in
+				       the device.
+
+		num_sched_domains:     Total number of scheduling domains in the
+				       device.
+

Sysfs is only slightly better than /proc as an API.
If it is just for testing than debugfs might be better.

Sysfs in our driver is not only for testing. It is used for the system configuration
at run time.

Could this be done with a real netlink interface?
Maybe as part of devlink?

Thanks for the suggestion. I will look into some sample implementations of
devlink/netlink. Our current plan is to stay with the configfs interface, and
find ways to resolve issues related to atomic update and resource reset at
tear-down time.

Mike