-----Original Message-----
From: Leon Romanovsky <leon@kernel.org>
Sent: Wednesday, March 12, 2025 4:11 PM
To: Nikolay Aleksandrov <redacted>
Cc: netdev@vger.kernel.org; shrijeet@enfabrica.net;
alex.badea@keysight.com; eric.davis@broadcom.com; rip.sohan@amd.com;
dsahern@kernel.org; Bernard Metzler [off-list ref];
roland@enfabrica.net; winston.liu@keysight.com;
dan.mihailescu@keysight.com; Kamal Heib [off-list ref];
parth.v.parikh@keysight.com; Dave Miller [off-list ref];
ian.ziemba@hpe.com; andrew.tauferner@cornelisnetworks.com; welch@hpe.com;
rakhahari.bhunia@keysight.com; kingshuk.mandal@keysight.com; linux-
rdma@vger.kernel.org; kuba@kernel.org; Paolo Abeni [off-list ref];
Jason Gunthorpe [off-list ref]
Subject: [EXTERNAL] Re: [RFC PATCH 00/13] Ultra Ethernet driver
introduction

On Wed, Mar 12, 2025 at 04:20:08PM +0200, Nikolay Aleksandrov wrote:

quoted

On 3/12/25 1:29 PM, Leon Romanovsky wrote:

quoted

On Wed, Mar 12, 2025 at 11:40:05AM +0200, Nikolay Aleksandrov wrote:

quoted

On 3/8/25 8:46 PM, Leon Romanovsky wrote:

quoted

On Fri, Mar 07, 2025 at 01:01:50AM +0200, Nikolay Aleksandrov wrote:

[snip]

quoted

quoted

Also we have the ephemeral PDC connections>> that come and go as

needed. There more such objects coming with more

quoted

quoted

state, configuration and lifecycle management. That is why we added a
separate netlink family to cleanly manage them without trying to fit
a square peg in a round hole so to speak.

Yeah, I saw that you are planning to use netlink to manage objects,
which is very questionable. It is slow, unreliable, requires sockets,
needs more parsing logic e.t.c

To avoid all this overhead, RDMA uses netlink-like ioctl calls, which
fits better for object configurations.

Thanks

We'd definitely like to keep using netlink for control path object
management. Also please note we're talking about genetlink family. It is
fast and reliable enough for us, very easily extensible,
has a nice precise object definition with policies to enforce various
limitations, has extensive tooling (e.g. ynl), communication can be
monitored in realtime for debugging (e.g. nlmon), has a nice human
readable error reporting, gives the ability to easily dump large object
groups with filters applied, YAML family definitions and so on.
Having sockets or parsing are not issues.

Of course it is issue as netlink relies on Netlink sockets, which means
that you constantly move your configuration data instead of doing
standard to whole linux kernel pattern of allocating configuration
structs in user-space and just providing pointer to that through ioctl
call.

However, this discussion is premature and as an intro it is worth to
read this cover letter for how object management is done in RDMA
subsystem.

https://lore.kernel.org/linux% 
2Drdma_1501765627-2D104860-2D1-2Dgit-2Dsend-2Demail-2Dmatanb-
40mellanox.com_&d=DwIBAg&c=BSDicqBQBDjDI9RkVyTcHQ&r=4ynb4Sj_4MUcZXbhvovE4tY
SbqxyOwdSiLedP4yO55g&m=U78K-khiLd-
LLkbuNRzBStNppsXFTXdM7br052fwal1mzxpaOcOSQXCnguAK8t3g&s=U9dQl07fp-
e9380xjR94fW-UGixoMsoxr5HfXKYggLk&e=

Nice old stuff. Often history teaches us something. 😉

I assume the correct way forward is to first clarify the
structure of all user-visible objects that need to be
created/controlled/destroyed, and to route them through
this interface. Some will require extensions to given objects,
some may be new, some will be as-is. rdma_netlink will probably
be the right interface to look at for job control.

Best,
Bernard.

Thanks

quoted

Cheers,
 Nik