quoted

Do you have multiple "adapters" in one ib/eth device?
If yes, at least for IB, it will be very unexpected to see.

Adapter is IB specific and one per VF/pcie device. It's the handle that is passed
between the management and VM for book keeping.

I think there are some confusions on the device.

There is one device, that supports both Ethernet and RDMA(through RoCE).

On the device, RoCE can't function without Ethernet. In the Linux, the RDMA device is modeled as auxiliary device to ethernet device.

The physical device exposes both Ethernet and RDMA management interfaces (as adapters) to the client (Linux kernel). When the MANA RDMA driver was first introduced, RAW QP was supported. There was no need to connect to the RDMA management interface. Many device hardware RDMA capabilities were hardcoded in the driver at the time. (There were some reviewers questioning this at the time).

With this patchset, we are connecting to the RDMA management interface on the device. This is for addressing the prior comments, and for supporting upcoming RC QP.

Thanks,

Long

quoted

Why do you have client and device when they are basically the same

objects?

quoted

I am not sure which ones you are referring to specifically , can you please
elaborate?

quoted

quoted

Please note that these are terms used for different purposes on the

management side.

We are discussing RDMA side of this series.

Thanks

quoted

quoted

Thanks

quoted

Thanks

quoted

-----Original Message-----
From: Leon Romanovsky <leon@kernel.org>
Sent: Monday, September 11, 2023 7:33 AM
To: sharmaajay@linuxonhyperv.com
Cc: Long Li <longli@microsoft.com>; Jason Gunthorpe
[off-list ref]; Dexuan Cui [off-list ref]; Wei Liu
[off-list ref]; David S. Miller [off-list ref];
Eric Dumazet [off-list ref]; Jakub Kicinski
[off-list ref]; Paolo Abeni [off-list ref]; linux-
rdma@vger.kernel.org; linux-hyperv@vger.kernel.org;
netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Ajay
Sharma [off-list ref]
Subject: [EXTERNAL] Re: [Patch v5 0/5] RDMA/mana_ib

On Thu, Sep 07, 2023 at 09:52:34AM -0700,
sharmaajay@linuxonhyperv.com
wrote:

quoted

From: Ajay Sharma <redacted>

Change from v4:
Send qp fatal error event to the context that created the qp.
Add lookup table for qp.

Ajay Sharma (5):
  RDMA/mana_ib : Rename all mana_ib_dev type variables to

mib_dev

quoted

quoted

quoted

quoted

quoted

  RDMA/mana_ib : Register Mana IB  device with Management SW
  RDMA/mana_ib : Create adapter and Add error eq
  RDMA/mana_ib : Query adapter capabilities
  RDMA/mana_ib : Send event to qp

I didn't look very deep into the series and has three very
initial

comments.

quoted

quoted

quoted

quoted

1. Please do git log drivers/infiniband/hw/mana/ and use same
format for commit messages.
2. Don't invent your own index-to-qp query mechanism in last
patch and use xarray.
3. Once you decided to export mana_gd_register_device, it
hinted me that it is time to move to auxbus infrastructure.

Thanks

quoted

 drivers/infiniband/hw/mana/cq.c               |  12 +-
 drivers/infiniband/hw/mana/device.c           |  81 +++--
 drivers/infiniband/hw/mana/main.c             | 288 +++++++++++++-

----

quoted

quoted

quoted

quoted

quoted

quoted

 drivers/infiniband/hw/mana/mana_ib.h          | 102 ++++++-
 drivers/infiniband/hw/mana/mr.c               |  42 ++-
 drivers/infiniband/hw/mana/qp.c               |  86 +++---
 drivers/infiniband/hw/mana/wq.c               |  21 +-
 .../net/ethernet/microsoft/mana/gdma_main.c   | 152 +++++----
 drivers/net/ethernet/microsoft/mana/mana_en.c |   3 +
 include/net/mana/gdma.h                       |  16 +-
 10 files changed, 545 insertions(+), 258 deletions(-)

--
2.25.1