[PATCH rdma-core 0/3] Add NDR support

STALE1718d

5 messages, 1 author, 2022-01-11 · open the first message on its own page

[PATCH rdma-core 0/3] Add NDR support

From: Yishai Hadas <yishaih@nvidia.com>
Date: 2021-12-29 08:55:42

This series extends verbs and diags to include the NDR support.

It includes the NDR support bit from PortInfo.CapabilityMask2 and the new rates
as were defined in IB Spec Release 1.5.

The matching utilities inside verbs to print and convert to/from the new rates
were updated as well.

In addition, the series includes some pyverbs extension to support the new NDR
definitions.

PR was sent:
https://github.com/linux-rdma/rdma-core/pull/1114

Yishai

Edward Srouji (1):
  pyverbs: Extend support of NDR rates

Maher Sanalla (2):
  verbs: Extend support of NDR rates
  ibdiags: Extend support of NDR rates

 infiniband-diags/ibportstate.c | 8 +++++++-
 libibmad/iba_types.h           | 3 +++
 libibverbs/examples/devinfo.c  | 1 +
 libibverbs/verbs.c             | 8 ++++++++
 libibverbs/verbs.h             | 3 +++
 pyverbs/device.pyx             | 3 ++-
 pyverbs/libibverbs_enums.pxd   | 1 +
 7 files changed, 25 insertions(+), 2 deletions(-)

-- 
1.8.3.1

[PATCH rdma-core 1/3] verbs: Extend support of NDR rates

From: Yishai Hadas <yishaih@nvidia.com>
Date: 2021-12-29 08:55:44

From: Maher Sanalla <redacted>

NDR(106.25 Gbps) support exposed new data rates:
800 Gbps - NDR 8x.
1200 Gbps - NDR 12x.

Utility methods were updated to support the new rates mentioned above:
1) Rate to mult - Convert the IB rate enum to a multiple of 2.5 Gbps.
2) Rate to mbps - Convert IB rate enum to the mbps value.

The NDR support bit from PortInfo.CapabilityMask2 was added to
libibverbs, as well as the new link speeds mentioned above.

In addition, speed_str() of ibv_devinfo was updated to consider the new
NDR rate.

Reference: IB Spec Release 1.5

Signed-off-by: Maher Sanalla <redacted>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 libibverbs/examples/devinfo.c | 1 +
 libibverbs/verbs.c            | 8 ++++++++
 libibverbs/verbs.h            | 3 +++
 3 files changed, 12 insertions(+)

diff --git a/libibverbs/examples/devinfo.c b/libibverbs/examples/devinfo.c
index 5db568b..cef6e2e 100644
--- a/libibverbs/examples/devinfo.c
+++ b/libibverbs/examples/devinfo.c

@@ -147,6 +147,7 @@ static const char *speed_str(uint8_t speed)
 	case 16: return "14.0 Gbps";
 	case 32: return "25.0 Gbps";
 	case 64: return "50.0 Gbps";
+	case 128: return "100.0 Gbps";
 	default: return "invalid speed";
 	}
 }

diff --git a/libibverbs/verbs.c b/libibverbs/verbs.c
index ee26b1d..69f82e5 100644
--- a/libibverbs/verbs.c
+++ b/libibverbs/verbs.c

@@ -72,6 +72,8 @@ int __attribute__((const)) ibv_rate_to_mult(enum ibv_rate rate)
 	case IBV_RATE_50_GBPS:  return 20;
 	case IBV_RATE_400_GBPS: return 160;
 	case IBV_RATE_600_GBPS: return 240;
+	case IBV_RATE_800_GBPS: return 320;
+	case IBV_RATE_1200_GBPS: return 480;
 	default:           return -1;
 	}
 }

@@ -92,6 +94,8 @@ enum ibv_rate __attribute__((const)) mult_to_ibv_rate(int mult)
 	case 20: return IBV_RATE_50_GBPS;
 	case 160: return IBV_RATE_400_GBPS;
 	case 240: return IBV_RATE_600_GBPS;
+	case 320: return IBV_RATE_800_GBPS;
+	case 480: return IBV_RATE_1200_GBPS;
 	default: return IBV_RATE_MAX;
 	}
 }

@@ -120,6 +124,8 @@ int  __attribute__((const)) ibv_rate_to_mbps(enum ibv_rate rate)
 	case IBV_RATE_50_GBPS:  return 53125;
 	case IBV_RATE_400_GBPS: return 425000;
 	case IBV_RATE_600_GBPS: return 637500;
+	case IBV_RATE_800_GBPS: return 850000;
+	case IBV_RATE_1200_GBPS: return 1275000;
 	default:               return -1;
 	}
 }

@@ -148,6 +154,8 @@ enum ibv_rate __attribute__((const)) mbps_to_ibv_rate(int mbps)
 	case 53125:  return IBV_RATE_50_GBPS;
 	case 425000: return IBV_RATE_400_GBPS;
 	case 637500: return IBV_RATE_600_GBPS;
+	case 850000: return IBV_RATE_800_GBPS;
+	case 1275000: return IBV_RATE_1200_GBPS;
 	default:     return IBV_RATE_MAX;
 	}
 }

diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 36b4142..6f7910b 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h

@@ -418,6 +418,7 @@ enum ibv_port_cap_flags2 {
 	IBV_PORT_SWITCH_PORT_STATE_TABLE_SUP	= 1 << 3,
 	IBV_PORT_LINK_WIDTH_2X_SUP		= 1 << 4,
 	IBV_PORT_LINK_SPEED_HDR_SUP		= 1 << 5,
+	IBV_PORT_LINK_SPEED_NDR_SUP		= 1 << 10,
 };
 
 struct ibv_port_attr {

@@ -721,6 +722,8 @@ enum ibv_rate {
 	IBV_RATE_50_GBPS  = 20,
 	IBV_RATE_400_GBPS = 21,
 	IBV_RATE_600_GBPS = 22,
+	IBV_RATE_800_GBPS = 23,
+	IBV_RATE_1200_GBPS = 24,
 };
 
 /**

-- 
1.8.3.1

[PATCH rdma-core 2/3] ibdiags: Extend support of NDR rates

From: Yishai Hadas <yishaih@nvidia.com>
Date: 2021-12-29 08:55:46

From: Maher Sanalla <redacted>

NDR(106.25 Gbps) support exposed new data rates:
800 Gbps - NDR 8x.
1200 Gbps - NDR 12x.

The NDR support bit from PortInfo.CapabilityMask2 was added to libibmad
as well as the new link speeds mentioned above.

validate_extended_speed() function in infiniband-diags was adjusted to
validate NDR speed.

Reference: IB Spec Release 1.5

Signed-off-by: Maher Sanalla <redacted>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 infiniband-diags/ibportstate.c | 8 +++++++-
 libibmad/iba_types.h           | 3 +++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/infiniband-diags/ibportstate.c b/infiniband-diags/ibportstate.c
index 7f3afb8..5319e68 100644
--- a/infiniband-diags/ibportstate.c
+++ b/infiniband-diags/ibportstate.c

@@ -353,7 +353,13 @@ static void validate_speed(int peerspeed, int lsa)
 
 static void validate_extended_speed(int peerespeed, int lsea)
 {
-	if ((espeed & peerespeed & 0x4)) {
+
+	if ((espeed & peerespeed & 0x8)) {
+		if (lsea != 8)
+			IBWARN
+			    ("Peer ports operating at active extended speed %d rather than 8 (106.25 Gbps)",
+			     lsea);
+	} else if ((espeed & peerespeed & 0x4)) {
 		if (lsea != 4)
 			IBWARN
 			    ("Peer ports operating at active extended speed %d rather than 4 (53.125 Gbps)",

diff --git a/libibmad/iba_types.h b/libibmad/iba_types.h
index 0805aa9..f0c15ef 100644
--- a/libibmad/iba_types.h
+++ b/libibmad/iba_types.h

@@ -749,6 +749,7 @@ typedef struct {
 #define IB_PORT_CAP2_IS_SWITCH_PORT_STATE_TBL_SUPP htobe16(0x0008)
 #define IB_PORT_CAP2_IS_LINK_WIDTH_2X_SUPPORTED htobe16(0x0010)
 #define IB_PORT_CAP2_IS_LINK_SPEED_HDR_SUPPORTED htobe16(0x0020)
+#define IB_PORT_CAP2_IS_LINK_SPEED_NDR_SUPPORTED htobe16(0x0400)
 typedef struct {
 	__be32 cap_mask;
 	__be16 fec_mode_active;

@@ -803,6 +804,8 @@ typedef struct {
 #define IB_PATH_RECORD_RATE_50_GBS 20
 #define IB_PATH_RECORD_RATE_400_GBS 21
 #define IB_PATH_RECORD_RATE_600_GBS 22
+#define IB_PATH_RECORD_RATE_800_GBS 23
+#define IB_PATH_RECORD_RATE_1200_GBS 24
 #define FDR10 0x01
 typedef struct {
 	uint8_t resvd1[3];

-- 
1.8.3.1

[PATCH rdma-core 3/3] pyverbs: Extend support of NDR rates

From: Yishai Hadas <yishaih@nvidia.com>
Date: 2021-12-29 08:55:47

From: Edward Srouji <redacted>

Add new NDR speed definitions/enums to support the new data rates.

Signed-off-by: Edward Srouji <redacted>
---
 pyverbs/device.pyx           | 3 ++-
 pyverbs/libibverbs_enums.pxd | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/pyverbs/device.pyx b/pyverbs/device.pyx
index 55d708e..6f68a68 100644
--- a/pyverbs/device.pyx
+++ b/pyverbs/device.pyx

@@ -1091,7 +1091,8 @@ def translate_port_cap_flags2(flags):
          e.IBV_PORT_VIRT_SUP: 'IBV_PORT_VIRT_SUP',
          e.IBV_PORT_SWITCH_PORT_STATE_TABLE_SUP: 'IBV_PORT_SWITCH_PORT_STATE_TABLE_SUP',
          e.IBV_PORT_LINK_WIDTH_2X_SUP: 'IBV_PORT_LINK_WIDTH_2X_SUP',
-         e.IBV_PORT_LINK_SPEED_HDR_SUP: 'IBV_PORT_LINK_SPEED_HDR_SUP'}
+         e.IBV_PORT_LINK_SPEED_HDR_SUP: 'IBV_PORT_LINK_SPEED_HDR_SUP',
+         e.IBV_PORT_LINK_SPEED_NDR_SUP: 'IBV_PORT_LINK_SPEED_NDR_SUP'}
     return str_from_flags(flags, l)

diff --git a/pyverbs/libibverbs_enums.pxd b/pyverbs/libibverbs_enums.pxd
index 93a0f13..b3ea226 100644
--- a/pyverbs/libibverbs_enums.pxd
+++ b/pyverbs/libibverbs_enums.pxd

@@ -74,6 +74,7 @@ cdef extern from '<infiniband/verbs.h>':
         IBV_PORT_SWITCH_PORT_STATE_TABLE_SUP
         IBV_PORT_LINK_WIDTH_2X_SUP
         IBV_PORT_LINK_SPEED_HDR_SUP
+        IBV_PORT_LINK_SPEED_NDR_SUP
 
     cpdef enum ibv_mtu:
         IBV_MTU_256

-- 
1.8.3.1

Re: [PATCH rdma-core 0/3] Add NDR support

From: Yishai Hadas <yishaih@nvidia.com>
Date: 2022-01-11 07:52:48

On 12/29/2021 10:54 AM, Yishai Hadas wrote:

This series extends verbs and diags to include the NDR support.

It includes the NDR support bit from PortInfo.CapabilityMask2 and the new rates
as were defined in IB Spec Release 1.5.

The matching utilities inside verbs to print and convert to/from the new rates
were updated as well.

In addition, the series includes some pyverbs extension to support the new NDR
definitions.

PR was sent:
https://github.com/linux-rdma/rdma-core/pull/1114

Yishai

Edward Srouji (1):
   pyverbs: Extend support of NDR rates

Maher Sanalla (2):
   verbs: Extend support of NDR rates
   ibdiags: Extend support of NDR rates

  infiniband-diags/ibportstate.c | 8 +++++++-
  libibmad/iba_types.h           | 3 +++
  libibverbs/examples/devinfo.c  | 1 +
  libibverbs/verbs.c             | 8 ++++++++
  libibverbs/verbs.h             | 3 +++
  pyverbs/device.pyx             | 3 ++-
  pyverbs/libibverbs_enums.pxd   | 1 +
  7 files changed, 25 insertions(+), 2 deletions(-)

The series was merged.

Yishai

`h`	back out one level
`j`	next message in thread
`k`	previous message in thread
`l`	drill in
`Esc`	close help / fold thread tree
`?`	toggle this help