Re: [RFC iproute2 0/8] RDMA tool

On Thu, May 04, 2017 at 03:26:13PM -0400, Dennis Dalessandro wrote:

On 05/04/2017 02:45 PM, Leon Romanovsky wrote:

quoted

On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:

quoted

On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:

quoted

On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:

quoted

On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:

quoted

Following our discussion both in mailing list [1] and at the LPC 2016 [2],
we would like to propose this RDMA tool to be part of iproute2 package
and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to
add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2,
like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
RDMA.

I do expect that iproute2 will be installed on every machine with any
type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent
our own for one specific tool.

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality
in iproute2. There is already a library for communication over netlink sockets,
namely libnl. Is there functionality that is in iproute2 but not in libnl and
that is needed for the new tool? If so, have you considered to create a new
library for that functionality?

It is not hard to create new tool, the hardest part is to ensure that it is
part of the distributions. Did you count how many months we are trying to
add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However at
the end of the day it's just a name for a repo/package. If the iproute folks
are fine to include rdma in their repo/package, great we can leverage their
code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would object
to that, but the interface is "rdma ... " so from users perspective it's
different tools. They don't need to care that it was sourced from a common
git repo.

Just as an aside this already works a bit with OPA:

 $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
link_layer InfiniBand
         phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really bogged
down this week, sorry.

Thanks Denny,

Before you are starting to test it, can you please provide your feedback
on my initial questions? Usability and need of sysfs.

----
This is initial phase to understand if user experience for this tool fits
RDMA and netdev communities exepectations. Also I would like to get feedback
if it is really worth to provide legacy sysfs for old kernels, or maybe I should
implement netlink from the beginning and abandon sysfs completely.
-----

P.S. I believe this will give you wrong output, because it parses IB port cap_mask.
$./rdma link show hfi1_0/1 cap_mask

Thanks

-Denny