On 9/10/21 3:23 PM, Bart Van Assche wrote:

On 9/10/21 12:38 PM, Pearson, Robert B wrote:

quoted

1. Which rdma-core are you running? Out of box or the github tree?

I'm using the rdma-core package included in openSUSE Tumbleweed. blktests
pass with that rdma-core package against older kernel versions so I think
the rdma-core package is fine. The version number of the rdma-core package
I'm using is as follows:
$ rpm -q rdma-core
rdma-core-36.0-1.1.x86_64

The rdma tool comes from the iproute2 package:
$ rpm -qf /sbin/rdma
iproute2-5.13-1.1.x86_64

quoted

3. Where did you get the kernel bits? Which git tree? Which branch?

Hmm ... wasn't that mentioned in my previous email? I mentioned a commit
SHA and these SHA numbers are unique and unambiguous. Anyway: commit
2169b908894d comes from the for-rc branch of the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Bart.

You'd be surprised how much I don't know. I do know the numbers are unique but I
haven't the faintest idea how to decode them into useful strings.

In theory you are correct and rdma-core and kernels are supposed to be forwards and
backwards compatible but that is a goal and sometimes regressions do occur. I can try
to run with that version just to make sure.

There is a problem I have seen where some newer distros do not create the default IPV6
address from the MAC address. They randomize it (Ubuntu does this) and rxe is broken
as a result. I end up having to add a line like 

sudo ip addr add dev enp6s0 fe80::b62e:99ff:fef9:fa2e/64
  (where the MAC address is b4:2e:99:f9:fa:2e) just before the line
sudo rdma link add rxe_1 type rxe netdev enp6s0

But, when this is an issue rxe is really broken and almost nothing works so that may not
be an issue for you.

I will try to recreate your setup and retest.

Thanks,

Bob