Fwd: [RFC PATCH 0/3] RDMA/rxe: Add dma-buf support

-------- Forwarded Message --------
Subject: Re: [RFC PATCH 0/3] RDMA/rxe: Add dma-buf support
Date: Wed, 8 Sep 2021 22:23:53 -0500
From: Bob Pearson <redacted>
To: Shunsuke Mie <redacted>
CC: Jason Gunthorpe <jgg@nvidia.com>

On 9/8/21 9:17 PM, Shunsuke Mie wrote:

Hi Bob,

I agree with you. The fix has to be merged firstly. I'd like to know
the patches.
Is the patch for that bug fix at the following link?
https://lore.kernel.org/linux-rdma/20210908052928.17375-1-rpearsonhpe@gmail.com/ (local)

Thanks,

2021年9月9日(木) 7:52 Bob Pearson [off-list ref]:

quoted

On 9/8/21 1:16 AM, Shunsuke Mie wrote:

quoted

This patch series add a support for rxe driver.

A dma-buf based memory registering has beed introduced to use the memory
region that lack of associated page structures (e.g. device memory and CMA
managed memory) [1]. However, to use the dma-buf based memory, each rdma
device drivers require add some implementation. The rxe driver has not
support yet.

[1] https://www.spinics.net/lists/linux-rdma/msg98592.html

To enable to use the memories in rxe rdma device, add some changes and
implementation in this patch series.

This series consists of three patches. The first patch changes the IB core
to support for rdma drivers that have not real dma device. The second
patch extracts a memory mapping process of rxe as a common function to use
a dma-buf support. The third patch adds the dma-buf support to rxe driver.

Related user space RDMA library changes are provided as a separate
patch.

Shunsuke Mie (3):
  RDMA/umem: Change for rdma devices has not dma device
  RDMA/rxe: Extract a mapping process into a function
  RDMA/rxe: Support dma-buf as memory region

 drivers/infiniband/core/umem_dmabuf.c |   2 +-
 drivers/infiniband/sw/rxe/rxe_loc.h   |   3 +
 drivers/infiniband/sw/rxe/rxe_mr.c    | 186 +++++++++++++++++++++-----
 drivers/infiniband/sw/rxe/rxe_verbs.c |  36 +++++
 4 files changed, 193 insertions(+), 34 deletions(-)

Mie,

This looks very interesting. But, I am afraid we are going to collide. I just submitted some patches
to fix blktest which will very likely overlap with yours. Please take a look. It should be easy to
merge them. I am anxious to clear the blktest bug ASAP though.

Bob Pearson

Mie,

I just forwarded to you the emails in the patch series. 1/5 and 2/5 are mostly elsewhere. But 2-5/5 are directly changing rxe_mr.c and especially 4/5 touches the SG to page list code. Per a recent exchange
with Jason we needed to keep two copies of the buffer list so that the kernel space API could be
used as intended and allow overlap of invalidating one MR while a new one is getting built and then
registered. In case, you can look at all the emails at https://lore.kernel.org/linux-rdma/.

Bob