Re: [PATCH] RDMA/mana_ib: Use v2 version of cfg_rx_steer_req to enable RX coalescing

On Wed, May 10, 2023 at 04:37:57PM +0000, Long Li wrote:

quoted

Subject: Re: [PATCH] RDMA/mana_ib: Use v2 version of cfg_rx_steer_req to
enable RX coalescing

On Tue, May 09, 2023 at 07:08:36PM +0000, Long Li wrote:

quoted

quoted

Subject: Re: [PATCH] RDMA/mana_ib: Use v2 version of
cfg_rx_steer_req to enable RX coalescing

On Mon, May 08, 2023 at 02:45:44PM +0000, Haiyang Zhang wrote:

quoted

quoted

-----Original Message-----
From: Leon Romanovsky <leon@kernel.org>
Sent: Monday, May 8, 2023 2:10 AM
To: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Long Li <longli@microsoft.com>; Jason Gunthorpe
[off-list ref]; Ajay Sharma [off-list ref]; Dexuan
Cui [off-list ref]; KY Srinivasan [off-list ref];
Wei Liu

[off-list ref]; David S.

quoted

quoted

Miller [off-list ref]; Eric Dumazet
[off-list ref]; Jakub Kicinski [off-list ref]; Paolo
Abeni [off-list ref];
linux- rdma@vger.kernel.org; linux-hyperv@vger.kernel.org;
netdev@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH] RDMA/mana_ib: Use v2 version of
cfg_rx_steer_req to enable RX coalescing

On Sun, May 07, 2023 at 09:39:27PM +0000, Haiyang Zhang wrote:

quoted

quoted

-----Original Message-----
From: Leon Romanovsky <leon@kernel.org>
Sent: Sunday, May 7, 2023 4:11 AM
To: Long Li <longli@microsoft.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>; Ajay Sharma
[off-list ref]; Dexuan Cui
[off-list ref]; KY Srinivasan [off-list ref];
Haiyang Zhang

[off-list ref];

quoted

quoted

Wei Liu [off-list ref]; David S. Miller
[off-list ref]; Eric Dumazet [off-list ref];

Jakub

quoted

quoted

quoted

quoted

Kicinski [off-list ref];

Paolo

quoted

quoted

Abeni [off-list ref]; linux-rdma@vger.kernel.org;
linux- hyperv@vger.kernel.org; netdev@vger.kernel.org;
linux- kernel@vger.kernel.org
Subject: Re: [PATCH] RDMA/mana_ib: Use v2 version of
cfg_rx_steer_req

to

quoted

quoted

enable RX coalescing

On Fri, May 05, 2023 at 11:51:48AM -0700,
longli@linuxonhyperv.com
wrote:

quoted

From: Long Li <longli@microsoft.com>

With RX coalescing, one CQE entry can be used to indicate
multiple

packets

quoted

quoted

quoted

on the receive queue. This saves processing time and PCI
bandwidth over the CQ.

Signed-off-by: Long Li <longli@microsoft.com>
---
 drivers/infiniband/hw/mana/qp.c |  5 ++++-
 include/net/mana/mana.h         | 17 +++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)

Why didn't you change mana_cfg_vport_steering() too?

The mana_cfg_vport_steering() is for mana_en (Enthernet)
driver, not the mana_ib driver.

The changes for mana_en will be done in a separate patch
together with changes for mana_en RX code patch to support
multiple packets /

CQE.

quoted

quoted

I'm aware of the difference between mana_en and mana_ib.

The change you proposed doesn't depend on "support multiple
packets / CQE."
and works perfectly with one packet/CQE also, does it?

No.
If we add the following setting to the mana_en /
mana_cfg_vport_steering(), the NIC may put multiple packets in one
CQE, so we need to have the changes for mana_en RX code path to
support

multiple packets / CQE.

quoted

+	req->cqe_coalescing_enable = true;

You can leave "cqe_coalescing_enable = false" for ETH and still
reuse your new
v2 struct.

I think your proposal will work for both Ethernet and IB.

The idea is that we want this patch to change the behavior of the IB driver. We

plan to make another patch for the Ethernet driver. This makes it easier to track
all changes for a driver.

And I don't want to deal with deletion of v1 struct for two/three kernel cycles
instead of one patch in one cycle.

I'm resubmitting this patch to replace v1 for both driver.

Great

quoted

quoted

quoted

H>

quoted

So we plan to set this cqe_coalescing_enable, and the changes for
mana_en RX code path to support multiple packets / CQE in another

patch.

quoted

quoted

And how does it work with IB without changing anything except this
proposed patch?

The RX CQE Coalescing is implemented in the user-mode. This feature is

always turned on from cluster. The user-mode code is written in a way that can
deal with both CQE Coalescing and CQE non-coalescing, so it doesn't depend on
kernel version for the correct behavior.

Yes, but how does userspace know that CQE coalescing was enabled?

The user-mode doesn't know if CQE is enabled in advance. If this information is required I can modify the patch to pass this information along to rdma-core. However, this is not useful as the cluster is running with CQE coalescing enabled by default, there is no need to know this information.

It is ok, not needed. if we can do something without UAPI involvement,
it will be always preferable way.

Thanks