Re: [PATCH net-next v3 1/2] net/smc: make wr buffer count configurable
From: Paolo Abeni <pabeni@redhat.com>
Date: 2025-09-25 09:27:43
Also in:
linux-doc, linux-rdma, linux-s390, lkml
On 9/21/25 11:44 PM, Halil Pasic wrote:
quoted hunk ↗ jump to hunk
diff --git a/Documentation/networking/smc-sysctl.rst b/Documentation/networking/smc-sysctl.rst index a874d007f2db..c94d750c7c84 100644 --- a/Documentation/networking/smc-sysctl.rst +++ b/Documentation/networking/smc-sysctl.rst@@ -71,3 +71,39 @@ smcr_max_conns_per_lgr - INTEGER acceptable value ranges from 16 to 255. Only for SMC-R v2.1 and later. Default: 255 + +smcr_max_send_wr - INTEGER + So called work request buffers are SMCR link (and RDMA queue pair) level + resources necessary for performing RDMA operations. Since up to 255 + connections can share a link group and thus also a link and the number + of the work request buffers is decided when the link is allocated, + depending on the workload it can a bottleneck in a sense that threads
missing 'be' or 'become' here^^
+ have to wait for work request buffers to become available. Before the + introduction of this control the maximal number of work request buffers + available on the send path used to be hard coded to 16. With this control + it becomes configurable. The acceptable range is between 2 and 2048. + + Please be aware that all the buffers need to be allocated as a physically + continuous array in which each element is a single buffer and has the size + of SMC_WR_BUF_SIZE (48) bytes. If the allocation fails we give up much + like before having this control. + + Default: 16 + +smcr_max_recv_wr - INTEGER + So called work request buffers are SMCR link (and RDMA queue pair) level + resources necessary for performing RDMA operations. Since up to 255 + connections can share a link group and thus also a link and the number + of the work request buffers is decided when the link is allocated, + depending on the workload it can a bottleneck in a sense that threads
same here^^ [...]
quoted hunk ↗ jump to hunk
@@ -683,6 +678,8 @@ int smc_ib_create_queue_pair(struct smc_link *lnk) }; int rc; + qp_attr.cap.max_send_wr = 3 * lnk->lgr->max_send_wr; + qp_attr.cap.max_recv_wr = lnk->lgr->max_recv_wr;
Possibly: cap = max(3 * lnk->lgr->max_send_wr, lnk->lgr->max_recv_wr); qp_attr.cap.max_send_wr = cap; qp_attr.cap.max_recv_wr = cap to avoid assumption on `max_send_wr`, `max_recv_wr` relative values. [...]
quoted hunk ↗ jump to hunk
diff --git a/net/smc/smc_sysctl.h b/net/smc/smc_sysctl.h index eb2465ae1e15..8538915af7af 100644 --- a/net/smc/smc_sysctl.h +++ b/net/smc/smc_sysctl.h@@ -25,6 +25,8 @@ static inline int smc_sysctl_net_init(struct net *net) net->smc.sysctl_autocorking_size = SMC_AUTOCORKING_DEFAULT_SIZE; net->smc.sysctl_max_links_per_lgr = SMC_LINKS_PER_LGR_MAX_PREFER; net->smc.sysctl_max_conns_per_lgr = SMC_CONN_PER_LGR_PREFER; + net->smc.sysctl_smcr_max_send_wr = SMCR_MAX_SEND_WR_DEF; + net->smc.sysctl_smcr_max_recv_wr = SMCR_MAX_RECV_WR_DEF; return 0; }diff --git a/net/smc/smc_wr.c b/net/smc/smc_wr.c index b04a21b8c511..f5b2772414fd 100644 --- a/net/smc/smc_wr.c +++ b/net/smc/smc_wr.c@@ -34,6 +34,7 @@ #define SMC_WR_MAX_POLL_CQE 10 /* max. # of compl. queue elements in 1 poll */ #define SMC_WR_RX_HASH_BITS 4 +
Please avoid unrelated whitespace only changes. Thanks, Paolo