Thread (5 messages) 5 messages, 2 authors, 9d ago
COOLING9d
Revisions (4)
  1. v1 [diff vs current]
  2. v1 [diff vs current]
  3. v3 current
  4. v4 [diff vs current]

[PATCH net-next v3 0/3] net/smc: transition to RDMA core CQ pooling

From: "D. Wythe" <alibuda@linux.alibaba.com>
Date: 2026-07-10 03:34:09
Also in: linux-rdma, linux-s390

This series transitions SMC-R completion handling to RDMA core CQ pooling
via the ib_cqe API. The new completion model improves scalability by
allowing per-link completion processing across multiple cores and enables
DIM-based interrupt moderation.

As a side effect, the increased concurrency can amplify contention for TX
slots on the shared wait queue. Patch 3 addresses this by switching TX slot
allocation from non-exclusive wait_event() to prepare_to_wait_exclusive(),
which avoids thundering-herd wakeups under contention.

Patch 1 fixes smc_wr_tx_put_slot() to clear the v2 pending slot and buffer
structures instead of the pointer variables.
Patch 2 replaces the global per-device CQ and manual tasklet polling model
with RDMA core CQ pooling.
Patch 3 reduces TX slot contention by using exclusive wait queue entries
during allocation.

Link: https://lore.kernel.org/netdev/20260305022323.96125-1-alibuda@linux.alibaba.com/ (local)

---
Changes v1 -> v2:
https://lore.kernel.org/netdev/20260508063718.101622-1-alibuda@linux.alibaba.com/ (local)
1. remove unnecessary inline from static CQE init helpers.
2. Use ib_drain_qp() with +1 max_send_wr; 
3. Fix v2 state clearing.
4. Add re-check after schedule_timeout() to fix timeout/signal races.

Changes v2 -> v3:
https://lore.kernel.org/netdev/20260528084819.6059-1-alibuda@linux.alibaba.com/ (local)
1. Reserve +3 instead of +1 for the SQ to cover the drain, FastReg and
   SMC-Rv2 SEND WRs, avoiding SQ exhaustion that breaks ib_drain_sq().
2. Guard the recv WR repost with a per-link percpu_ref so no WR is
   reposted after ib_drain_qp(), fixing the RX repost/drain use-after-free.
3. Split the smc_wr_tx_put_slot() v2 clearing fix into a separate patch
   (1/3) with a Fixes: tag and reworded to the verifiable root cause.

D. Wythe (3):
  net/smc: clear the correct v2 slot and buffer in smc_wr_tx_put_slot()
  net/smc: transition to RDMA core CQ pooling
  net/smc: reduce TX slot contention with exclusive wait

 net/smc/smc_core.c |  10 +-
 net/smc/smc_core.h |  32 +++-
 net/smc/smc_ib.c   | 115 ++++---------
 net/smc/smc_ib.h   |   7 -
 net/smc/smc_tx.c   |   1 -
 net/smc/smc_wr.c   | 420 ++++++++++++++++++++++-----------------------
 net/smc/smc_wr.h   |  48 ++----
 7 files changed, 285 insertions(+), 348 deletions(-)

-- 
2.45.0
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help