On 02/02/26 3:18 pm, D. Wythe wrote:

The current SMC-R implementation relies on global per-device CQs
and manual polling within tasklets, which introduces severe
scalability bottlenecks due to global lock contention and tasklet
scheduling overhead, resulting in poor performance as concurrency
increases.

Refactor the completion handling to utilize the ib_cqe API and
standard RDMA core CQ pooling. This transition provides several key
advantages:

1. Multi-CQ: Shift from a single shared per-device CQ to multiple
link-specific CQs via the CQ pool. This allows completion processing
to be parallelized across multiple CPU cores, effectively eliminating
the global CQ bottleneck.

2. Leverage DIM: Utilizing the standard CQ pool with IB_POLL_SOFTIRQ
enables Dynamic Interrupt Moderation from the RDMA core, optimizing
interrupt frequency and reducing CPU load under high pressure.

3. O(1) Context Retrieval: Replaces the expensive wr_id based lookup
logic (e.g., smc_wr_tx_find_pending_index) with direct context retrieval
using container_of() on the embedded ib_cqe.

4. Code Simplification: This refactoring results in a reduction of
~150 lines of code. It removes redundant sequence tracking, complex lookup
helpers, and manual CQ management, significantly improving maintainability.

Performance Test: redis-benchmark with max 32 connections per QP
Data format: Requests Per Second (RPS), Percentage in brackets
represents the gain/loss compared to TCP.

| Clients | TCP      | SMC (original)      | SMC (cq_pool)       |
|---------|----------|---------------------|---------------------|
| c = 1   | 24449    | 31172  (+27%)       | 34039  (+39%)       |
| c = 2   | 46420    | 53216  (+14%)       | 64391  (+38%)       |
| c = 16  | 159673   | 83668  (-48%)  <--  | 216947 (+36%)       |
| c = 32  | 164956   | 97631  (-41%)  <--  | 249376 (+51%)       |
| c = 64  | 166322   | 118192 (-29%)  <--  | 249488 (+50%)       |
| c = 128 | 167700   | 121497 (-27%)  <--  | 249480 (+48%)       |
| c = 256 | 175021   | 146109 (-16%)  <--  | 240384 (+37%)       |
| c = 512 | 168987   | 101479 (-40%)  <--  | 226634 (+34%)       |

The results demonstrate that this optimization effectively resolves the
scalability bottleneck, with RPS increasing by over 110% at c=64
compared to the original implementation.

I applied your patch to the latest kernel(6.19-rc8) & saw below
Performance results:

1) In my evaluation, I ran several *uperf* based workloads using a
request/response (RR) pattern, and I observed performance *degradation*
ranging from *4%* to *59%*, depending on the specific read/write sizes
used. For example, with a TCP RR workload using 50 parallel clients
(nprocs=50) sending a 200‑byte request and reading a 1000‑byte response
over a 60‑second run, I measured approximately 59% degradation compared
to SMC‑R original performance.

2) In contrast, with uperf *streaming‑type* workloads, your patch shows
clear gains. I observed performance *improvement* ranging from *11%* to
*75%*, again depending on the specific streaming parameters. One
representative case is a TCP streaming/bulk‑receive workload with 250
parallel clients (nprocs=250) performing 640 reads per burst with 30 KB
per read, running continuously for 60 seconds, where I measured
approximately *75%* *improvement* over the SMC‑R original performance.

Note: I ran above tests with default WR(work request buffers), default
receive & transmit buffer size with smc_run.

I am looking for additional details regarding the redis-benchmark
performance results you previously shared. I would like to understand
whether the workload behaved more like a traditional request/response
(RR) pattern or a streaming-type workload, and what SMC‑R configuration
was used during the tests?

1) SMC Work Request (WR) Settings - Did your test environment use the
default SMC‑R work request buffers?
  net.smc.smcr_max_recv_wr = 48
  net.smc.smcr_max_send_wr = 16
2) SMC-R Buffer sizes used via smc_run - Did you use default transmit &
receive buffer sizes(smc_run -r <recv_size> -t <send_size>)?
3) Additional system or network tuning e.g CPU affinity, NIC offload
settings etc?