On Sun, 28 Sep 2025 19:42:54 +0800
Dust Li [off-list ref] wrote:

quoted

quoted

We have at most 2 RDMA Write for 1 RDMA send. So 3 times is necessary.
That is explained in the original comments. Maybe it's better to keep it.

.cap = {
                /* include unsolicited rdma_writes as well,
                 * there are max. 2 RDMA_WRITE per 1 WR_SEND
                 */  

But what are "the unsolicited" rdma_writes? I have heard of
unsolicited receive, where the data is received without
consuming a WR previously put on the RQ on the receiving end, but
the concept of unsolicited rdma_writes eludes me completely.  

unsolicited RDMA Writes means those RDMA Writes won't generate
CQEs on the local side. You can refer to:
https://www.rdmamojo.com/2014/05/27/solicited-event/

quoted

I guess what you are trying to say, and what I understand is
that we first put the payload into the RMB of the remote, which
may require up 2 RDMA_WRITE operations, probably because we may
cross the end (and start) of the array that hosts the circular
buffer, and then we send a CDC message to update the cursor.

For the latter a  ib_post_send() is used in smc_wr_tx_send()
and AFAICT it consumes a WR from wr_tx_bufs. For the former
we consume a single wr_tx_rdmas which and each wr_tx_rdmas
has 2 WR allocated.  

Right.

Thank you Dust Li! Unfortunately I have already spinned a v4. Let
me add back that comment, as for people knowledgeable enough it does
not appear to be confusing at all. I can try to improve that comment
and maybe add a new one on the reason why we do need more WRs on the
receive end than on the send end, after this series has been merged. Or
if you want to do it yourself, I'm happy with it as well. In the end
it is you who me helped get a better understanding of this :)

Thank you again!

Regards,
Halil