On 7/26/2021 1:53 PM, Chuck Lever III wrote:

quoted

On Jul 26, 2021, at 12:50 PM, Tom Talpey [off-list ref] wrote:

On 7/26/2021 10:46 AM, Chuck Lever wrote:

quoted

  /**
   * svc_rdma_wc_send - Invoked by RDMA provider for each polled Send WC
   * @cq: Completion Queue context

@@ -275,11 +289,9 @@ static void svc_rdma_wc_send(struct ib_cq *cq, struct ib_wc *wc)
    	trace_svcrdma_wc_send(wc, &ctxt->sc_cid);
  +	svc_rdma_wake_send_waiters(rdma, 1);
  	complete(&ctxt->sc_done);
  -	atomic_inc(&rdma->sc_sq_avail);
-	wake_up(&rdma->sc_send_wait);

This appears to change the order of wake_up() vs complete().
Is that intentional?

IIRC I reversed the order here to be consistent with the other
Send completion handlers.

quoted

Is there any possibility of a false
scheduler activation, later leading to a second wakeup or poll?

The two "wake-ups" here are not related to each other, and RPC
Replies are transmitted already so this shouldn't have a direct
impact on server latency. But I might not understand your
question.

IIUC, you're saying that the thread which is awaiting the
completion of ctxt->sc_done is not also waiting to send
anything, therefore no thread is going to experience a
fire drill. Ok.

Feel free to add my
   Reviewed-By: Tom Talpey [off-list ref]
to the series.

Tom.