Re: [PATCH RFC net-next 5/6] rds: support for zcopy completion notification

quoted

This changes the semantics of these fields. Please add a new SO_EE_CODE flag,
even if the semantics can be derived from the packet family (for now).

sounds good, I'll take care of this (and other review comments)
for the next rev.

quoted

Even better would be if we can avoid the cookies completely. I understand
the issue with concurrent send threads racing on obtaining a zckey value. If
the sender could learn which zckey was chosen for a call, would that suffice?

I'm not sure I understand the proposal- you want sendmsg to return
the cookie ("zckey") for you? How?

even if we mangled sendmsg() to return something other than
the existing POSIX semantics, how will the application be asynchronously
notified that send has completed (on a per-message/per-datagram) basis?

I'm purposely glossing over how the kernel returns this item for now.
Was just wondering whether we can then assume mostly in order delivery
and reuse the existing notification interface from tcp zerocopy.

From your experiments, it sounds like this is not the case. In which case

there is little benefit to trying to force linear IDs derived from sk->sk_zckey.

quoted

I suspect that in even with concurrent senders, notifications arrive
largely in
order, in which case we could just maintain the existing semantics and even
reuse that implementation.

not so. rds-stress [1] with -d8 -t8 quickly disproves this on my 10G ixgbe
connection.

Okay. In that case, the cmsg cookie approach sounds fine. I had the same
in an early tcp zerocopy test version, as a matter of fact.

quoted

quoted

                } rdma;
                struct rm_data_op {
                        unsigned int            op_active:1;
-                       unsigned int            op_notify:1;
+                       unsigned int            op_notify:1,
+                                               op_zcopy:1,

                                                     :

quoted

quoted

+                       struct rds_znotifier    *op_mmp_znotifier;

not necessary if op_mmp_znotifier is NULL unless set in
rds_message_copy_from_user

To make sure I dont misunderstand, you are suggesting that we dont
need op_zcopy, but can just check for the null-ness of
op_mmp_znotifier (yes, true, I agree)? or something else?

Yes, that's what I meant.