Re: [PATCH rdma-next 0/3] Support out of order data placement

On Tue, Aug 01, 2017 at 10:06:14PM +0000, Parav Pandit wrote:

quoted

quoted

quoted

 Initial Condition VA=0 Data = 0
 RDMA-W VA=0 Data=1
 RDMA-R VA=0

Spec says 1 must be returned, but sounds like this relaxed version could

return 0.

quoted

No. Table 76 stays as is as described before.

How is this possible?

I am not sure what more can I explain you Jason.
Requester side HCA follows HCA Table-76.
Incoming read responses are not processed until previous writes are
ACKed implicitly (in read responses) or explicitly by ACK packets.
Same as before described in spec. No extra description needed for
this patchset.

But doing that pretty much destroys much of the entire point of having
a relaxed ordering :P

quoted

quoted

quoted

 RDMA-W VA=0 Data=1
 RDMA-W VA=0 Data=2
 SEND

Sounds like with the relaxed version the app could see 1 at SEND CQ time.

So RDMA-W -> RDMA-W degrades to a F

quoted

No. Table-76 is based on  how requester sees the execution.
So it stays as '#'.

How is this possible?

Please don't mix requester side ordering with responder side execution.
C9-28 on responder side is relaxed - as explained few times before.

No, I see what you are tring to say now. I disagree with
this. Table-76 and C9-28 are describing the same thing, you cannot
weaken C9-28 without also restating Table 76.

Table 76 is clearly talking about the entire system, including the
execution and memory subsystem of the completer.

It covers only requester side.
Send with invalidate execution on responder side is described in 9.4.1.1.1

I suppose 9.4.1.1.1 point #1 already allows the out of order behavior.

You are proposing a different behavior and attribute which may be
done for a HCA that support such thing.  Please submit a different
patch for it whenever its appropriate.  Current query HCA attribute
is bit field for future relaxation. May be what you described can be
done.

Since you guys are hell bent on avoiding the IBTA for your new verbs
features, you do have to define them in a sensible and usefully broad
way using the community process.

Other out of order atomics such as
Atomic->Read
Write->Atomic may be done in future under different attribute.

I think that is a mistake, you should start with them being out of
order and require the app to fence to bring order back, even if the
current HCAs execute them in order anyhow.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html