On Mon, 2014-11-17 at 19:13 -0800, Alexander Duyck wrote:

ARM adds some funky things.  They have two different types of 
primitives, a dmb() which is a data memory barrier, and a dsb() which is 
a data synchronization barrier.  Then with each of those they have the 
"domains" the barriers are effective within.

So for example on ARM a rmb() is dsb(sy) which means it is a system wide 
synchronization barrier which stops execution on the CPU core until the 
read completes.  

That's amazingly heavy handed ... I can see that being useful for MMIO,
we do something similar in our MMIO accessors by using a special variant
of trap instruction that never traps to make the core thing the load
value has been consumed. But that's typically only needed to guarantee
MMIO timings.

However the smp_rmb() is a dmb(ish) which means it is 
only a barrier as far as the inner shareable domain which I believe only 
goes as far as the local shared cache hierarchy and only guarantees read 
ordering without necessarily halting the CPU or stopping in-order 
speculative reads.  So what a coherent_rmb() would be in my setup is 
dmb(sy) which means the barrier runs all the way out to memory, and it 
is allowed to speculative read as long as it does it in order.

Correct, which is thus the same as smp_rmb() ... which was my original
point, or am I missing something else ?

If it is still unclear you might check out Will Deacon's talk on the 
topic at https://www.youtube.com/watch?v=6ORn6_35kKo, at about 7:00 in 
he explains the whole domains thing, and at 13:30 he explains dmb()/dsb().

Ok, I'll try to watch that when I get a chance.

Cheers,
Ben.