On 11/15/2012 05:21 PM, Will Deacon wrote:

Hi Gregory,

On Thu, Nov 15, 2012 at 03:54:39PM +0000, Gregory CLEMENT wrote:

quoted

On 11/15/2012 11:17 AM, Will Deacon wrote:

quoted

Interesting, thanks for asking them about this. Does this mean that:

Here come the answers to your new questions

Great, thanks for the quick turn-around!

quoted

quoted

	1. When not running coherently (i.e. before initialising the
	   coherency fabric), memory is treated as non-shareable,
	   non-cacheable?

It can be cacheable. The shared memory (as defined on the page table)
will NOT be coherent by HW.

Ok, so we really are incoherent before enabling the fabric.

quoted

quoted

	2. If (1), then are exclusive accesses the only way to achieve
	   coherent memory accesses in this scenario?

I quote: "I suspect there is terminology miss-use: exclusive accesses
are NOT used to achieve memory coherency - they are used to achieve
atomicity. To achieve memory coherency while fabric is configured to
be non-coherent, SW should use maintenance operations over the L1
caches."

Ok, so if I'm understanding correctly then I don't really see the usefulness
of having working exclusives that are incoherent. Surely it means that you
can guarantee mutual exclusion on a lock variable, but the value you actually
end up reading from the lock is junk unless you litter the accessors with cache
clean operations?

Anyway, that's by-the-by as this is all called early enough that we
shouldn't care. The thing I don't like now is that the fabric initialisation
is done entirely differently on the primary CPU than the secondaries. The
primary probes the device-tree (well, it's also now hard-coded for v2) and
accesses the registers from a C function(armada_370_xp_set_cpu_coherent) whilst
the secondaries have hardcoded addresses and access via asm
(armada_xp_secondary_startup).

Now it is hardcoded in both case as you pointed it. So the last
difference is setup from a C function or via asm.

The differences between primary and secondary CPU when they enable the
coherency, is due to the fact that we really are in a different
situation. For primary CPU, as it is the only CPU online it doesn't
need to enable the coherency from the beginning, so we can wait to
have MMU enable and convenient feature. Whereas for the secondary CPU
they need the coherency from the very beginning are by definition they
won't be alone. That's why this very first instruction are written in
asm and they use physical address.

I don't see how to handle it in a different way.

Gregory