LDREX/STREX and pre-emption on SMP hardware

Catalin Marinas wrote:

With interrupts (I1, I2 interrupt handlers)

I1			I2
LDREX
			LDREX
			STREX (succeeds)
STREX (fails)

In the interrupt case, they are nested so the STREX in I2 is always
executed before STREX in I1 (you can extrapolate with several nested
interrupts).

This assumes LDREX/STREX are always called in pairs.  But this is in
fact _not_ the case.  Take a look at atomic_cmpxchg:

	do {
		__asm__ __volatile__("@ atomic_cmpxchg\n"
		"ldrex	%1, [%2]\n"
		"mov	%0, #0\n"
		"teq	%1, %3\n"
		"strexeq %0, %4, [%2]\n"
		    : "=&r" (res), "=&r" (oldval)
		    : "r" (&ptr->counter), "Ir" (old), "r" (new)
		    : "cc");
	} while (res);

In the case where ptr->counter != old, STREX is not executed, and the
do{...}while loop does not loop.  Thus LDREX/STREX aren't paired.

It may be that atomic_cmpxchg() is always called in a loop, but if so
I don't think that's a documented requirement for all callers.

A simple solution is to either call CLREXNE after STREXEQ, or change
STREXEQ to STREX and change the logic so that if ptr->counter != old,
the value it tries to store is what it read.  The latter uses fewer
instructions - and even eliminates the do...while, making it really
compact, but may cause more cache line dirtying.

If you think I'm right and tell me what you prefer I'll prepare a
patch.

-- Jamie