On Mon, Mar 29, 2021 at 9:56 PM Arnd Bergmann [off-list ref] wrote:

On Mon, Mar 29, 2021 at 2:52 PM Guo Ren [off-list ref] wrote:

quoted

On Mon, Mar 29, 2021 at 7:31 PM Peter Zijlstra [off-list ref] wrote:

quoted

On Mon, Mar 29, 2021 at 01:16:53PM +0200, Peter Zijlstra wrote:

quoted

Anyway, an additional 'funny' is that I suspect you cannot prove fwd
progress of the entire primitive with any of this on. But who cares
about details anyway.. :/

What's the architectural guarantee on LL/SC progress for RISC-V ?

funct5    | aq | rl   | rs2 |  rs1  | funct3 | rd | opcode
     5          1    1      5       5         3        5          7
LR.W/D  ordering  0     addr    width   dest    AMO
SC.W/D  ordering  src  addr    width   dest    AMO

LR.W loads a word from the address in rs1, places the sign-extended
value in rd, and registers a reservation set—a set of bytes that
subsumes the bytes in the addressed word. SC.W conditionally writes a
word in rs2 to the address in rs1: the SC.W succeeds only if the
reservation is still valid and the reservation set contains the bytes
being written. If the SC.W succeeds, the instruction writes the word
in rs2 to memory, and it writes zero to rd. If the SC.W fails, the
instruction does not write to memory, and it writes a nonzero value to
rd. Regardless of success or failure, executing an SC.W instruction
*invalidates any reservation held by this hart*.

More details, ref:
https://github.com/riscv/riscv-isa-manual

I think section "3.5.3.2 Reservability PMA" [1] would be a more relevant
link, as this defines memory areas that either do or do not have
forward progress guarantees, including this part:

   "When LR/SC is used for memory locations marked RsrvNonEventual,
     software should provide alternative fall-back mechanisms used when
     lack of progress is detected."

My reading of this is that if the example you tried stalls, then either
the PMA is not RsrvEventual, and it is wrong to rely on ll/sc on this,
or that the PMA is marked RsrvEventual but the implementation is
buggy.

Yes, PMA just defines physical memory region attributes, But in our
processor, when MMU is enabled (satp's value register > 2) in s-mode,
it will look at our custom PTE's attributes BIT(63) ref [1]:

   PTE format:
   | 63 | 62 | 61 | 60 | 59 | 58-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
     SO   C    B    SH   SE    RSW   D   A   G   U   X   W   R   V
     ^    ^    ^    ^    ^
   BIT(63): SO - Strong Order
   BIT(62): C  - Cacheable
   BIT(61): B  - Bufferable
   BIT(60): SH - Shareable
   BIT(59): SE - Security

So the memory also could be RsrvNone/RsrvEventual.

[1] https://github.com/c-sky/csky-linux/commit/e837aad23148542771794d8a2fcc52afd0fcbf88

It also seems that the current "amoswap" based implementation
would be reliable independent of RsrvEventual/RsrvNonEventual.

Yes, the hardware implementation of AMO could be different from LR/SC.
AMO could use ACE snoop holding to lock the bus in hw coherency
design, but LR/SC uses an exclusive monitor without locking the bus.

arm64 is already in the situation of having to choose between
two cmpxchg() implementation at runtime to allow falling back to
a slower but more general version, but it's best to avoid that if you
can.

Current RISC-V needn't multiple versions to select, and all AMO &
LR/SC has been defined in the spec.

RISC-V hasn't CAS instructions, and it uses LR/SC for cmpxchg. I don't
think LR/SC would be slower than CAS, and CAS is just good for code
size.

         Arnd

[1] http://www.five-embeddev.com/riscv-isa-manual/latest/machine.html#atomicity-pmas

--
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/