[PATCH v13 06/15] asm-generic: barrier: Add smp_cond_load_acquire_timeout()
From: Ankur Arora <hidden>
Date: 2026-07-02 01:34:33
Also in:
bpf, linux-arch, linux-pm, lkml
Subsystem:
generic include/asm header files, the rest · Maintainers:
Arnd Bergmann, Linus Torvalds
Add the acquire variant of smp_cond_load_relaxed_timeout(). This reuses the relaxed variant, with additional LOAD->LOAD ordering via smp_acquire__after_ctrl_dep(). To ensure that the necessary control dependency on the dereference of @ptr exists (which does not in the timeout path), re-evaluate the cond_expr branch. Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Will Deacon <will@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-arch@vger.kernel.org Cc: bpf@vger.kernel.org Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Haris Okanovic <redacted> Tested-by: Haris Okanovic <redacted> Signed-off-by: Ankur Arora <redacted> --- Notes: addresses missed control dependency in the timeout case [1] by documenting that the acquire ordering is only in the success (non-timeout) case. However, we do need to re-evaluate the cond_expr branch to handle the case where we hit both the timeout and the loop condition. AFAICT, rqspinlock only cares about acquire ordering in the success case so this change should be okay. Kumar, Alexei: please let me know if otherwise. Catalin, Haris: I've retained your R-by on this. Please let me know if you aren't okay with this change. [1] https://lore.kernel.org/all/20260608082746.399BB1F00898@smtp.kernel.org/ (local) --- include/asm-generic/barrier.h | 40 +++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index d53e75a92598..5d21ad9f358f 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h@@ -356,6 +356,46 @@ do { \ }) #endif +/** + * smp_cond_load_acquire_timeout() - (Spin) wait for cond until a timeout + * expires. ACQUIRE ordering when @cond_expr is satisfied. + * @ptr: pointer to the variable to wait on. + * @cond_expr: boolean expression to wait for. + * @time_expr_ns: monotonic expression that evaluates to time in ns or, + * on failure, returns a negative value. + * @timeout_ns: timeout value in ns + * (Both of the above are assumed to be compatible with s64.) + * + * Equivalent to using smp_cond_load_acquire() on the condition variable with + * a timeout. + */ +#ifndef smp_cond_load_acquire_timeout +#define smp_cond_load_acquire_timeout(ptr, cond_expr, \ + time_expr_ns, timeout_ns) \ +({ \ + __unqual_scalar_typeof(*(ptr)) VAL; \ + VAL = smp_cond_load_relaxed_timeout(ptr, cond_expr, \ + time_expr_ns, \ + timeout_ns); \ + /* \ + * We arrive here once the loop condition is hit, on timeout, \ + * or, if we hit both the timeout and the loop condition. \ + * \ + * The last case is low probability, but possible in the last \ + * iteration, especially on architectures with waiting \ + * cpu_poll_relax() implementations (ex. arm64). \ + * Now since the loop condition is not evaluated on timeout, \ + * we have a missed control dependency. \ + * \ + * So, force a re-evaluation of the control dependency to \ + * provide an ACQUIRE ordering for that case as well. \ + */ \ + if (cond_expr) \ + smp_acquire__after_ctrl_dep(); \ + (typeof(*(ptr)))VAL; \ +}) +#endif + /* * pmem_wmb() ensures that all stores for which the modification * are written to persistent storage by preceding instructions have
--
2.43.7