[PATCH v5 12/13] rcu-tasks-trace: Assert no reader is held on return to userspace
flat view
COOLING13d
From: Josef Bacik <josef@toxicpanda.com>
Date: 2026-09-22 02:34:09
Also in:
bpf, linux-trace-kernel, lkml, rcu
Subsystem:
generic entry code, read-copy update (rcu), the rest · Maintainers:
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, "Paul E. McKenney", Frederic Weisbecker, Neeraj Upadhyay, Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki, Linus Torvalds
With trampolines now taking rcu_read_lock_trace() open-coded from assembly and JIT-emitted code, an unbalanced reader would silently turn every later Tasks Trace grace period on that task into a stall. No task can legitimately reach userspace with current->trc_reader_nesting non-zero, so under CONFIG_PROVE_RCU check it in the generic entry code's return-to-user validation, next to the existing kmap and lockdep assertions. Compiles away otherwise. Assisted-by: LLM Signed-off-by: Josef Bacik <josef@toxicpanda.com> --- include/linux/irq-entry-common.h | 2 ++ include/linux/rcupdate_trace.h | 7 +++++++ 2 files changed, 9 insertions(+)
diff --git a/include/linux/irq-entry-common.h b/include/linux/irq-entry-common.h
index e19b41ee6b18..d42568a488a7 100644
--- a/include/linux/irq-entry-common.h
+++ b/include/linux/irq-entry-common.h@@ -5,6 +5,7 @@ #include <linux/context_tracking.h> #include <linux/hrtimer_rearm.h> #include <linux/kmsan.h> +#include <linux/rcupdate_trace.h> #include <linux/rseq_entry.h> #include <linux/static_call_types.h> #include <linux/syscalls.h>
@@ -214,6 +215,7 @@ static __always_inline void __exit_to_user_mode_validate(void) { /* Ensure that kernel state is sane for a return to userspace */ kmap_assert_nomap(); + rcu_tasks_trace_assert_idle(); lockdep_assert_irqs_disabled(); lockdep_sys_exit(); }
diff --git a/include/linux/rcupdate_trace.h b/include/linux/rcupdate_trace.h
index dcdb11643496..9eb7f92c710b 100644
--- a/include/linux/rcupdate_trace.h
+++ b/include/linux/rcupdate_trace.h@@ -217,6 +217,12 @@ unsigned long rcu_tasks_trace_batches_completed(void); // Placeholders to enable stepwise transition. void __init rcu_tasks_trace_suppress_unused(void); +/* A task must never reach userspace inside an rcu_read_lock_trace() reader. */ +static inline void rcu_tasks_trace_assert_idle(void) +{ + WARN_ON_ONCE(IS_ENABLED(CONFIG_PROVE_RCU) && READ_ONCE(current->trc_reader_nesting)); +} + #else static inline unsigned long rcu_tasks_trace_batches_completed(void) { return 0; } /*
@@ -226,6 +232,7 @@ static inline unsigned long rcu_tasks_trace_batches_completed(void) { return 0; static inline void call_rcu_tasks_trace(struct rcu_head *rhp, rcu_callback_t func) { BUG(); } static inline void rcu_read_lock_trace(void) { BUG(); } static inline void rcu_read_unlock_trace(void) { BUG(); } +static inline void rcu_tasks_trace_assert_idle(void) { } #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ DEFINE_LOCK_GUARD_0(rcu_tasks_trace,
--
2.55.0