[PATCH RFC v3 12/13] rcu-tasks-trace: Assert no reader is held on return to userspace
flat view
COLD20d
From: Josef Bacik <josef@toxicpanda.com>
Date: 2026-09-15 13:18:12
Also in:
bpf, lkml, rcu, xen-devel
Subsystem:
generic entry code, read-copy update (rcu), the rest · Maintainers:
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, "Paul E. McKenney", Frederic Weisbecker, Neeraj Upadhyay, Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki, Linus Torvalds
With trampolines and their glue now taking rcu_read_lock_trace() from assembly and from several C paths, an unbalanced reader would silently turn every later Tasks Trace grace period on that task into a stall. No task can legitimately reach userspace with current->trc_reader_nesting non-zero, so under CONFIG_PROVE_RCU check it in the generic entry code's return-to-user validation, next to the existing kmap and lockdep assertions. Compiles away otherwise. Assisted-by: LLM Signed-off-by: Josef Bacik <josef@toxicpanda.com> --- include/linux/irq-entry-common.h | 2 ++ include/linux/rcupdate_trace.h | 7 +++++++ 2 files changed, 9 insertions(+)
diff --git a/include/linux/irq-entry-common.h b/include/linux/irq-entry-common.h
index b811b469b0a7..58086cc1140c 100644
--- a/include/linux/irq-entry-common.h
+++ b/include/linux/irq-entry-common.h@@ -5,6 +5,7 @@ #include <linux/context_tracking.h> #include <linux/hrtimer_rearm.h> #include <linux/kmsan.h> +#include <linux/rcupdate_trace.h> #include <linux/rseq_entry.h> #include <linux/static_call_types.h> #include <linux/syscalls.h>
@@ -214,6 +215,7 @@ static __always_inline void __exit_to_user_mode_validate(void) { /* Ensure that kernel state is sane for a return to userspace */ kmap_assert_nomap(); + rcu_tasks_trace_assert_idle(); lockdep_assert_irqs_disabled(); lockdep_sys_exit(); }
diff --git a/include/linux/rcupdate_trace.h b/include/linux/rcupdate_trace.h
index 4035054309d7..f5a51de4a4ef 100644
--- a/include/linux/rcupdate_trace.h
+++ b/include/linux/rcupdate_trace.h@@ -209,6 +209,12 @@ unsigned long rcu_tasks_trace_batches_completed(void); // Placeholders to enable stepwise transition. void __init rcu_tasks_trace_suppress_unused(void); +/* A task must never reach userspace inside an rcu_read_lock_trace() reader. */ +static inline void rcu_tasks_trace_assert_idle(void) +{ + WARN_ON_ONCE(IS_ENABLED(CONFIG_PROVE_RCU) && READ_ONCE(current->trc_reader_nesting)); +} + #else static inline unsigned long rcu_tasks_trace_batches_completed(void) { return 0; } /*
@@ -218,6 +224,7 @@ static inline unsigned long rcu_tasks_trace_batches_completed(void) { return 0; static inline void call_rcu_tasks_trace(struct rcu_head *rhp, rcu_callback_t func) { BUG(); } static inline void rcu_read_lock_trace(void) { BUG(); } static inline void rcu_read_unlock_trace(void) { BUG(); } +static inline void rcu_tasks_trace_assert_idle(void) { } #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ DEFINE_LOCK_GUARD_0(rcu_tasks_trace,
--
2.55.0