Re: [PATCH] kprobes: Make optprobe optimizer multi-generational and asynchronous
flat view
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Date: 2026-09-26 02:17:36
Also in:
bpf, linux-trace-kernel, lkml, rcu
On Thu, 24 Sep 2026 16:17:38 +0000 (UTC) bot+bpf-ci@kernel.org wrote:
quoted
diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 6337da5cab9e..1e25980303c4 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c[ ... ]quoted
@@ -530,50 +553,78 @@ static DECLARE_COMPLETION(optimizer_completion); #define OPTIMIZE_DELAY 5 +static bool optprobe_has_queued_probes(void) +{ + struct optprobe_generation *gen = &optprobe_gens[optprobe_cur_gen]; + + return !list_empty(&gen->optimizing_list) || + !list_empty(&gen->unoptimizing_list); +}Does this logic need to check gen->freeing_list as well?
Ah, indeed. It should check the freeing_list too. [ ... ]
quoted
@@ -622,67 +675,119 @@ static void do_free_cleaned_kprobes(void) static void kick_kprobe_optimizer(void); +static void optprobe_generation_rcu_cb(struct rcu_head *rcu) +{ + struct optprobe_generation *gen; + + gen = container_of(rcu, struct optprobe_generation, rcu); + WRITE_ONCE(gen->ready, true); + wake_up(&kprobe_optimizer_wait); +} + +static void optprobe_finalize_generation(struct optprobe_generation *gen) +{ + lockdep_assert_held(&kprobe_mutex); + + scoped_guard(cpus_read_lock) { + guard(mutex)(&text_mutex); + + /* Optimization never be done when disarmed */ + if (!kprobes_all_disarmed && kprobes_allow_optimization && + !list_empty(&gen->optimizing_list)) + arch_optimize_kprobes(&gen->optimizing_list); + } + + /* Free cleaned kprobes after quiescence period */ + do_free_cleaned_kprobes(&gen->freeing_list); + + gen->in_flight = false; + WRITE_ONCE(gen->ready, false); +} + +static void optprobe_dispatch_generation(void) +{ + struct optprobe_generation *gen; + + lockdep_assert_held(&kprobe_mutex); + + if (!optprobe_can_fire() || !optprobe_has_queued_probes()) + return; + + gen = &optprobe_gens[optprobe_cur_gen]; + + scoped_guard(cpus_read_lock) { + guard(mutex)(&text_mutex); + + /* + * Unoptimize kprobes and collect cleaned (unused and disarmed) + * kprobes before waiting for quiescence period. + */ + do_unoptimize_kprobes(&gen->unoptimizing_list, &gen->freeing_list); + } + + /* Advance cur_gen to the next generation slot */ + optprobe_cur_gen = (optprobe_cur_gen + 1) % OPTPROBE_GEN_MAX; + + gen->in_flight = true; + WRITE_ONCE(gen->ready, false); + + call_rcu_tasks(&gen->rcu, optprobe_generation_rcu_cb); +}Now that kprobe_mutex is dropped during the Tasks RCU grace period, can an aggregator probe that was unoptimized but is still in use be freed while a task is still preempted inside its detour buffer?
Good catch! Yes, we need to keep tracking the unoptimized but not freed kprobes until finalize it. [ ... ]
quoted
@@ -707,12 +812,11 @@ static void wait_for_kprobe_optimizer_locked(void) { lockdep_assert_held(&kprobe_mutex); - while (!list_empty(&optimizing_list) || !list_empty(&unoptimizing_list)) { + while (optprobe_optimizer_busy()) { init_completion(&optimizer_completion); - /* - * Set state to OPTIMIZER_ST_FLUSHING and wake up the thread if it's - * idle. If it's already kicked, it will see the state change. - */ + optprobe_flush_requested = true; + + /* Wake up optimizer thread */ if (atomic_xchg_acquire(&optimizer_state, OPTIMIZER_ST_FLUSHING) != OPTIMIZER_ST_FLUSHING) wake_up(&kprobe_optimizer_wait);Andrea Parri raised a concern on 2026-09-24 about a race condition in this code (https://lore.kernel.org/all/arTuBodyGqQ0mErI@andreayoga.localdomain/ (local)).
Yes, so I pick his patch at first and rebase this on it. Thanks, -- Masami Hiramatsu (Google) [off-list ref]