Re: [PATCH 2/2] rcutorture: Nudge ksoftirqd priority for RCU boost testing

On Wed, Aug 04, 2021 at 11:18:11AM +0100, Valentin Schneider wrote:

On 03/08/21 16:42, Paul E. McKenney wrote:

quoted

On Tue, Aug 03, 2021 at 11:54:37PM +0100, Valentin Schneider wrote:

quoted

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 680f66b65f14..3dd5fa75f469 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c

@@ -948,12 +948,26 @@ static int rcu_torture_boost(void *arg)
     unsigned long endtime;
     unsigned long oldstarttime;
     struct rcu_boost_inflight rbi = { .inflight = 0 };
+	struct task_struct *ksoftirqd = this_cpu_ksoftirqd();

     VERBOSE_TOROUT_STRING("rcu_torture_boost started");

     /* Set real-time priority. */
     sched_set_fifo_low(current);

+	/*
+	 * Boost testing requires TIMER_SOFTIRQ to run at a higher priority
+	 * than the CPU-hogging torture kthreads, otherwise said threads
+	 * will never let timer expiry for the RCU GP kthread happen, which will
+	 * prevent any boosting.
+	 */
+	if (current->normal_prio < ksoftirqd->normal_prio) {

Would it make sense to add IS_ENABLED(CONFIG_PREEMPT_RT) to the above
condition?

Hm so v5.13-rt1 has this commit:

  5e59fba573e6 ("rcutorture: Fix testing of RCU priority boosting")

which gates RCU boost torture testing under CONFIG_PREEMPT_RT. Now, AFAICT
the TIMER_SOFTIRQ priority problem is there regardless of
CONFIG_PREEMPT_RT, so this patch would (should?) make sense even on
!CONFIG_PREEMPT_RT.

What rcutorture scenario TREE03 does is to boot with tree.use_softirq=0
and threadirqs.  I see your point about timers and softirq, but this
does run reliably for me.

Ah, I see why.  Commit ea6d962e80b6 ("rcutorture: Judge RCU priority
boosting on grace periods, not callbacks") includes boosting the priority
of the ksoftirqd kthreads.  But only when running rcutorture builtin,
not as a module.  Here is the code in rcu_torture_init():

		// Testing RCU priority boosting requires rcutorture do
		// some serious abuse.  Counter this by running ksoftirqd
		// at higher priority.
		if (IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)) {
			for_each_online_cpu(cpu) {
				struct sched_param sp;
				struct task_struct *t;

				t = per_cpu(ksoftirqd, cpu);
				WARN_ON_ONCE(!t);
				sp.sched_priority = 2;
				sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
			}
		}

I take it that you were running rcutorture as a module?

This describes how to run it built-in, if that works for you:

https://paulmck.livejournal.com/61432.html

More specifically: https://paulmck.livejournal.com/57769.html

Alternatively, the "IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)" check could be
removed in the above code, and the ksoftirqd kthreads could have their
original priority restored in rcu_torture_cleanup().

Thoughts?

							Thanx, Paul

quoted

quoted

+		struct sched_param sp = { .sched_priority = 2 };
+
+		pr_alert("%s(): Adjusting %s priority\n", __func__, ksoftirqd->comm);
+		sched_setscheduler_nocheck(ksoftirqd, SCHED_FIFO, &sp);
+	}
+
     init_rcu_head_on_stack(&rbi.rcu);
     /* Each pass through the following loop does one boost-test cycle. */
     do {
--
2.25.1