Re: [RFC PATCH 00/11] rv: Add scheduler specification monitors

On Fri, 2025-02-07 at 15:27 +0100, Juri Lelli wrote:

On 07/02/25 12:36, Gabriele Monaco wrote:

quoted

On Fri, 2025-02-07 at 11:55 +0100, Juri Lelli wrote:

quoted

Hi Gabriele,

On 06/02/25 09:09, Gabriele Monaco wrote:

quoted

This patchset starts including adapted scheduler specifications
from
Daniel's task model [1].

Thanks a lot for working on this. Apart from being cool stuff
per-se,
it
means a lot personally to see Daniel's work continuing to be
developed.

quoted

As the model is fairly complicated, it is split in several
generators
and specifications. The tool used to create the model can
output a
unified model, but that would be hardly readable (9k states).

RV allows monitors to run and react concurrently. Running the
cumulative
model is equivalent to running single components using the same
reactors, with the advantage that it's easier to point out
which
specification failed in case of error.

We allow this by introducing nested monitors, in short, the
sysfs
monitor folder will contain a monitor named sched, which is
nothing
but
an empty container for other monitors. Controlling the sched
monitor
(enable, disable, set reactors) controls all nested monitors.

The task model proposed by Daniel includes 12 generators and 33
specifications. The generators are good for documentation but
are
usually implied in some specifications.
Not all monitors work out of the box, mainly because of those
reasons:
* need to distinguish if preempt disable leads to schedule
* need to distinguish if irq disable comes from an actual irq
* assumptions not always true on SMP

The original task model was designed for PREEMPT_RT and this
patchset is
only tested on an upstream kernel with full preemption enabled.

I played with your additions a bit and I was able to
enable/disable
monitors, switch reactors, etc., w/o noticing any issue.

Thanks for trying it out!

quoted

I wonder if you also had ways to test that the monitors actually
react
properly in case of erroneous conditions (so that we can see a
reactor
actually react :).

Well, in my understanding, reactors should fire if there is a
problem
either in the kernel or in the model logic.

Right. I guess I wonder if we can find a way to inject kernel
problems
somehow, so that model(s) can be further tested explicitly thus
making
us confident that they will be able to identify real problems when
they
occur.

Just for the sake of testing reactors there is already the wwnr monitor
which is intentionally broken exactly for that reason, Daniel described
a bit what scenario can trigger the error (some IRQ, so it may be
harder to see in a VM).

If we want any monitor to react, yeah that could be a bit trickier. We
could perhaps do something like livepatching/kprobes.

I'd assume we'd need some care though, since some of those monitors are
pretty basic and making them fail may cause pretty bad errors in the
kernel.
Another approach could be to just inject the tracepoint/handler call
those are all static functions but we may have some luck there and
wouldn't break the system, just trick the monitor.

But good point, I can have a look.