On Sun, Dec 06, 2020 at 12:25:45AM +0100, Thomas Gleixner wrote:

Pavel,

On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote:

quoted

So... what kind of guarantees does this aim to provide / what tasks it
is useful for?

For real time response, we have other approaches.

Depends on your requirements. Some problems are actually better solved
with busy polling. See below.

quoted

If you want to guarantee performnace of the "isolated" task... I don't
see how that works. Other tasks on the system still compete for DRAM
bandwidth, caches, etc...

Applications which want to run as undisturbed as possible. There is
quite a range of those:

  - Hardware in the loop simulation is today often done with that crude
    approach of "offlining" a CPU and then instead of playing dead
    jumping to a preloaded bare metal executable. That's a horrible hack
    and impossible to debug, but gives them the results they need to
    achieve. These applications are well optimized vs. cache and memory
    foot print, so they don't worry about these things too much and they
    surely don't run on SMI and BIOS value add inflicted machines.

    Don't even think about waiting for an interrupt to achieve what
    these folks are doing. So no, there are problems which a general
    purpose realtime OS cannot solve ever.

  - HPC computations on large data sets. While the memory foot print is
    large the access patterns are cache optimized. 

    The problem there is that any unnecessary IPI, tick interrupt or
    whatever nuisance is disturbing the carefully optimized cache usage
    and alone getting rid of the timer interrupt gained them measurable
    performance. Even very low single digit percentage of runtime saving
    is valuable for these folks because the compute time on such beasts
    is expensive.

  - Realtime guests in KVM. With posted interrupts and a fully populated
    host side page table there is no point in running host side
    interrupts or IPIs for random accounting or whatever purposes as
    they affect the latency in the guest. With all the side effects
    mitigated and a properly set up guest and host it is possible to get
    to a zero exit situation after the bootup phase which means pretty
    much matching bare metal behaviour.

    Yes, you can do that with e.g. Jailhouse as well, but you lose lots
    of the fancy things KVM provides. And people care about these not
    just because they are fancy. They care because their application
    scenario needs them.

There are more reasons why people want to be able to get as much
isolation from the OS as possible but at the same time have a sane
execution environment, debugging, performance monitoring and the OS
provided protection mechanisms instead of horrible hacks.

Isolation makes sense for a range of applications and there is no reason
why Linux should not support them. 

One good client for the task isolation is Open Data Plane. There are
even some code stubs supposed to enable isolation where needed.

quoted

If you want to guarantee performnace of the "isolated" task... I don't
see how that works. Other tasks on the system still compete for DRAM
bandwidth, caches, etc...

My experiments say that typical delay caused by dry IPI or syscall is
2000-20000 'ticks'. Typical delay caused by cache miss is 3-30 ticks.

To guarantee cache / memory bandwidth, one can use resctrl. Linux has
implementation of it for x86 only, but arm64 has support for for
resctrl on CPU side.

Thanks,
Yury

Thanks,

        tglx