On Sun, 21 Mar 2021, Jonathan Schwender wrote:

On 2/22/21 5:20 PM, Ahmed S. Darwish wrote:

quoted

On Mon, Feb 22, 2021 at 04:28:30PM +0100, Jonathan Schwender wrote:

quoted

Hi John,

This patch adds the option --mainaffinity to specify the affinity of
the main pid.
This is mainly useful if you want to bind the main thread to a
different (e.g. housekeeping ) CPU than the measurement threads.

Do you also have any numbers showing different results with/without
"--mainaffinity"?

Sorry for the delay. I do have some numbers now, and there is a benefit to
using this option if CPU isolation + CAT are used. Otherwise it's not really
visible.

Rendered Markdown:
https://gist.github.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e

# Testing proposed cyclictest --mainaffinity option

System information:
- 2 socket Intel Xeon E5-2643 v4 @ 3.40Ghz
- Turbo boost is disabled.
- Fedora 33 with kernel 5.10.1-rt20 with small patch (see cmdline isolcpus)
- Cmdline: `nosmt isolcpus=domain,managed_irq,wq,rcu,misc,kthread,3,5,7,9,11
rcu_nocbs=3,5,7,9,11 irqaffinity=0,2,4 maxcpus=12 rcu_nocb_poll nowatchdog
tsc=nowatchdog processor.max_cstate=1 intel_idle.max_cstate=0
systemd.unified_cgroup_hierarchy=0`
    - The additional isolcpus arguments set the HK_FLAG with the corresponding
    name.
      This cmdline adds all HK_FLAGs usually set by nohz_full, except the
      actual
      nohz flags `tick` and `timer`. This improves cyclictest latencies on my
      system.
- Rteval is running on all CPUs from node 0 + CPU 1, but not on the isolated
CPUs.
- L3 Cache is reserved for the isolated CPUs via `resctrl` (CPU based
allocation)
- Test duration 24 hours, interval 200 µs


## Test 1: 5 cyclictest instance with main pid on same cpu as the measurement
thread

This test simply starts 5 cyclictest instances (via numactl) with one
measurement thread each and bound to a single
CPU via `--affinity`, so that the main thread is also bound to the same CPU.

![Figure: 5 cyclictest instances with main pid pinned to same CPU as
measurement
thread](https://gist.githubusercontent.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e/raw/e27c3f284cf4bbeecded84865dfee5676b47fe88/2021-03-11.png)

## Test 2: Single cyclictest instance with --mainaffinity=1 for isolated CPUs
3,5,7,9,11
The main thread was placed on CPU 1 via `--mainaffinity` and
`--refresh_on_max` was added for good measure to keep the
logfile small.

![Figure: Single cyclictest instance with --mainaffinity=1 for isolated CPUs
3,5,7,9,11](https://gist.githubusercontent.com/jschwe/d4c46026aec57b10a2b0e6f72258b96e/raw/afd81b2a70a3e88bdebc46615d0f60e24238b405/2021-03-19.png)

quoted

Thanks,

--
Ahmed S. Darwish

Jonathan

Alright, I can imagine how this could be useful. Would you respin the 
patches against the latest upstream and resend to me?

Thanks

John