[btrfs] 53c5e5a1d8: fsmark.files_per_sec -14.4% regression
From: kernel test robot <hidden>
Date: 2021-01-13 01:52:39
Also in:
oe-lkp
Greeting,
FYI, we noticed a -14.4% regression of fsmark.files_per_sec due to commit:
commit: 53c5e5a1d87e3341d06a00746c772dc7eff6a65b ("btrfs: simplify the logic in need_preemptive_flushing")
https://github.com/kdave/btrfs-devel.git for-next-20210104
in testcase: fsmark
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
iterations: 1x
nr_threads: 1t
disk: 1BRD_48G
fs: btrfs
filesize: 4M
test_size: 24G
sync_method: fsyncBeforeClose
cpufreq_governor: performance
ucode: 0x5003003
test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload.
test-url: https://sourceforge.net/projects/fsmark/
In addition to that, the commit also has significant impact on the following tests:
+------------------+------------------------------------------------------------------------+
| testcase: change | fio-basic: fio.write_iops -59.9% regression |
| test machine | 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory |
| test parameters | bs=4k |
| | cpufreq_governor=performance |
| | disk=1SSD |
| | fs=btrfs |
| | ioengine=sync |
| | nr_task=8 |
| | runtime=300s |
| | rw=write |
| | test_size=256g |
| | ucode=0x4003003 |
+------------------+------------------------------------------------------------------------+
| testcase: change | aim7: aim7.jobs-per-min -3.5% regression |
| test machine | 144 threads Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory |
| test parameters | cpufreq_governor=performance |
| | disk=1BRD_48G |
| | fs=btrfs |
| | load=4 |
| | test=creat-clo |
| | ucode=0x700001e |
+------------------+------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <redacted>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode:
gcc-9/performance/1BRD_48G/4M/btrfs/1x/x86_64-rhel-8.3/1t/debian-10.4-x86_64-20200603.cgz/fsyncBeforeClose/lkp-csl-2ap2/24G/fsmark/0x5003003
commit:
5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size")
53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing")
5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
0:4 1% 0:4 perf-profile.children.cycles-pp.error_entry
0:4 1% 0:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
239.48 -14.4% 205.05 fsmark.files_per_sec
27.49 +15.7% 31.79 fsmark.time.elapsed_time
27.49 +15.7% 31.79 fsmark.time.elapsed_time.max
26.81 +15.9% 31.08 fsmark.time.system_time
0.21 ± 3% -0.0 0.18 mpstat.cpu.all.usr%
1922 ± 11% +23.6% 2377 ± 7% sched_debug.cpu.avg_idle.min
836321 -12.2% 734615 vmstat.io.bo
1840 ± 9% -25.0% 1379 ± 17% meminfo.Writeback
1813949 ± 2% -12.0% 1595838 meminfo.max_used_kB
2706 ± 48% +221.4% 8700 ± 71% softirqs.NET_RX
1101600 +10.6% 1218682 softirqs.SCHED
6106677 ± 99% -99.8% 14876 ± 35% numa-numastat.node2.local_node
6193240 ± 98% -98.4% 101442 ± 5% numa-numastat.node2.numa_hit
346322 ± 75% -81.2% 64945 ± 57% numa-numastat.node3.other_node
4.651e+09 ± 23% +29.3% 6.013e+09 cpuidle.C1E.time
10275557 ± 15% +24.2% 12766663 cpuidle.C1E.usage
37470 ± 5% +34.0% 50206 ± 24% cpuidle.POLL.time
9995 +36.8% 13675 ± 9% cpuidle.POLL.usage
486.00 ±170% +117.8% 1058 ± 91% numa-meminfo.node0.Dirty
3199694 ±158% +97.8% 6328688 ± 94% numa-meminfo.node0.FilePages
2886568 ±173% +107.5% 5989331 ± 98% numa-meminfo.node0.Inactive(file)
5952654 ± 96% -98.3% 99890 ± 49% numa-meminfo.node2.Inactive
13431706 ± 92% -93.0% 936825 ± 12% numa-meminfo.node2.MemUsed
65018 ± 45% +153.4% 164769 ± 58% numa-meminfo.node3.AnonPages
65783 ± 44% +156.2% 168505 ± 57% numa-meminfo.node3.Inactive(anon)
6776 ± 5% +23.7% 8383 ± 14% numa-meminfo.node3.KernelStack
1129 ± 62% +387.2% 5504 ± 68% numa-meminfo.node3.PageTables
107132 ± 3% -5.8% 100954 ± 2% proc-vmstat.nr_anon_pages
109883 ± 3% -5.7% 103640 ± 2% proc-vmstat.nr_inactive_anon
425.25 ± 16% -21.4% 334.25 ± 13% proc-vmstat.nr_writeback
109883 ± 3% -5.7% 103640 ± 2% proc-vmstat.nr_zone_inactive_anon
593623 ± 3% +16.5% 691600 ± 8% proc-vmstat.numa_foreign
593623 ± 3% +16.5% 691600 ± 8% proc-vmstat.numa_miss
853428 ± 2% +11.5% 951407 ± 5% proc-vmstat.numa_other
150853 ± 5% +9.8% 165681 ± 2% proc-vmstat.pgfault
7385 ± 2% +13.0% 8347 proc-vmstat.pgreuse
796087 ±157% +98.2% 1577690 ± 94% numa-vmstat.node0.nr_file_pages
717788 ±173% +108.0% 1492825 ± 98% numa-vmstat.node0.nr_inactive_file
2259 ± 24% -20.2% 1803 ± 6% numa-vmstat.node0.nr_mapped
717788 ±173% +108.0% 1492819 ± 98% numa-vmstat.node0.nr_zone_inactive_file
15938 ± 26% -21.1% 12572 ± 21% numa-vmstat.node1.nr_slab_unreclaimable
16329 ± 44% +150.8% 40954 ± 58% numa-vmstat.node3.nr_anon_pages
16522 ± 44% +152.7% 41751 ± 58% numa-vmstat.node3.nr_inactive_anon
6772 ± 5% +23.7% 8374 ± 14% numa-vmstat.node3.nr_kernel_stack
280.75 ± 63% +388.2% 1370 ± 68% numa-vmstat.node3.nr_page_table_pages
16521 ± 44% +152.7% 41751 ± 58% numa-vmstat.node3.nr_zone_inactive_anon
499113 ± 5% +689.0% 3938239 ± 84% numa-vmstat.node3.numa_hit
384758 ± 12% +892.7% 3819327 ± 86% numa-vmstat.node3.numa_local
11.83 ± 2% -11.7% 10.44 perf-stat.i.MPKI
1.272e+09 +3.3% 1.314e+09 perf-stat.i.branch-instructions
1.78 -0.2 1.54 perf-stat.i.branch-miss-rate%
27489113 ± 3% -10.0% 24734118 perf-stat.i.branch-misses
69581732 ± 2% -7.2% 64558270 perf-stat.i.cache-references
2.57 ± 2% -9.6% 2.32 perf-stat.i.cpi
1.598e+10 ± 2% -7.1% 1.484e+10 perf-stat.i.cpu-cycles
251.20 -2.8% 244.05 perf-stat.i.cpu-migrations
7.777e+08 -5.2% 7.369e+08 perf-stat.i.dTLB-stores
56.13 ± 2% -1.5 54.62 perf-stat.i.iTLB-load-miss-rate%
5000319 ± 4% -4.6% 4768521 perf-stat.i.iTLB-load-misses
3899306 +1.6% 3960144 perf-stat.i.iTLB-loads
1311 ± 3% +6.6% 1397 perf-stat.i.instructions-per-iTLB-miss
0.41 +9.7% 0.44 perf-stat.i.ipc
7.20 ± 3% -13.5% 6.22 ± 5% perf-stat.i.major-faults
0.08 ± 2% -7.1% 0.08 perf-stat.i.metric.GHz
1297657 ± 3% -5.6% 1224413 ± 2% perf-stat.i.node-load-misses
10.63 -8.9% 9.69 perf-stat.overall.MPKI
2.16 ± 2% -0.3 1.88 perf-stat.overall.branch-miss-rate%
2.44 -8.8% 2.22 perf-stat.overall.cpi
56.15 ± 2% -1.5 54.63 perf-stat.overall.iTLB-load-miss-rate%
1313 ± 3% +6.4% 1397 perf-stat.overall.instructions-per-iTLB-miss
0.41 +9.6% 0.45 perf-stat.overall.ipc
1.226e+09 +3.5% 1.269e+09 perf-stat.ps.branch-instructions
26520603 ± 2% -10.1% 23847387 perf-stat.ps.branch-misses
67153636 ± 2% -6.9% 62502238 perf-stat.ps.cache-references
1.541e+10 ± 2% -6.8% 1.435e+10 perf-stat.ps.cpu-cycles
242.48 -2.7% 235.91 perf-stat.ps.cpu-migrations
7.492e+08 -4.9% 7.123e+08 perf-stat.ps.dTLB-stores
4819104 ± 4% -4.2% 4617865 perf-stat.ps.iTLB-load-misses
3758462 +2.0% 3835078 perf-stat.ps.iTLB-loads
6.317e+09 +2.1% 6.452e+09 perf-stat.ps.instructions
7.02 ± 3% -14.5% 6.00 perf-stat.ps.major-faults
1.807e+11 +16.6% 2.108e+11 perf-stat.total.instructions
0.86 ± 6% -0.1 0.75 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.82 ± 6% -0.1 0.71 ± 13% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.79 ± 6% +0.1 0.86 ± 2% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
0.00 +0.9 0.85 ± 15% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata
5.52 ± 10% +1.9 7.38 ± 13% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
5.50 ± 10% +1.9 7.36 ± 12% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write
5.60 ± 10% +1.9 7.47 ± 13% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.58 ± 10% +1.9 7.45 ± 13% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.41 ± 8% +2.0 9.40 ± 8% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +2.1 2.12 ± 12% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write
0.00 +2.2 2.17 ± 12% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
0.00 +2.2 2.17 ± 12% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.00 +2.2 2.23 ± 12% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
0.88 ± 7% -0.1 0.77 ± 14% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.52 ± 14% -0.1 0.41 ± 15% perf-profile.children.cycles-pp.end_page_writeback
0.46 ± 13% -0.1 0.35 ± 14% perf-profile.children.cycles-pp.test_clear_page_writeback
0.24 ± 8% -0.0 0.19 ± 9% perf-profile.children.cycles-pp.__clear_extent_bit
0.24 ± 8% -0.0 0.19 ± 9% perf-profile.children.cycles-pp.clear_extent_bit
0.16 ± 5% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.alloc_extent_state
0.14 ± 6% +0.0 0.18 ± 21% perf-profile.children.cycles-pp.__schedule
0.31 ± 6% +0.0 0.35 ± 7% perf-profile.children.cycles-pp.update_rq_clock
0.04 ± 59% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush
0.04 ± 60% +0.1 0.10 ± 17% perf-profile.children.cycles-pp.btrfs_alloc_tree_block
0.21 ± 4% +0.1 0.28 perf-profile.children.cycles-pp.arch_scale_freq_tick
0.18 ± 10% +0.1 0.26 ± 15% perf-profile.children.cycles-pp.btrfs_search_slot
0.12 ± 19% +0.1 0.20 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block
0.12 ± 19% +0.1 0.20 ± 15% perf-profile.children.cycles-pp.btrfs_cow_block
0.83 ± 3% +0.1 0.93 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +0.1 0.14 ± 17% perf-profile.children.cycles-pp.find_next_bit
0.12 ± 16% +0.6 0.67 ± 15% perf-profile.children.cycles-pp._find_next_bit
0.00 +1.0 0.99 ± 15% perf-profile.children.cycles-pp.cpumask_next
5.52 ± 10% +1.9 7.38 ± 13% perf-profile.children.cycles-pp.btrfs_file_write_iter
5.50 ± 10% +1.9 7.36 ± 13% perf-profile.children.cycles-pp.btrfs_buffered_write
7.49 ± 8% +2.0 9.48 ± 8% perf-profile.children.cycles-pp.ksys_write
7.42 ± 8% +2.0 9.41 ± 8% perf-profile.children.cycles-pp.new_sync_write
7.46 ± 8% +2.0 9.46 ± 8% perf-profile.children.cycles-pp.vfs_write
0.11 ± 12% +2.1 2.23 ± 12% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.06 ± 11% +2.2 2.25 ± 12% perf-profile.children.cycles-pp.__reserve_bytes
0.00 +2.2 2.19 ± 12% perf-profile.children.cycles-pp.__percpu_counter_sum
0.03 ±100% +2.2 2.24 ± 12% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
0.21 ± 4% +0.1 0.28 perf-profile.self.cycles-pp.arch_scale_freq_tick
0.83 ± 3% +0.1 0.92 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.00 +0.1 0.14 ± 17% perf-profile.self.cycles-pp.find_next_bit
0.00 +0.3 0.30 ± 15% perf-profile.self.cycles-pp.cpumask_next
0.11 ± 14% +0.5 0.66 ± 16% perf-profile.self.cycles-pp._find_next_bit
0.00 +1.2 1.17 ± 10% perf-profile.self.cycles-pp.__percpu_counter_sum
2506 ± 18% -29.0% 1779 ± 19% interrupts.CPU0.CAL:Function_call_interrupts
57908 +15.2% 66697 interrupts.CPU0.LOC:Local_timer_interrupts
58115 +14.8% 66696 interrupts.CPU1.LOC:Local_timer_interrupts
58148 +14.7% 66677 interrupts.CPU10.LOC:Local_timer_interrupts
58110 +14.7% 66638 interrupts.CPU100.LOC:Local_timer_interrupts
58085 +14.7% 66623 interrupts.CPU101.LOC:Local_timer_interrupts
58066 +14.8% 66634 interrupts.CPU102.LOC:Local_timer_interrupts
57776 +15.4% 66656 interrupts.CPU103.LOC:Local_timer_interrupts
58053 +14.8% 66625 interrupts.CPU104.LOC:Local_timer_interrupts
58042 +14.5% 66472 interrupts.CPU105.LOC:Local_timer_interrupts
58036 +14.8% 66631 interrupts.CPU106.LOC:Local_timer_interrupts
58034 +14.8% 66597 interrupts.CPU107.LOC:Local_timer_interrupts
57578 ± 2% +15.7% 66607 interrupts.CPU108.LOC:Local_timer_interrupts
58018 +14.8% 66613 interrupts.CPU109.LOC:Local_timer_interrupts
58074 +14.7% 66640 interrupts.CPU11.LOC:Local_timer_interrupts
58051 +14.7% 66592 interrupts.CPU110.LOC:Local_timer_interrupts
58069 +14.8% 66653 interrupts.CPU111.LOC:Local_timer_interrupts
58050 +14.8% 66617 interrupts.CPU112.LOC:Local_timer_interrupts
58054 +14.8% 66638 interrupts.CPU113.LOC:Local_timer_interrupts
58097 +14.7% 66635 interrupts.CPU114.LOC:Local_timer_interrupts
58048 +14.7% 66602 interrupts.CPU115.LOC:Local_timer_interrupts
232.50 ± 20% -31.2% 160.00 ± 34% interrupts.CPU115.NMI:Non-maskable_interrupts
232.50 ± 20% -31.2% 160.00 ± 34% interrupts.CPU115.PMI:Performance_monitoring_interrupts
58068 +14.7% 66620 interrupts.CPU116.LOC:Local_timer_interrupts
58073 +14.7% 66606 interrupts.CPU117.LOC:Local_timer_interrupts
58073 +14.7% 66601 interrupts.CPU118.LOC:Local_timer_interrupts
57930 +15.0% 66637 interrupts.CPU119.LOC:Local_timer_interrupts
226.25 ± 21% -42.4% 130.25 ± 14% interrupts.CPU119.NMI:Non-maskable_interrupts
226.25 ± 21% -42.4% 130.25 ± 14% interrupts.CPU119.PMI:Performance_monitoring_interrupts
58108 +14.8% 66699 interrupts.CPU12.LOC:Local_timer_interrupts
58024 +14.8% 66604 interrupts.CPU120.LOC:Local_timer_interrupts
58007 +14.5% 66430 interrupts.CPU121.LOC:Local_timer_interrupts
58028 +14.8% 66625 interrupts.CPU122.LOC:Local_timer_interrupts
58032 +14.7% 66585 interrupts.CPU123.LOC:Local_timer_interrupts
58036 +14.8% 66606 interrupts.CPU124.LOC:Local_timer_interrupts
58033 +14.8% 66615 interrupts.CPU125.LOC:Local_timer_interrupts
58024 +14.8% 66627 interrupts.CPU126.LOC:Local_timer_interrupts
58015 +14.8% 66593 interrupts.CPU127.LOC:Local_timer_interrupts
58011 +14.8% 66598 interrupts.CPU128.LOC:Local_timer_interrupts
58016 +14.8% 66594 interrupts.CPU129.LOC:Local_timer_interrupts
58089 +14.9% 66753 interrupts.CPU13.LOC:Local_timer_interrupts
58005 +14.8% 66591 interrupts.CPU130.LOC:Local_timer_interrupts
58016 +14.8% 66623 interrupts.CPU131.LOC:Local_timer_interrupts
57791 +15.3% 66640 interrupts.CPU132.LOC:Local_timer_interrupts
58027 +14.7% 66578 interrupts.CPU133.LOC:Local_timer_interrupts
58043 +14.7% 66594 interrupts.CPU134.LOC:Local_timer_interrupts
58005 +14.8% 66600 interrupts.CPU135.LOC:Local_timer_interrupts
58003 +14.8% 66594 interrupts.CPU136.LOC:Local_timer_interrupts
58019 +14.8% 66590 interrupts.CPU137.LOC:Local_timer_interrupts
58044 +14.7% 66588 interrupts.CPU138.LOC:Local_timer_interrupts
58036 +14.7% 66575 interrupts.CPU139.LOC:Local_timer_interrupts
58111 +14.7% 66629 interrupts.CPU14.LOC:Local_timer_interrupts
58032 +14.7% 66581 interrupts.CPU140.LOC:Local_timer_interrupts
58026 +14.8% 66593 interrupts.CPU141.LOC:Local_timer_interrupts
58003 +14.8% 66605 interrupts.CPU142.LOC:Local_timer_interrupts
57998 +14.9% 66617 interrupts.CPU143.LOC:Local_timer_interrupts
57886 +15.0% 66570 interrupts.CPU144.LOC:Local_timer_interrupts
918.25 ± 88% -85.3% 135.25 ± 10% interrupts.CPU144.NMI:Non-maskable_interrupts
918.25 ± 88% -85.3% 135.25 ± 10% interrupts.CPU144.PMI:Performance_monitoring_interrupts
58017 +14.8% 66607 interrupts.CPU145.LOC:Local_timer_interrupts
58017 +14.8% 66597 interrupts.CPU146.LOC:Local_timer_interrupts
58016 +14.8% 66583 interrupts.CPU147.LOC:Local_timer_interrupts
57938 +14.9% 66573 interrupts.CPU148.LOC:Local_timer_interrupts
58031 +14.8% 66600 interrupts.CPU149.LOC:Local_timer_interrupts
58103 +14.7% 66639 interrupts.CPU15.LOC:Local_timer_interrupts
58004 +14.8% 66581 interrupts.CPU150.LOC:Local_timer_interrupts
58000 +14.7% 66527 interrupts.CPU151.LOC:Local_timer_interrupts
57945 +14.9% 66586 interrupts.CPU152.LOC:Local_timer_interrupts
58027 +14.7% 66574 interrupts.CPU153.LOC:Local_timer_interrupts
58036 +14.7% 66567 interrupts.CPU154.LOC:Local_timer_interrupts
58031 +14.8% 66597 interrupts.CPU155.LOC:Local_timer_interrupts
58012 +14.8% 66582 interrupts.CPU156.LOC:Local_timer_interrupts
58009 +14.8% 66599 interrupts.CPU157.LOC:Local_timer_interrupts
58069 +14.7% 66584 interrupts.CPU158.LOC:Local_timer_interrupts
57997 +14.8% 66596 interrupts.CPU159.LOC:Local_timer_interrupts
58081 +14.7% 66633 interrupts.CPU16.LOC:Local_timer_interrupts
58014 +14.8% 66583 interrupts.CPU160.LOC:Local_timer_interrupts
58016 +14.9% 66633 interrupts.CPU161.LOC:Local_timer_interrupts
57998 +14.8% 66579 interrupts.CPU162.LOC:Local_timer_interrupts
58019 +14.7% 66563 interrupts.CPU163.LOC:Local_timer_interrupts
57996 +14.8% 66558 interrupts.CPU164.LOC:Local_timer_interrupts
58019 +14.7% 66570 interrupts.CPU165.LOC:Local_timer_interrupts
58041 +14.7% 66581 interrupts.CPU166.LOC:Local_timer_interrupts
57854 +15.1% 66600 interrupts.CPU167.LOC:Local_timer_interrupts
57991 +14.8% 66594 interrupts.CPU168.LOC:Local_timer_interrupts
58009 +14.8% 66610 interrupts.CPU169.LOC:Local_timer_interrupts
58089 +14.7% 66643 interrupts.CPU17.LOC:Local_timer_interrupts
58035 +14.8% 66595 interrupts.CPU170.LOC:Local_timer_interrupts
57732 +15.4% 66622 interrupts.CPU171.LOC:Local_timer_interrupts
58030 +14.8% 66599 interrupts.CPU172.LOC:Local_timer_interrupts
58019 +13.7% 65945 interrupts.CPU173.LOC:Local_timer_interrupts
180.00 ± 26% +225.0% 585.00 ± 56% interrupts.CPU173.NMI:Non-maskable_interrupts
180.00 ± 26% +225.0% 585.00 ± 56% interrupts.CPU173.PMI:Performance_monitoring_interrupts
58013 +14.5% 66447 interrupts.CPU174.LOC:Local_timer_interrupts
58015 +14.8% 66594 interrupts.CPU175.LOC:Local_timer_interrupts
58014 +14.8% 66592 interrupts.CPU176.LOC:Local_timer_interrupts
58033 +14.7% 66588 interrupts.CPU177.LOC:Local_timer_interrupts
57906 +15.0% 66602 interrupts.CPU178.LOC:Local_timer_interrupts
58020 +14.8% 66636 interrupts.CPU179.LOC:Local_timer_interrupts
58075 +14.8% 66648 interrupts.CPU18.LOC:Local_timer_interrupts
58017 +14.8% 66587 interrupts.CPU180.LOC:Local_timer_interrupts
58019 +14.8% 66622 interrupts.CPU181.LOC:Local_timer_interrupts
58042 +14.7% 66586 interrupts.CPU182.LOC:Local_timer_interrupts
58006 +14.8% 66582 interrupts.CPU183.LOC:Local_timer_interrupts
57993 +14.8% 66583 interrupts.CPU184.LOC:Local_timer_interrupts
58008 +14.8% 66583 interrupts.CPU185.LOC:Local_timer_interrupts
58018 +14.8% 66592 interrupts.CPU186.LOC:Local_timer_interrupts
57998 +14.7% 66524 interrupts.CPU187.LOC:Local_timer_interrupts
58005 +14.8% 66581 interrupts.CPU188.LOC:Local_timer_interrupts
58010 +14.6% 66507 interrupts.CPU189.LOC:Local_timer_interrupts
58109 +14.7% 66660 interrupts.CPU19.LOC:Local_timer_interrupts
58006 +14.8% 66580 interrupts.CPU190.LOC:Local_timer_interrupts
58048 +14.7% 66595 interrupts.CPU191.LOC:Local_timer_interrupts
57029 ± 3% +16.1% 66183 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts
58085 +14.7% 66645 interrupts.CPU20.LOC:Local_timer_interrupts
58075 +14.8% 66680 interrupts.CPU21.LOC:Local_timer_interrupts
58095 +14.7% 66656 interrupts.CPU22.LOC:Local_timer_interrupts
58050 +14.8% 66636 interrupts.CPU23.LOC:Local_timer_interrupts
58056 +14.8% 66641 interrupts.CPU24.LOC:Local_timer_interrupts
58014 +14.9% 66641 interrupts.CPU25.LOC:Local_timer_interrupts
58088 +14.7% 66646 interrupts.CPU26.LOC:Local_timer_interrupts
1797 ± 8% +33.5% 2400 ± 35% interrupts.CPU27.CAL:Function_call_interrupts
58030 +13.1% 65636 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts
58061 +14.7% 66620 interrupts.CPU28.LOC:Local_timer_interrupts
58048 +14.8% 66621 interrupts.CPU29.LOC:Local_timer_interrupts
58023 +14.8% 66631 interrupts.CPU3.LOC:Local_timer_interrupts
58074 +14.7% 66614 interrupts.CPU30.LOC:Local_timer_interrupts
58100 +14.7% 66620 interrupts.CPU31.LOC:Local_timer_interrupts
58061 +14.7% 66615 interrupts.CPU32.LOC:Local_timer_interrupts
58096 +14.2% 66368 interrupts.CPU33.LOC:Local_timer_interrupts
58066 +14.7% 66616 interrupts.CPU34.LOC:Local_timer_interrupts
58066 +14.7% 66626 interrupts.CPU35.LOC:Local_timer_interrupts
58025 +14.7% 66580 interrupts.CPU36.LOC:Local_timer_interrupts
58036 +14.8% 66616 interrupts.CPU37.LOC:Local_timer_interrupts
58072 +14.7% 66612 interrupts.CPU38.LOC:Local_timer_interrupts
58052 +14.7% 66610 interrupts.CPU39.LOC:Local_timer_interrupts
58094 +14.8% 66666 interrupts.CPU4.LOC:Local_timer_interrupts
58063 +14.7% 66614 interrupts.CPU40.LOC:Local_timer_interrupts
58052 +14.8% 66627 interrupts.CPU41.LOC:Local_timer_interrupts
58035 +14.8% 66616 interrupts.CPU42.LOC:Local_timer_interrupts
58048 +14.8% 66636 interrupts.CPU43.LOC:Local_timer_interrupts
58080 +14.7% 66631 interrupts.CPU44.LOC:Local_timer_interrupts
58050 +14.8% 66625 interrupts.CPU45.LOC:Local_timer_interrupts
58060 +14.7% 66621 interrupts.CPU46.LOC:Local_timer_interrupts
58072 +14.7% 66631 interrupts.CPU47.LOC:Local_timer_interrupts
58005 +14.6% 66448 interrupts.CPU48.LOC:Local_timer_interrupts
58069 +14.7% 66614 interrupts.CPU49.LOC:Local_timer_interrupts
58103 +14.7% 66625 interrupts.CPU5.LOC:Local_timer_interrupts
58049 +14.8% 66613 interrupts.CPU50.LOC:Local_timer_interrupts
58056 +14.7% 66615 interrupts.CPU51.LOC:Local_timer_interrupts
57464 ± 2% +15.9% 66581 interrupts.CPU52.LOC:Local_timer_interrupts
58050 +14.7% 66598 interrupts.CPU53.LOC:Local_timer_interrupts
58046 +14.7% 66584 interrupts.CPU54.LOC:Local_timer_interrupts
58080 +14.0% 66219 interrupts.CPU55.LOC:Local_timer_interrupts
57786 +15.2% 66582 interrupts.CPU56.LOC:Local_timer_interrupts
58058 +14.7% 66596 interrupts.CPU57.LOC:Local_timer_interrupts
58066 +14.5% 66482 interrupts.CPU58.LOC:Local_timer_interrupts
58044 +14.7% 66581 interrupts.CPU59.LOC:Local_timer_interrupts
58093 +14.7% 66625 interrupts.CPU6.LOC:Local_timer_interrupts
58038 +14.8% 66609 interrupts.CPU60.LOC:Local_timer_interrupts
58033 +14.7% 66590 interrupts.CPU61.LOC:Local_timer_interrupts
58015 +14.8% 66605 interrupts.CPU62.LOC:Local_timer_interrupts
58081 +14.7% 66604 interrupts.CPU63.LOC:Local_timer_interrupts
58048 +14.8% 66634 interrupts.CPU64.LOC:Local_timer_interrupts
58025 +14.4% 66400 interrupts.CPU65.LOC:Local_timer_interrupts
58043 +14.7% 66597 interrupts.CPU66.LOC:Local_timer_interrupts
58050 +14.7% 66585 interrupts.CPU67.LOC:Local_timer_interrupts
58083 +14.6% 66573 interrupts.CPU68.LOC:Local_timer_interrupts
58043 +14.7% 66601 interrupts.CPU69.LOC:Local_timer_interrupts
58012 +14.9% 66639 interrupts.CPU7.LOC:Local_timer_interrupts
58046 +14.7% 66580 interrupts.CPU70.LOC:Local_timer_interrupts
58040 +14.8% 66619 interrupts.CPU71.LOC:Local_timer_interrupts
57810 +15.1% 66538 interrupts.CPU72.LOC:Local_timer_interrupts
58076 +14.7% 66616 interrupts.CPU73.LOC:Local_timer_interrupts
58033 +14.8% 66636 interrupts.CPU74.LOC:Local_timer_interrupts
57667 +15.5% 66598 interrupts.CPU75.LOC:Local_timer_interrupts
58034 +14.8% 66622 interrupts.CPU76.LOC:Local_timer_interrupts
58045 +14.7% 66568 interrupts.CPU77.LOC:Local_timer_interrupts
176.50 ± 25% +328.0% 755.50 ± 91% interrupts.CPU77.NMI:Non-maskable_interrupts
176.50 ± 25% +328.0% 755.50 ± 91% interrupts.CPU77.PMI:Performance_monitoring_interrupts
58031 +13.8% 66059 interrupts.CPU78.LOC:Local_timer_interrupts
58026 +14.8% 66603 interrupts.CPU79.LOC:Local_timer_interrupts
58056 +14.7% 66597 interrupts.CPU8.LOC:Local_timer_interrupts
58027 +14.3% 66306 interrupts.CPU80.LOC:Local_timer_interrupts
58030 +14.7% 66538 interrupts.CPU81.LOC:Local_timer_interrupts
57995 +14.9% 66616 interrupts.CPU82.LOC:Local_timer_interrupts
58047 +14.7% 66558 interrupts.CPU83.LOC:Local_timer_interrupts
58036 +14.8% 66608 interrupts.CPU84.LOC:Local_timer_interrupts
58038 +14.8% 66607 interrupts.CPU85.LOC:Local_timer_interrupts
58032 +14.8% 66609 interrupts.CPU86.LOC:Local_timer_interrupts
57907 +15.1% 66632 interrupts.CPU87.LOC:Local_timer_interrupts
58018 +14.8% 66621 interrupts.CPU88.LOC:Local_timer_interrupts
58029 +14.5% 66471 interrupts.CPU89.LOC:Local_timer_interrupts
58101 +14.6% 66596 interrupts.CPU9.LOC:Local_timer_interrupts
58065 +14.7% 66609 interrupts.CPU90.LOC:Local_timer_interrupts
58050 +14.5% 66495 interrupts.CPU91.LOC:Local_timer_interrupts
58057 +14.7% 66612 interrupts.CPU92.LOC:Local_timer_interrupts
58026 +14.8% 66612 interrupts.CPU93.LOC:Local_timer_interrupts
58044 +14.8% 66610 interrupts.CPU94.LOC:Local_timer_interrupts
58046 +14.8% 66615 interrupts.CPU95.LOC:Local_timer_interrupts
58050 +14.8% 66616 interrupts.CPU96.LOC:Local_timer_interrupts
58071 +14.6% 66547 interrupts.CPU97.LOC:Local_timer_interrupts
57871 +15.1% 66606 interrupts.CPU98.LOC:Local_timer_interrupts
58066 +14.8% 66642 interrupts.CPU99.LOC:Local_timer_interrupts
11139415 +14.8% 12784101 interrupts.LOC:Local_timer_interrupts
fsmark.files_per_sec
250 +---------------------------------------------------------------------+
245 |-+ + + |
| + + +++. .+ .++ + ++ + +. .+ :+ .++ +. |
240 |++.+ :+ + + ++++ ++++ +++.++ ++.++ ++++ ++++ + + ++ ++|
235 |-+ + + |
| |
230 |-+ |
225 |-+ |
220 |-+ |
| |
215 |-+ |
210 |-+ OO O O |
|OO O OO OOOO O O O OO OOOO OOOOO O O |
205 |-+ O OO O O OO |
200 +---------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2ap1: 192 threads Intel(R) Xeon(R) CPU @ 2.20GHz with 192G memory
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/ucode:
4k/gcc-9/performance/1SSD/btrfs/sync/x86_64-rhel-8.3/8/debian-10.4-x86_64-20200603.cgz/300s/write/lkp-csl-2ap1/256g/fio-basic/0x4003003
commit:
5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size")
53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing")
5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 100% 4:4 stderr.Events_enabled
:4 100% 4:4 stderr.[perf_record:Captured_and_wrote#MB/tmp/lkp/perf-sched.data(#samples)]
:4 100% 4:4 stderr.[perf_record:Woken_up#times_to_write_data]
%stddev %change %stddev
\ | \
0.01 +1.1 1.12 ± 8% fio.latency_100us%
62.86 ± 3% -62.7 0.12 ±132% fio.latency_10us%
19.90 ± 15% -19.8 0.12 ±149% fio.latency_20us%
0.01 -0.0 0.00 fio.latency_2us%
16.87 ± 14% -16.9 0.01 fio.latency_4us%
0.03 ± 2% -0.0 0.00 fio.latency_50ms%
0.33 ± 13% +98.3 98.64 fio.latency_50us%
127.41 +137.0% 301.96 fio.time.elapsed_time
127.41 +137.0% 301.96 fio.time.elapsed_time.max
5.369e+08 -4.1% 5.151e+08 fio.time.file_system_outputs
1594 ± 6% +293.4% 6274 ± 3% fio.time.involuntary_context_switches
420.75 ± 3% +89.9% 799.00 fio.time.percent_of_cpu_this_job_got
480.22 ± 4% +390.9% 2357 fio.time.system_time
36993 -13.3% 32063 fio.time.voluntary_context_switches
67108864 -4.1% 64389238 fio.workload
2090 -59.9% 838.40 fio.write_bw_MBps
11616 ± 3% +263.6% 42240 fio.write_clat_90%_us
13248 ± 3% +239.1% 44928 fio.write_clat_95%_us
17408 ± 2% +189.7% 50432 fio.write_clat_99%_us
14517 +154.7% 36975 fio.write_clat_mean_us
388357 -98.6% 5502 ± 9% fio.write_clat_stddev
535275 -59.9% 214629 fio.write_iops
39055446 ± 26% +79.5% 70092416 ± 32% cpuidle.C1E.usage
248292 ± 3% +40.8% 349605 ± 19% cpuidle.POLL.usage
2.26 ± 2% -92.8% 0.16 ± 3% iostat.cpu.iowait
3.70 ± 2% +39.6% 5.16 iostat.cpu.system
176.92 +99.6% 353.08 uptime.boot
31486 +102.3% 63696 uptime.idle
2.29 ± 2% -2.1 0.16 ± 3% mpstat.cpu.all.iowait%
1.12 ± 2% -0.3 0.84 ± 8% mpstat.cpu.all.irq%
0.09 ± 3% -0.0 0.07 mpstat.cpu.all.soft%
2.53 ± 3% +1.7 4.27 mpstat.cpu.all.sys%
0.28 -0.2 0.13 ± 2% mpstat.cpu.all.usr%
93.00 +1.1% 94.00 vmstat.cpu.id
1837418 -57.0% 789462 ± 2% vmstat.io.bo
1.293e+08 -10.7% 1.155e+08 ± 3% vmstat.memory.cache
65719106 +22.0% 80155172 ± 5% vmstat.memory.free
4.25 ± 10% -100.0% 0.00 vmstat.procs.b
5.50 ± 9% +50.0% 8.25 ± 5% vmstat.procs.r
50381 ± 2% -73.0% 13588 vmstat.system.cs
8058918 ± 89% +181.2% 22659297 ± 5% numa-numastat.node0.local_node
8122514 ± 88% +179.8% 22724178 ± 5% numa-numastat.node0.numa_hit
8081647 ± 67% -99.7% 23653 ±173% numa-numastat.node0.numa_miss
8145258 ± 67% -98.9% 88543 ± 72% numa-numastat.node0.other_node
6489935 ± 95% -96.4% 232918 ± 76% numa-numastat.node1.numa_foreign
12996328 ± 70% -100.0% 0.00 numa-numastat.node2.numa_foreign
6677566 ± 15% +71.1% 11424129 ± 28% numa-numastat.node3.local_node
3818447 ± 87% -97.1% 108841 ±173% numa-numastat.node3.numa_foreign
6737072 ± 15% +70.6% 11495370 ± 28% numa-numastat.node3.numa_hit
10218977 ± 29% -100.0% 0.00 numa-numastat.node3.numa_miss
10278486 ± 29% -99.3% 71246 ± 21% numa-numastat.node3.other_node
117151 +62.7% 190593 ± 4% meminfo.Active
5415 +1142.0% 67255 ± 8% meminfo.Active(anon)
133307 ± 2% +32.2% 176190 meminfo.AnonHugePages
1.284e+08 -10.5% 1.149e+08 ± 3% meminfo.Cached
133470 ± 17% +31.5% 175470 ± 10% meminfo.CmaFree
1.276e+08 -10.6% 1.14e+08 ± 3% meminfo.Inactive
1.261e+08 -10.7% 1.126e+08 ± 3% meminfo.Inactive(file)
66215274 +21.3% 80314069 ± 5% meminfo.MemFree
1.315e+08 -10.7% 1.174e+08 ± 3% meminfo.Memused
309160 -11.6% 273349 meminfo.SUnreclaim
722723 -8.7% 659694 meminfo.Slab
19767477 ± 2% -86.3% 2702733 ± 4% meminfo.Writeback
1522111 -59.9% 610959 ± 3% meminfo.max_used_kB
6716 ± 51% +593.0% 46543 ± 17% numa-meminfo.node0.Active
4579 ± 93% +880.8% 44916 ± 18% numa-meminfo.node0.Active(file)
2632191 ± 17% +39.0% 3658947 ± 9% numa-meminfo.node0.Dirty
29868469 ± 23% +30.9% 39092332 ± 4% numa-meminfo.node0.FilePages
29706221 ± 23% +30.8% 38862897 ± 4% numa-meminfo.node0.Inactive
29324351 ± 25% +31.3% 38502867 ± 3% numa-meminfo.node0.Inactive(file)
18493244 ± 37% -49.2% 9395979 ± 19% numa-meminfo.node0.MemFree
30661618 ± 22% +29.7% 39758883 ± 4% numa-meminfo.node0.MemUsed
4820384 ± 13% -80.9% 922511 ± 11% numa-meminfo.node0.Writeback
7716 ± 7% -11.8% 6808 ± 8% numa-meminfo.node1.KernelStack
4733390 ± 14% -84.5% 733265 ± 12% numa-meminfo.node1.Writeback
3085152 ± 8% -28.5% 2205686 ± 23% numa-meminfo.node2.Dirty
36156759 ± 9% -28.6% 25803094 ± 19% numa-meminfo.node2.FilePages
35933307 ± 9% -28.7% 25615675 ± 19% numa-meminfo.node2.Inactive
35585652 ± 9% -29.9% 24928486 ± 21% numa-meminfo.node2.Inactive(file)
12572134 ± 28% +83.4% 23052389 ± 21% numa-meminfo.node2.MemFree
36962324 ± 9% -28.4% 26482069 ± 19% numa-meminfo.node2.MemUsed
5324212 ± 4% -89.0% 584306 ± 26% numa-meminfo.node2.Writeback
945.50 ± 92% +6446.5% 61897 ± 8% numa-meminfo.node3.Active(anon)
2861051 ± 10% -37.8% 1778186 ± 30% numa-meminfo.node3.Dirty
32510001 ± 15% -34.6% 21269226 ± 26% numa-meminfo.node3.FilePages
32349393 ± 16% -35.0% 21023730 ± 27% numa-meminfo.node3.Inactive
31940087 ± 15% -35.3% 20657807 ± 28% numa-meminfo.node3.Inactive(file)
16148531 ± 32% +71.1% 27626525 ± 20% numa-meminfo.node3.MemFree
33343003 ± 15% -34.4% 21865009 ± 26% numa-meminfo.node3.MemUsed
4867117 ± 12% -90.6% 457350 ± 27% numa-meminfo.node3.Writeback
1272 ± 2% -52.9% 598.75 ± 4% slabinfo.biovec-max.active_objs
1361 -49.1% 693.50 ± 3% slabinfo.biovec-max.num_objs
8179 +9.3% 8943 ± 3% slabinfo.btrfs_extent_buffer.num_objs
1605 ± 4% +6.6% 1712 ± 5% slabinfo.btrfs_ordered_extent.active_objs
1605 ± 4% +6.6% 1712 ± 5% slabinfo.btrfs_ordered_extent.num_objs
2117 ± 5% +23.1% 2607 ± 3% slabinfo.buffer_head.active_objs
2157 ± 5% +20.9% 2608 ± 3% slabinfo.buffer_head.num_objs
5316 ± 11% +103.5% 10821 ± 7% slabinfo.dmaengine-unmap-16.active_objs
127.50 ± 10% +103.3% 259.25 ± 7% slabinfo.dmaengine-unmap-16.active_slabs
5385 ± 10% +102.5% 10906 ± 7% slabinfo.dmaengine-unmap-16.num_objs
127.50 ± 10% +103.3% 259.25 ± 7% slabinfo.dmaengine-unmap-16.num_slabs
8840 ± 11% +22.0% 10780 ± 8% slabinfo.fsnotify_mark_connector.active_objs
8929 ± 11% +21.7% 10865 ± 7% slabinfo.fsnotify_mark_connector.num_objs
5642 ± 3% -7.6% 5214 ± 2% slabinfo.kmalloc-rcl-64.active_objs
5642 ± 3% -7.6% 5214 ± 2% slabinfo.kmalloc-rcl-64.num_objs
2207 ± 3% -21.4% 1735 ± 3% slabinfo.mnt_cache.active_objs
2291 ± 2% -20.2% 1828 ± 3% slabinfo.mnt_cache.num_objs
983.00 ± 11% +14.4% 1124 ± 5% slabinfo.proc_inode_cache.active_slabs
11805 ± 11% +14.4% 13504 ± 5% slabinfo.proc_inode_cache.num_objs
983.00 ± 11% +14.4% 1124 ± 5% slabinfo.proc_inode_cache.num_slabs
588271 -9.0% 535256 ± 2% slabinfo.radix_tree_node.active_objs
42019 -9.0% 38232 ± 2% slabinfo.radix_tree_node.active_slabs
588272 -9.0% 535256 ± 2% slabinfo.radix_tree_node.num_objs
42019 -9.0% 38232 ± 2% slabinfo.radix_tree_node.num_slabs
3437 ± 3% +16.0% 3986 slabinfo.trace_event_file.active_objs
3444 ± 3% +16.1% 3999 slabinfo.trace_event_file.num_objs
21837 +25.3% 27365 slabinfo.vmap_area.active_objs
21863 +25.4% 27407 slabinfo.vmap_area.num_objs
676874 ± 43% +122.4% 1505328 ± 39% proc-vmstat.compact_free_scanned
661.00 ± 10% +83.2% 1211 ± 26% proc-vmstat.kswapd_high_wmark_hit_quickly
216.50 ± 55% +880.4% 2122 ± 65% proc-vmstat.kswapd_low_wmark_hit_quickly
1352 +1138.6% 16751 ± 8% proc-vmstat.nr_active_anon
27860 +10.5% 30795 ± 2% proc-vmstat.nr_active_file
67188751 -4.1% 64460748 proc-vmstat.nr_dirtied
2830874 ± 2% -8.6% 2588405 proc-vmstat.nr_dirty
32053250 -10.4% 28710117 ± 3% proc-vmstat.nr_file_pages
33479 ± 17% +31.1% 43876 ± 10% proc-vmstat.nr_free_cma
16593949 +21.1% 20098175 ± 5% proc-vmstat.nr_free_pages
31485171 -10.7% 28118905 ± 3% proc-vmstat.nr_inactive_file
283883 +1.6% 288335 proc-vmstat.nr_mapped
279467 +7.2% 299680 proc-vmstat.nr_shmem
103212 -6.5% 96498 proc-vmstat.nr_slab_reclaimable
77257 -11.5% 68335 proc-vmstat.nr_slab_unreclaimable
4936950 ± 2% -86.4% 673846 ± 4% proc-vmstat.nr_writeback
1352 +1138.6% 16751 ± 8% proc-vmstat.nr_zone_active_anon
27873 +10.5% 30800 ± 2% proc-vmstat.nr_zone_active_file
31485279 -10.7% 28118952 ± 3% proc-vmstat.nr_zone_inactive_file
7768354 -58.0% 3262485 proc-vmstat.nr_zone_write_pending
29916365 ± 9% -70.7% 8750752 ± 50% proc-vmstat.numa_foreign
38401814 ± 7% +49.2% 57291640 ± 6% proc-vmstat.numa_hit
38140485 ± 7% +49.5% 57031158 ± 6% proc-vmstat.numa_local
29916365 ± 9% -70.7% 8750752 ± 50% proc-vmstat.numa_miss
30177694 ± 9% -70.1% 9011234 ± 48% proc-vmstat.numa_other
955.00 ± 16% +269.9% 3532 ± 33% proc-vmstat.pageoutrun
13359 +322.7% 56468 ± 8% proc-vmstat.pgactivate
68321287 -3.5% 65940158 proc-vmstat.pgalloc_normal
1085032 +61.4% 1750813 proc-vmstat.pgfault
28022 +131.2% 64789 proc-vmstat.pgreuse
27637 ± 82% +244.0% 95087 ± 21% sched_debug.cfs_rq:/.load.avg
628606 ± 13% +36.0% 854981 sched_debug.cfs_rq:/.load.max
107249 ± 45% +124.2% 240458 ± 10% sched_debug.cfs_rq:/.load.stddev
86480 ± 7% +19.2% 103104 ± 5% sched_debug.cfs_rq:/.min_vruntime.avg
113366 ± 2% +78.5% 202336 ± 3% sched_debug.cfs_rq:/.min_vruntime.max
7058 ± 7% +232.3% 23457 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev
0.05 ± 42% +109.5% 0.11 ± 19% sched_debug.cfs_rq:/.nr_running.avg
0.21 ± 20% +38.2% 0.28 ± 7% sched_debug.cfs_rq:/.nr_running.stddev
79.54 ± 7% +41.7% 112.71 ± 16% sched_debug.cfs_rq:/.runnable_avg.avg
834.79 ± 3% +22.7% 1024 sched_debug.cfs_rq:/.runnable_avg.max
160.25 ± 4% +69.1% 270.95 ± 8% sched_debug.cfs_rq:/.runnable_avg.stddev
36862 ± 34% +197.7% 109730 ± 9% sched_debug.cfs_rq:/.spread0.max
7074 ± 7% +231.6% 23457 ± 11% sched_debug.cfs_rq:/.spread0.stddev
79.49 ± 7% +41.7% 112.67 ± 16% sched_debug.cfs_rq:/.util_avg.avg
834.58 ± 4% +22.7% 1024 sched_debug.cfs_rq:/.util_avg.max
160.21 ± 4% +69.1% 270.90 ± 8% sched_debug.cfs_rq:/.util_avg.stddev
13.27 ± 53% +520.7% 82.36 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.avg
682.00 ± 11% +50.3% 1025 sched_debug.cfs_rq:/.util_est_enqueued.max
74.42 ± 17% +225.1% 241.91 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.stddev
2970391 ± 34% -52.6% 1408456 ± 6% sched_debug.cpu.avg_idle.max
35686 ± 14% +188.3% 102883 ± 26% sched_debug.cpu.avg_idle.min
282434 ± 23% -50.1% 140876 ± 6% sched_debug.cpu.avg_idle.stddev
93684 ± 14% +90.0% 178021 ± 6% sched_debug.cpu.clock.avg
93701 ± 14% +90.0% 178034 ± 6% sched_debug.cpu.clock.max
93667 ± 14% +90.0% 178007 ± 6% sched_debug.cpu.clock.min
9.76 ± 6% -23.1% 7.50 ± 5% sched_debug.cpu.clock.stddev
93008 ± 14% +90.1% 176795 ± 6% sched_debug.cpu.clock_task.avg
93308 ± 14% +89.8% 177069 ± 6% sched_debug.cpu.clock_task.max
84353 ± 16% +100.2% 168853 ± 7% sched_debug.cpu.clock_task.min
121.07 ± 24% +79.9% 217.78 ± 2% sched_debug.cpu.curr->pid.avg
5219 ± 9% +49.6% 7806 ± 5% sched_debug.cpu.curr->pid.max
674.76 ± 12% +45.6% 982.71 ± 2% sched_debug.cpu.curr->pid.stddev
1485195 ± 34% -52.2% 710548 ± 8% sched_debug.cpu.max_idle_balance_cost.max
89431 ± 54% -70.3% 26578 ± 21% sched_debug.cpu.max_idle_balance_cost.stddev
0.03 ± 25% +53.8% 0.05 ± 3% sched_debug.cpu.nr_running.avg
0.18 ± 13% +26.8% 0.22 sched_debug.cpu.nr_running.stddev
824.12 ± 13% +36.8% 1126 ± 12% sched_debug.cpu.nr_switches.min
0.02 ± 22% -88.0% 0.00 ± 25% sched_debug.cpu.nr_uninterruptible.avg
93667 ± 14% +90.0% 178008 ± 6% sched_debug.cpu_clk
92796 ± 14% +90.9% 177135 ± 6% sched_debug.ktime
94042 ± 14% +89.7% 178380 ± 6% sched_debug.sched_clk
1145 ± 92% +880.4% 11230 ± 18% numa-vmstat.node0.nr_active_file
8172691 ± 31% +55.7% 12720947 ± 9% numa-vmstat.node0.nr_dirtied
655987 ± 17% +39.5% 914941 ± 9% numa-vmstat.node0.nr_dirty
7465216 ± 23% +30.9% 9772897 ± 4% numa-vmstat.node0.nr_file_pages
4625156 ± 37% -49.2% 2349212 ± 19% numa-vmstat.node0.nr_free_pages
7328653 ± 25% +31.3% 9625646 ± 3% numa-vmstat.node0.nr_inactive_file
1206608 ± 13% -80.9% 230245 ± 10% numa-vmstat.node0.nr_writeback
6309984 ± 36% +83.5% 11575697 ± 9% numa-vmstat.node0.nr_written
1148 ± 92% +878.0% 11232 ± 18% numa-vmstat.node0.nr_zone_active_file
7328667 ± 25% +31.3% 9625676 ± 3% numa-vmstat.node0.nr_zone_inactive_file
1862745 ± 14% -38.5% 1145274 ± 9% numa-vmstat.node0.nr_zone_write_pending
4611857 ± 95% +192.1% 13471453 ± 8% numa-vmstat.node0.numa_hit
4514907 ± 97% +196.2% 13371363 ± 8% numa-vmstat.node0.numa_local
4280076 ± 69% -100.0% 1088 ±173% numa-vmstat.node0.numa_miss
4377034 ± 68% -97.7% 101182 ± 60% numa-vmstat.node0.numa_other
7710 ± 7% -11.8% 6799 ± 9% numa-vmstat.node1.nr_kernel_stack
1184420 ± 14% -84.5% 183552 ± 12% numa-vmstat.node1.nr_writeback
1870949 ± 14% -54.0% 861047 ± 8% numa-vmstat.node1.nr_zone_write_pending
3181676 ±125% -97.3% 84818 ±125% numa-vmstat.node1.numa_foreign
10400753 ± 13% -37.3% 6523007 ± 23% numa-vmstat.node2.nr_dirtied
769495 ± 8% -28.3% 551406 ± 23% numa-vmstat.node2.nr_dirty
9042967 ± 9% -28.7% 6451632 ± 19% numa-vmstat.node2.nr_file_pages
3139286 ± 27% +83.6% 5762258 ± 22% numa-vmstat.node2.nr_free_pages
8899645 ± 9% -30.0% 6232951 ± 21% numa-vmstat.node2.nr_inactive_file
1332398 ± 4% -89.0% 146368 ± 26% numa-vmstat.node2.nr_writeback
8899674 ± 9% -30.0% 6232954 ± 21% numa-vmstat.node2.nr_zone_inactive_file
2102021 ± 5% -66.8% 697822 ± 24% numa-vmstat.node2.nr_zone_write_pending
7084511 ± 73% -100.0% 0.00 numa-vmstat.node2.numa_foreign
236.25 ± 92% +6461.2% 15500 ± 8% numa-vmstat.node3.nr_active_anon
8919994 ± 20% -40.8% 5283319 ± 31% numa-vmstat.node3.nr_dirtied
715123 ± 10% -37.8% 444600 ± 30% numa-vmstat.node3.nr_dirty
8132346 ± 15% -34.6% 5317735 ± 26% numa-vmstat.node3.nr_file_pages
33475 ± 17% +31.0% 43866 ± 10% numa-vmstat.node3.nr_free_cma
4032260 ± 32% +71.3% 6906250 ± 20% numa-vmstat.node3.nr_free_pages
7990046 ± 15% -35.4% 5165114 ± 28% numa-vmstat.node3.nr_inactive_file
1215444 ± 12% -90.6% 114335 ± 27% numa-vmstat.node3.nr_writeback
236.25 ± 92% +6461.2% 15500 ± 8% numa-vmstat.node3.nr_zone_active_anon
7990067 ± 15% -35.4% 5165114 ± 28% numa-vmstat.node3.nr_zone_inactive_file
1930722 ± 10% -71.0% 558966 ± 29% numa-vmstat.node3.nr_zone_write_pending
1901104 ± 90% -99.7% 5165 ±173% numa-vmstat.node3.numa_foreign
4150742 ± 14% +45.5% 6039465 ± 26% numa-vmstat.node3.numa_hit
4016578 ± 15% +46.4% 5878550 ± 27% numa-vmstat.node3.numa_local
5594513 ± 41% -100.0% 0.00 numa-vmstat.node3.numa_miss
5728682 ± 40% -97.2% 160916 ± 8% numa-vmstat.node3.numa_other
11.78 ± 3% -40.4% 7.02 ± 6% perf-stat.i.MPKI
2.286e+09 -4.8% 2.177e+09 perf-stat.i.branch-instructions
0.65 -0.2 0.49 perf-stat.i.branch-miss-rate%
15863136 ± 2% -30.8% 10985192 ± 2% perf-stat.i.branch-misses
87495643 ± 4% -47.7% 45751362 ± 7% perf-stat.i.cache-misses
1.392e+08 ± 3% -45.8% 75512046 ± 7% perf-stat.i.cache-references
51205 ± 2% -73.5% 13590 perf-stat.i.context-switches
2.65 +64.0% 4.34 perf-stat.i.cpi
3.067e+10 ± 2% +43.6% 4.404e+10 perf-stat.i.cpu-cycles
455.07 ± 5% +169.3% 1225 ± 9% perf-stat.i.cycles-between-cache-misses
273457 ± 65% -67.1% 89970 ± 17% perf-stat.i.dTLB-load-misses
3.329e+09 -22.9% 2.566e+09 perf-stat.i.dTLB-loads
64168 ± 37% -52.7% 30346 ± 7% perf-stat.i.dTLB-store-misses
1.672e+09 -41.6% 9.756e+08 perf-stat.i.dTLB-stores
61.66 ± 2% -8.9 52.72 perf-stat.i.iTLB-load-miss-rate%
7608055 ± 6% -29.7% 5350529 perf-stat.i.iTLB-load-misses
4712088 +1.8% 4794810 perf-stat.i.iTLB-loads
1.164e+10 -10.9% 1.036e+10 perf-stat.i.instructions
1532 ± 6% +26.0% 1931 perf-stat.i.instructions-per-iTLB-miss
0.39 -39.4% 0.24 perf-stat.i.ipc
2212 -57.9% 930.49 perf-stat.i.major-faults
0.16 ± 2% +44.1% 0.23 perf-stat.i.metric.GHz
0.83 ± 18% -38.0% 0.52 ± 16% perf-stat.i.metric.K/sec
38.83 -22.1% 30.26 perf-stat.i.metric.M/sec
3836 -0.9% 3801 perf-stat.i.minor-faults
13804926 ± 4% -40.0% 8289551 ± 3% perf-stat.i.node-load-misses
1907846 ± 9% -44.5% 1058077 ± 7% perf-stat.i.node-loads
4552731 ± 6% -34.0% 3003065 ± 4% perf-stat.i.node-store-misses
287448 ± 48% -42.0% 166584 ± 14% perf-stat.i.node-stores
6049 -21.8% 4731 perf-stat.i.page-faults
11.96 ± 3% -39.1% 7.28 ± 6% perf-stat.overall.MPKI
0.70 -0.2 0.51 perf-stat.overall.branch-miss-rate%
2.63 +61.4% 4.25 perf-stat.overall.cpi
351.94 ± 5% +174.9% 967.33 ± 7% perf-stat.overall.cycles-between-cache-misses
61.70 ± 2% -9.0 52.74 perf-stat.overall.iTLB-load-miss-rate%
1537 ± 6% +26.1% 1937 perf-stat.overall.instructions-per-iTLB-miss
0.38 -38.0% 0.24 perf-stat.overall.ipc
22102 +119.7% 48551 perf-stat.overall.path-length
2.268e+09 -4.4% 2.17e+09 perf-stat.ps.branch-instructions
15776375 ± 2% -30.4% 10975709 ± 2% perf-stat.ps.branch-misses
86567030 ± 4% -47.3% 45589562 ± 7% perf-stat.ps.cache-misses
1.381e+08 ± 3% -45.5% 75270910 ± 7% perf-stat.ps.cache-references
50561 ± 2% -73.2% 13544 perf-stat.ps.context-switches
3.04e+10 ± 2% +44.3% 4.387e+10 perf-stat.ps.cpu-cycles
3.302e+09 -22.6% 2.557e+09 perf-stat.ps.dTLB-loads
63958 ± 37% -52.3% 30512 ± 7% perf-stat.ps.dTLB-store-misses
1.658e+09 -41.4% 9.723e+08 perf-stat.ps.dTLB-stores
7542975 ± 6% -29.3% 5331814 perf-stat.ps.iTLB-load-misses
4669996 +2.3% 4777284 perf-stat.ps.iTLB-loads
1.155e+10 -10.6% 1.033e+10 perf-stat.ps.instructions
2191 -57.5% 931.32 perf-stat.ps.major-faults
13652567 ± 4% -39.5% 8255968 ± 3% perf-stat.ps.node-load-misses
1887494 ± 9% -44.1% 1054935 ± 7% perf-stat.ps.node-loads
4506248 ± 6% -33.7% 2989856 ± 4% perf-stat.ps.node-store-misses
286096 ± 48% -41.9% 166311 ± 14% perf-stat.ps.node-stores
6008 -21.4% 4723 perf-stat.ps.page-faults
1.483e+12 +110.8% 3.126e+12 perf-stat.total.instructions
0.01 ± 53% -87.5% 0.00 ±173% perf-sched.sch_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
0.00 ± 23% -66.7% 0.00 perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
5.36 ± 62% -99.9% 0.01 ± 10% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.07 ± 39% -91.8% 0.01 ± 26% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.11 ± 33% -85.0% 0.02 ± 22% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
0.01 ± 17% +162.1% 0.02 ± 12% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.05 ± 54% -87.5% 0.01 ± 28% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
1.42 ± 60% +230.1% 4.68 ± 19% perf-sched.sch_delay.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.06 ± 17% -73.2% 0.02 ± 11% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 ±173% +2066.7% 0.02 ± 15% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
22.77 ± 59% -99.9% 0.01 ± 15% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.00 ±173% +1150.0% 0.01 ± 14% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
20.53 ± 34% -85.8% 2.90 ± 83% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
2.30 +197.5% 6.84 ± 10% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork
0.02 ± 27% -81.2% 0.00 perf-sched.total_sch_delay.average.ms
27.75 ± 20% -74.1% 7.19 ± 16% perf-sched.total_sch_delay.max.ms
0.32 ± 18% +7823.9% 25.73 ± 13% perf-sched.total_wait_and_delay.average.ms
3635 ± 4% +2913.9% 109554 ± 17% perf-sched.total_wait_and_delay.count.ms
158.86 ± 5% +5057.9% 8194 ± 6% perf-sched.total_wait_and_delay.max.ms
0.31 ± 19% +8233.6% 25.73 ± 13% perf-sched.total_wait_time.average.ms
158.85 ± 5% +5058.2% 8194 ± 6% perf-sched.total_wait_time.max.ms
0.51 ±173% +44191.4% 226.99 ± 2% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
158.86 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
0.07 ± 2% +45.3% 0.10 ± 19% perf-sched.wait_and_delay.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.04 ± 6% +1.2e+05% 46.19 perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.02 ± 14% +21001.2% 4.38 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
6.88 ± 59% +8627.8% 600.56 ± 56% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
3.86 ± 5% +13.4% 4.38 perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
1.25 ± 26% +60259.2% 756.75 perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
7.66 ± 11% +2755.3% 218.84 ± 10% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork
1.50 ±173% +20216.7% 304.75 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
1.00 -100.0% 0.00 perf-sched.wait_and_delay.count.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
2255 ± 2% +4082.4% 94312 ± 19% perf-sched.wait_and_delay.count.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
660.75 ± 17% +428.1% 3489 perf-sched.wait_and_delay.count.pipe_read.new_sync_read.vfs_read.ksys_read
312.50 ± 17% +575.9% 2112 perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
3.25 ± 59% +376.9% 15.50 ± 65% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
41.00 ± 3% +5462.8% 2280 perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
314.75 ± 17% +694.1% 2499 perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
29.50 ± 56% +7337.3% 2194 ± 9% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
2.83 ±173% +1.3e+05% 3654 ± 31% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
158.86 ± 5% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
3.27 ± 46% +27193.5% 892.91 ± 76% perf-sched.wait_and_delay.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.11 ± 29% +9.2e+05% 1018 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
5.50 ± 3% +18112.0% 1000 perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
22.75 ± 59% +28414.4% 6485 ± 53% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
51.79 ±114% +3718.9% 1977 ± 32% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
75.64 ± 19% +8319.0% 6368 ± 16% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
0.53 ±166% +42910.9% 226.99 ± 2% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
158.85 ± 5% -99.9% 0.13 ±173% perf-sched.wait_time.avg.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
0.07 ± 2% +48.9% 0.10 ± 20% perf-sched.wait_time.avg.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.04 ± 8% +1.3e+05% 46.19 perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.02 ± 14% +21001.2% 4.38 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
1.57 ± 88% +38243.4% 600.55 ± 56% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
3.85 ± 5% +13.5% 4.37 perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
1.18 ± 28% +63976.9% 756.75 perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
7.55 ± 12% +2796.8% 218.82 ± 10% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork
2.90 ±167% +1.3e+05% 3654 ± 31% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
158.85 ± 5% -99.7% 0.46 ±173% perf-sched.wait_time.max.ms.futex_wait_queue_me.futex_wait.do_futex.__x64_sys_futex
2.79 ± 57% +31839.1% 892.54 ± 76% perf-sched.wait_time.max.ms.io_schedule.rq_qos_wait.wbt_wait.__rq_qos_throttle
0.11 ± 32% +9.6e+05% 1018 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
5.50 ± 3% +18111.8% 1000 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.affine_move_task.__set_cpus_allowed_ptr
6.38 ± 88% +1e+05% 6485 ± 53% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
42.79 ±150% +4522.0% 1977 ± 32% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
75.61 ± 19% +8322.7% 6368 ± 16% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
23273 ± 5% +79.4% 41758 ± 8% softirqs.CPU0.SCHED
19224 ± 3% +102.6% 38950 softirqs.CPU1.SCHED
17936 ± 3% +117.1% 38939 ± 3% softirqs.CPU10.SCHED
17462 ± 4% +113.1% 37218 ± 6% softirqs.CPU100.SCHED
9336 ± 24% -56.9% 4027 ± 3% softirqs.CPU101.RCU
16134 ± 17% +114.7% 34644 ± 19% softirqs.CPU102.SCHED
18025 ± 2% +89.8% 34220 ± 19% softirqs.CPU103.SCHED
17976 ± 3% +114.2% 38508 softirqs.CPU104.SCHED
17940 ± 3% +110.4% 37752 ± 3% softirqs.CPU105.SCHED
17976 ± 3% +112.8% 38259 softirqs.CPU106.SCHED
17892 ± 5% +105.6% 36785 ± 5% softirqs.CPU107.SCHED
17998 ± 3% +93.0% 34732 ± 18% softirqs.CPU108.SCHED
17812 ± 4% +104.9% 36492 ± 5% softirqs.CPU109.SCHED
509.25 ±129% +2290.8% 12175 ± 77% softirqs.CPU11.NET_RX
17542 ± 3% +97.5% 34636 ± 19% softirqs.CPU11.SCHED
17930 ± 3% +107.5% 37206 ± 4% softirqs.CPU110.SCHED
16041 ± 15% +131.7% 37174 ± 7% softirqs.CPU111.SCHED
17512 ± 7% +85.2% 32425 ± 19% softirqs.CPU112.SCHED
17732 ± 4% +101.9% 35800 ± 14% softirqs.CPU113.SCHED
17731 ± 5% +99.1% 35301 ± 17% softirqs.CPU114.SCHED
18034 ± 2% +73.3% 31260 ± 22% softirqs.CPU115.SCHED
18000 ± 2% +71.9% 30947 ± 23% softirqs.CPU116.SCHED
18037 ± 2% +71.8% 30995 ± 25% softirqs.CPU117.SCHED
17819 ± 4% +115.9% 38469 softirqs.CPU118.SCHED
18069 ± 2% +106.0% 37229 ± 5% softirqs.CPU119.SCHED
16670 ± 4% +63.4% 27233 ± 16% softirqs.CPU120.SCHED
17658 ± 3% +101.2% 35533 ± 16% softirqs.CPU122.SCHED
16398 ± 16% +136.4% 38771 softirqs.CPU123.SCHED
17833 ± 3% +117.8% 38834 softirqs.CPU124.SCHED
17981 ± 2% +116.6% 38948 softirqs.CPU125.SCHED
17859 ± 2% +134.7% 41917 ± 12% softirqs.CPU126.SCHED
17735 ± 3% +115.3% 38177 softirqs.CPU127.SCHED
17671 ± 3% +117.7% 38463 softirqs.CPU128.SCHED
17888 ± 2% +89.5% 33904 ± 19% softirqs.CPU129.SCHED
17864 ± 2% +115.3% 38462 softirqs.CPU130.SCHED
17937 ± 2% +116.3% 38796 softirqs.CPU131.SCHED
17823 ± 2% +116.6% 38613 softirqs.CPU132.SCHED
17817 ± 2% +115.6% 38423 softirqs.CPU133.SCHED
17467 ± 6% +120.5% 38521 softirqs.CPU134.SCHED
17443 ± 5% +90.5% 33227 ± 20% softirqs.CPU135.SCHED
17409 ± 5% +119.3% 38180 softirqs.CPU136.SCHED
17427 ± 6% +122.5% 38779 softirqs.CPU137.SCHED
17994 +115.9% 38853 softirqs.CPU138.SCHED
18000 ± 2% +112.9% 38319 softirqs.CPU139.SCHED
17685 ± 3% +96.7% 34793 ± 20% softirqs.CPU140.SCHED
17559 ± 4% +120.8% 38767 softirqs.CPU141.SCHED
17880 ± 2% +117.1% 38810 softirqs.CPU142.SCHED
18018 ± 2% +114.8% 38700 softirqs.CPU143.SCHED
15816 ± 13% +52.0% 24042 ± 16% softirqs.CPU144.SCHED
17373 ± 4% +120.4% 38293 softirqs.CPU145.SCHED
17900 +117.5% 38927 softirqs.CPU146.SCHED
15928 ± 18% +145.0% 39017 softirqs.CPU147.SCHED
15242 ± 18% +154.5% 38791 softirqs.CPU148.SCHED
17218 ± 4% +125.7% 38866 softirqs.CPU149.SCHED
17388 ± 6% +106.6% 35926 ± 9% softirqs.CPU15.SCHED
17972 ± 2% +115.0% 38643 softirqs.CPU150.SCHED
16587 ± 8% +132.9% 38637 softirqs.CPU151.SCHED
17468 ± 2% +121.6% 38703 softirqs.CPU152.SCHED
17060 ± 7% +127.6% 38826 softirqs.CPU153.SCHED
17641 ± 2% +119.1% 38646 softirqs.CPU154.SCHED
17892 +78.1% 31866 ± 23% softirqs.CPU155.SCHED
17924 +115.9% 38701 softirqs.CPU156.SCHED
17856 +115.9% 38551 softirqs.CPU157.SCHED
17950 +114.0% 38405 softirqs.CPU158.SCHED
17923 +113.7% 38302 softirqs.CPU159.SCHED
17541 ± 2% +98.6% 34837 ± 20% softirqs.CPU160.SCHED
17833 +116.4% 38583 softirqs.CPU161.SCHED
14192 ± 29% +173.0% 38749 softirqs.CPU162.SCHED
17193 ± 3% +122.9% 38327 softirqs.CPU163.SCHED
16821 ± 9% +131.5% 38948 softirqs.CPU164.SCHED
16698 ± 8% +133.0% 38903 softirqs.CPU165.SCHED
16020 ± 12% +138.9% 38264 ± 2% softirqs.CPU166.SCHED
16919 ± 8% +130.1% 38934 softirqs.CPU167.SCHED
16632 ± 2% +32.4% 22019 ± 17% softirqs.CPU168.SCHED
17823 ± 2% +116.6% 38604 ± 2% softirqs.CPU169.SCHED
17983 +119.0% 39386 softirqs.CPU170.SCHED
17669 +122.6% 39324 softirqs.CPU171.SCHED
17927 +116.3% 38781 softirqs.CPU172.SCHED
18141 +116.2% 39216 softirqs.CPU173.SCHED
17888 +118.6% 39099 softirqs.CPU174.SCHED
18076 +116.9% 39214 softirqs.CPU175.SCHED
17994 +115.3% 38746 softirqs.CPU176.SCHED
17615 ± 3% +121.8% 39065 softirqs.CPU177.SCHED
17871 ± 2% +119.4% 39203 softirqs.CPU178.SCHED
16330 ± 19% +138.6% 38966 softirqs.CPU179.SCHED
16932 ± 10% +73.9% 29442 ± 31% softirqs.CPU18.SCHED
17972 +109.4% 37633 ± 6% softirqs.CPU180.SCHED
17896 +114.7% 38419 softirqs.CPU181.SCHED
18036 +116.6% 39073 softirqs.CPU182.SCHED
17883 ± 2% +118.8% 39123 softirqs.CPU183.SCHED
6988 ± 28% -49.0% 3563 ± 6% softirqs.CPU184.RCU
18068 +118.3% 39452 softirqs.CPU184.SCHED
18037 +116.9% 39131 softirqs.CPU185.SCHED
18445 ± 2% +111.6% 39021 softirqs.CPU186.SCHED
17861 ± 2% +119.1% 39126 softirqs.CPU187.SCHED
18094 +116.5% 39179 softirqs.CPU188.SCHED
18041 +116.1% 38988 softirqs.CPU189.SCHED
17981 +118.7% 39332 softirqs.CPU190.SCHED
18050 +115.7% 38940 softirqs.CPU191.SCHED
17923 ± 3% +113.4% 38253 softirqs.CPU2.SCHED
17588 ± 3% +89.1% 33267 ± 24% softirqs.CPU20.SCHED
17894 ± 2% +88.0% 33642 ± 21% softirqs.CPU21.SCHED
17032 ± 10% +122.3% 37859 softirqs.CPU22.SCHED
15841 ± 19% +138.4% 37766 softirqs.CPU23.SCHED
15987 ± 4% +38.6% 22162 ± 22% softirqs.CPU24.SCHED
17548 ± 4% +93.9% 34034 ± 19% softirqs.CPU25.SCHED
17581 ± 3% +111.1% 37114 ± 4% softirqs.CPU26.SCHED
17571 ± 3% +116.2% 37995 softirqs.CPU27.SCHED
17797 ± 2% +113.3% 37968 softirqs.CPU28.SCHED
17877 +113.8% 38215 softirqs.CPU29.SCHED
17757 ± 2% +95.3% 34689 ± 17% softirqs.CPU3.SCHED
17713 ± 3% +92.0% 34004 ± 20% softirqs.CPU30.SCHED
16785 ± 6% +126.0% 37939 softirqs.CPU31.SCHED
17665 ± 3% +113.8% 37762 softirqs.CPU32.SCHED
17874 ± 2% +112.8% 38032 softirqs.CPU33.SCHED
17826 ± 2% +111.6% 37717 softirqs.CPU34.SCHED
17848 +112.7% 37964 softirqs.CPU35.SCHED
17898 +111.8% 37914 softirqs.CPU36.SCHED
18035 ± 2% +109.2% 37733 softirqs.CPU37.SCHED
17727 ± 2% +113.0% 37751 softirqs.CPU38.SCHED
17779 ± 2% +106.8% 36770 ± 5% softirqs.CPU39.SCHED
16919 ± 6% +123.6% 37824 softirqs.CPU4.SCHED
17784 ± 4% +110.5% 37430 softirqs.CPU40.SCHED
17540 ± 4% +116.5% 37977 softirqs.CPU41.SCHED
17772 ± 2% +113.9% 38021 softirqs.CPU42.SCHED
17473 ± 4% +114.7% 37515 softirqs.CPU43.SCHED
17717 ± 2% +115.0% 38090 softirqs.CPU44.SCHED
17551 ± 5% +116.2% 37950 softirqs.CPU45.SCHED
17784 +113.6% 37995 softirqs.CPU46.SCHED
17887 ± 2% +112.5% 38019 softirqs.CPU47.SCHED
17224 ± 2% +97.3% 33983 ± 16% softirqs.CPU49.SCHED
17391 +94.1% 33758 ± 20% softirqs.CPU5.SCHED
17685 +120.2% 38937 ± 3% softirqs.CPU50.SCHED
8016 ± 27% -46.0% 4329 ± 2% softirqs.CPU51.RCU
17841 +113.4% 38072 softirqs.CPU51.SCHED
17600 ± 3% +116.9% 38171 softirqs.CPU52.SCHED
17239 ± 2% +121.2% 38137 softirqs.CPU53.SCHED
17822 +113.4% 38030 softirqs.CPU54.SCHED
17055 ± 5% +121.8% 37830 softirqs.CPU55.SCHED
17653 ± 2% +115.4% 38022 softirqs.CPU56.SCHED
17894 +112.4% 38004 softirqs.CPU57.SCHED
17582 ± 3% +115.4% 37869 softirqs.CPU58.SCHED
17678 +84.8% 32662 ± 17% softirqs.CPU59.SCHED
17560 ± 4% +90.6% 33468 ± 20% softirqs.CPU6.SCHED
17824 +112.4% 37864 softirqs.CPU60.SCHED
17914 +115.9% 38674 ± 3% softirqs.CPU61.SCHED
17675 +113.5% 37735 softirqs.CPU62.SCHED
17695 +113.5% 37774 softirqs.CPU63.SCHED
17783 +114.3% 38110 softirqs.CPU64.SCHED
17911 +112.9% 38129 softirqs.CPU65.SCHED
17941 +112.6% 38136 softirqs.CPU66.SCHED
17838 +110.9% 37629 softirqs.CPU67.SCHED
17747 +115.4% 38234 softirqs.CPU68.SCHED
17318 ± 3% +120.1% 38109 softirqs.CPU69.SCHED
17585 +104.6% 35981 ± 10% softirqs.CPU70.SCHED
17432 ± 2% +117.9% 37991 softirqs.CPU71.SCHED
14703 ± 5% +50.7% 22151 ± 12% softirqs.CPU72.SCHED
17016 ± 4% +97.0% 33521 ± 24% softirqs.CPU73.SCHED
17008 ± 6% +125.7% 38384 softirqs.CPU74.SCHED
17574 ± 2% +119.6% 38600 softirqs.CPU75.SCHED
17676 ± 3% +115.6% 38107 softirqs.CPU76.SCHED
17819 +115.6% 38416 softirqs.CPU77.SCHED
17807 +114.1% 38120 softirqs.CPU78.SCHED
17659 +116.9% 38296 softirqs.CPU79.SCHED
17796 ± 2% +113.1% 37930 softirqs.CPU8.SCHED
15987 ± 18% +137.9% 38037 softirqs.CPU80.SCHED
17684 ± 2% +116.2% 38237 softirqs.CPU81.SCHED
17850 +115.5% 38466 softirqs.CPU82.SCHED
17789 +115.0% 38252 softirqs.CPU83.SCHED
17743 +113.3% 37843 softirqs.CPU85.SCHED
17804 +115.6% 38388 softirqs.CPU86.SCHED
17931 +114.2% 38416 softirqs.CPU87.SCHED
17271 ± 4% +121.8% 38309 softirqs.CPU88.SCHED
17798 +115.9% 38423 softirqs.CPU89.SCHED
17862 +113.5% 38140 softirqs.CPU90.SCHED
17849 +119.2% 39127 ± 4% softirqs.CPU91.SCHED
17846 +115.3% 38424 softirqs.CPU92.SCHED
17699 +116.5% 38316 softirqs.CPU93.SCHED
17868 +115.6% 38524 softirqs.CPU94.SCHED
18023 ± 2% +112.8% 38355 softirqs.CPU97.SCHED
16630 ± 15% +134.1% 38934 softirqs.CPU98.SCHED
18017 ± 3% +115.8% 38889 softirqs.CPU99.SCHED
6591 ± 56% +299.9% 26359 ± 47% softirqs.NET_RX
1256534 ± 8% -37.6% 783562 softirqs.RCU
3375418 +108.0% 7022041 softirqs.SCHED
44803 ± 3% +32.1% 59178 softirqs.TIMER
775.00 ±142% +2464.5% 19875 ± 92% interrupts.32:PCI-MSI.524290-edge.eth0-TxRx-1
258.50 +134.8% 607.00 interrupts.9:IO-APIC.9-fasteoi.acpi
257240 +135.8% 606454 interrupts.CPU0.LOC:Local_timer_interrupts
0.00 +2e+105% 1951 ±146% interrupts.CPU0.NMI:Non-maskable_interrupts
0.00 +2e+105% 1951 ±146% interrupts.CPU0.PMI:Performance_monitoring_interrupts
258.50 +134.8% 607.00 interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
257926 +136.8% 610646 interrupts.CPU1.LOC:Local_timer_interrupts
0.50 ±100% +64450.0% 322.75 ± 32% interrupts.CPU1.NMI:Non-maskable_interrupts
0.50 ±100% +64450.0% 322.75 ± 32% interrupts.CPU1.PMI:Performance_monitoring_interrupts
258039 +135.1% 606641 interrupts.CPU10.LOC:Local_timer_interrupts
258014 +136.7% 610642 interrupts.CPU100.LOC:Local_timer_interrupts
257987 +135.0% 606348 interrupts.CPU101.LOC:Local_timer_interrupts
257966 +135.0% 606229 interrupts.CPU102.LOC:Local_timer_interrupts
257652 +135.0% 605468 interrupts.CPU103.LOC:Local_timer_interrupts
257994 +134.7% 605530 interrupts.CPU104.LOC:Local_timer_interrupts
257551 +135.4% 606204 interrupts.CPU105.LOC:Local_timer_interrupts
257967 +135.2% 606681 interrupts.CPU106.LOC:Local_timer_interrupts
257986 +135.1% 606441 interrupts.CPU107.LOC:Local_timer_interrupts
257993 +134.7% 605584 interrupts.CPU108.LOC:Local_timer_interrupts
257977 +135.1% 606448 interrupts.CPU109.LOC:Local_timer_interrupts
775.00 ±142% +2464.5% 19875 ± 92% interrupts.CPU11.32:PCI-MSI.524290-edge.eth0-TxRx-1
258008 +135.1% 606672 interrupts.CPU11.LOC:Local_timer_interrupts
257964 +135.1% 606566 interrupts.CPU110.LOC:Local_timer_interrupts
257917 +135.0% 606175 interrupts.CPU111.LOC:Local_timer_interrupts
257983 +135.2% 606679 interrupts.CPU112.LOC:Local_timer_interrupts
9.00 ±134% +597.2% 62.75 ± 88% interrupts.CPU112.RES:Rescheduling_interrupts
257969 +134.9% 606062 interrupts.CPU113.LOC:Local_timer_interrupts
7.00 ±111% +553.6% 45.75 ± 99% interrupts.CPU113.RES:Rescheduling_interrupts
258017 +135.0% 606228 interrupts.CPU114.LOC:Local_timer_interrupts
7.50 ±150% +753.3% 64.00 ± 76% interrupts.CPU114.RES:Rescheduling_interrupts
258012 +135.1% 606643 interrupts.CPU115.LOC:Local_timer_interrupts
1.25 ± 87% +6020.0% 76.50 ± 82% interrupts.CPU115.RES:Rescheduling_interrupts
258004 +134.9% 606081 interrupts.CPU116.LOC:Local_timer_interrupts
1.00 ±100% +8200.0% 83.00 ± 70% interrupts.CPU116.RES:Rescheduling_interrupts
258008 +135.1% 606547 interrupts.CPU117.LOC:Local_timer_interrupts
1.50 ±173% +1.6e+05% 2394 ±146% interrupts.CPU117.NMI:Non-maskable_interrupts
1.50 ±173% +1.6e+05% 2394 ±146% interrupts.CPU117.PMI:Performance_monitoring_interrupts
1.25 ± 87% +4300.0% 55.00 ±116% interrupts.CPU117.RES:Rescheduling_interrupts
257990 +135.7% 608177 interrupts.CPU118.LOC:Local_timer_interrupts
258010 +135.1% 606654 interrupts.CPU119.LOC:Local_timer_interrupts
258067 +135.0% 606502 interrupts.CPU12.LOC:Local_timer_interrupts
257434 +135.3% 605760 interrupts.CPU120.LOC:Local_timer_interrupts
258027 +134.6% 605384 interrupts.CPU121.LOC:Local_timer_interrupts
255276 +137.2% 605433 interrupts.CPU122.LOC:Local_timer_interrupts
256928 +135.3% 604514 interrupts.CPU123.LOC:Local_timer_interrupts
257505 +135.2% 605681 interrupts.CPU124.LOC:Local_timer_interrupts
257792 +134.9% 605545 interrupts.CPU125.LOC:Local_timer_interrupts
256996 +135.6% 605474 interrupts.CPU126.LOC:Local_timer_interrupts
257022 +135.6% 605654 interrupts.CPU127.LOC:Local_timer_interrupts
258006 +134.8% 605780 interrupts.CPU128.LOC:Local_timer_interrupts
257568 +135.2% 605783 interrupts.CPU129.LOC:Local_timer_interrupts
258047 +135.1% 606566 interrupts.CPU13.LOC:Local_timer_interrupts
31.75 ±165% +392.9% 156.50 ± 65% interrupts.CPU13.RES:Rescheduling_interrupts
257934 +134.9% 605825 interrupts.CPU130.LOC:Local_timer_interrupts
257688 +135.1% 605785 interrupts.CPU131.LOC:Local_timer_interrupts
257934 +134.9% 605802 interrupts.CPU132.LOC:Local_timer_interrupts
258031 +134.8% 605766 interrupts.CPU133.LOC:Local_timer_interrupts
257182 +135.6% 605817 interrupts.CPU134.LOC:Local_timer_interrupts
0.50 ±173% +55750.0% 279.25 ± 8% interrupts.CPU134.NMI:Non-maskable_interrupts
0.50 ±173% +55750.0% 279.25 ± 8% interrupts.CPU134.PMI:Performance_monitoring_interrupts
256554 +136.0% 605423 interrupts.CPU135.LOC:Local_timer_interrupts
256889 +135.6% 605342 interrupts.CPU136.LOC:Local_timer_interrupts
257503 +135.2% 605773 interrupts.CPU137.LOC:Local_timer_interrupts
257142 +135.6% 605805 interrupts.CPU138.LOC:Local_timer_interrupts
0.25 ±173% +1e+05% 261.00 ± 25% interrupts.CPU138.NMI:Non-maskable_interrupts
0.25 ±173% +1e+05% 261.00 ± 25% interrupts.CPU138.PMI:Performance_monitoring_interrupts
1951 ± 13% +44.1% 2812 ± 28% interrupts.CPU139.CAL:Function_call_interrupts
257098 +135.6% 605636 interrupts.CPU139.LOC:Local_timer_interrupts
257999 +135.1% 606552 interrupts.CPU14.LOC:Local_timer_interrupts
257879 +135.0% 605887 interrupts.CPU140.LOC:Local_timer_interrupts
0.25 ±173% +1e+05% 262.50 ± 25% interrupts.CPU140.NMI:Non-maskable_interrupts
0.25 ±173% +1e+05% 262.50 ± 25% interrupts.CPU140.PMI:Performance_monitoring_interrupts
257988 +134.8% 605783 interrupts.CPU141.LOC:Local_timer_interrupts
0.25 ±173% +1.8e+05% 457.75 ± 59% interrupts.CPU141.NMI:Non-maskable_interrupts
0.25 ±173% +1.8e+05% 457.75 ± 59% interrupts.CPU141.PMI:Performance_monitoring_interrupts
256928 +135.8% 605788 interrupts.CPU142.LOC:Local_timer_interrupts
257551 +135.0% 605360 interrupts.CPU143.LOC:Local_timer_interrupts
258028 +134.8% 605867 interrupts.CPU144.LOC:Local_timer_interrupts
258006 +134.7% 605541 interrupts.CPU145.LOC:Local_timer_interrupts
258024 +134.8% 605783 interrupts.CPU146.LOC:Local_timer_interrupts
257614 +135.0% 605470 interrupts.CPU147.LOC:Local_timer_interrupts
257820 +134.9% 605659 interrupts.CPU148.LOC:Local_timer_interrupts
257485 +135.3% 605741 interrupts.CPU149.LOC:Local_timer_interrupts
258573 +134.6% 606547 interrupts.CPU15.LOC:Local_timer_interrupts
257711 +134.8% 605001 interrupts.CPU150.LOC:Local_timer_interrupts
257968 +134.4% 604801 interrupts.CPU151.LOC:Local_timer_interrupts
258008 +134.6% 605227 interrupts.CPU152.LOC:Local_timer_interrupts
340.25 ± 55% -95.3% 16.00 ± 74% interrupts.CPU152.RES:Rescheduling_interrupts
258013 +135.3% 607224 interrupts.CPU153.LOC:Local_timer_interrupts
257749 +135.6% 607168 interrupts.CPU154.LOC:Local_timer_interrupts
258017 +134.8% 605925 interrupts.CPU155.LOC:Local_timer_interrupts
3.25 ±102% +2130.8% 72.50 ± 93% interrupts.CPU155.RES:Rescheduling_interrupts
257962 +136.2% 609340 interrupts.CPU156.LOC:Local_timer_interrupts
257730 +135.1% 605840 interrupts.CPU157.LOC:Local_timer_interrupts
257296 +135.3% 605306 interrupts.CPU158.LOC:Local_timer_interrupts
257269 +135.4% 605609 interrupts.CPU159.LOC:Local_timer_interrupts
258025 +135.0% 606370 interrupts.CPU16.LOC:Local_timer_interrupts
1842 ± 6% +17.6% 2166 ± 6% interrupts.CPU160.CAL:Function_call_interrupts
257442 +135.3% 605884 interrupts.CPU160.LOC:Local_timer_interrupts
257136 +135.6% 605841 interrupts.CPU161.LOC:Local_timer_interrupts
257820 +135.0% 605922 interrupts.CPU162.LOC:Local_timer_interrupts
95.25 ± 80% -89.2% 10.25 ± 68% interrupts.CPU162.RES:Rescheduling_interrupts
257981 +134.9% 605945 interrupts.CPU163.LOC:Local_timer_interrupts
0.50 ±173% +51000.0% 255.50 ± 24% interrupts.CPU163.NMI:Non-maskable_interrupts
0.50 ±173% +51000.0% 255.50 ± 24% interrupts.CPU163.PMI:Performance_monitoring_interrupts
1934 ± 13% +23.0% 2378 ± 8% interrupts.CPU164.CAL:Function_call_interrupts
257995 +134.9% 605908 interrupts.CPU164.LOC:Local_timer_interrupts
257989 +134.9% 605899 interrupts.CPU165.LOC:Local_timer_interrupts
257990 +134.7% 605509 interrupts.CPU166.LOC:Local_timer_interrupts
257814 +134.8% 605385 interrupts.CPU167.LOC:Local_timer_interrupts
257441 +135.4% 605963 interrupts.CPU168.LOC:Local_timer_interrupts
20.00 ± 30% +732.5% 166.50 ± 29% interrupts.CPU168.RES:Rescheduling_interrupts
257218 +135.7% 606154 interrupts.CPU169.LOC:Local_timer_interrupts
0.50 ±173% +1.2e+05% 597.50 ± 80% interrupts.CPU169.NMI:Non-maskable_interrupts
0.50 ±173% +1.2e+05% 597.50 ± 80% interrupts.CPU169.PMI:Performance_monitoring_interrupts
258040 +135.0% 606392 interrupts.CPU17.LOC:Local_timer_interrupts
257869 +135.1% 606194 interrupts.CPU170.LOC:Local_timer_interrupts
257719 +135.2% 606164 interrupts.CPU171.LOC:Local_timer_interrupts
1.50 ±173% +18966.7% 286.00 ± 25% interrupts.CPU171.NMI:Non-maskable_interrupts
1.50 ±173% +18966.7% 286.00 ± 25% interrupts.CPU171.PMI:Performance_monitoring_interrupts
257274 +135.6% 606183 interrupts.CPU172.LOC:Local_timer_interrupts
0.00 +3.1e+104% 306.50 ± 30% interrupts.CPU172.NMI:Non-maskable_interrupts
0.00 +3.1e+104% 306.50 ± 30% interrupts.CPU172.PMI:Performance_monitoring_interrupts
256796 +136.0% 606108 interrupts.CPU173.LOC:Local_timer_interrupts
0.50 ±100% +57050.0% 285.75 ± 25% interrupts.CPU173.NMI:Non-maskable_interrupts
0.50 ±100% +57050.0% 285.75 ± 25% interrupts.CPU173.PMI:Performance_monitoring_interrupts
255382 +137.3% 606147 interrupts.CPU174.LOC:Local_timer_interrupts
256716 +135.9% 605483 interrupts.CPU175.LOC:Local_timer_interrupts
0.50 ±100% +48500.0% 243.00 ± 34% interrupts.CPU175.NMI:Non-maskable_interrupts
0.50 ±100% +48500.0% 243.00 ± 34% interrupts.CPU175.PMI:Performance_monitoring_interrupts
257752 +135.1% 605873 interrupts.CPU176.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 304.25 ± 30% interrupts.CPU176.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 304.25 ± 30% interrupts.CPU176.PMI:Performance_monitoring_interrupts
257997 +135.0% 606185 interrupts.CPU177.LOC:Local_timer_interrupts
257592 +135.2% 605961 interrupts.CPU178.LOC:Local_timer_interrupts
257711 +135.2% 606176 interrupts.CPU179.LOC:Local_timer_interrupts
1.50 ±173% +18883.3% 284.75 ± 24% interrupts.CPU179.NMI:Non-maskable_interrupts
1.50 ±173% +18883.3% 284.75 ± 24% interrupts.CPU179.PMI:Performance_monitoring_interrupts
260122 ± 2% +135.2% 611875 interrupts.CPU18.LOC:Local_timer_interrupts
257975 +135.0% 606231 interrupts.CPU180.LOC:Local_timer_interrupts
257985 +135.0% 606187 interrupts.CPU181.LOC:Local_timer_interrupts
0.25 ±173% +1e+05% 261.00 ± 42% interrupts.CPU181.NMI:Non-maskable_interrupts
0.25 ±173% +1e+05% 261.00 ± 42% interrupts.CPU181.PMI:Performance_monitoring_interrupts
257981 +134.9% 605957 interrupts.CPU182.LOC:Local_timer_interrupts
258014 +134.9% 606058 interrupts.CPU183.LOC:Local_timer_interrupts
258003 +134.9% 606132 interrupts.CPU184.LOC:Local_timer_interrupts
257628 +135.3% 606131 interrupts.CPU185.LOC:Local_timer_interrupts
256870 +135.9% 605987 interrupts.CPU186.LOC:Local_timer_interrupts
257549 +135.2% 605680 interrupts.CPU187.LOC:Local_timer_interrupts
257981 +135.0% 606175 interrupts.CPU188.LOC:Local_timer_interrupts
1.25 ±173% +22280.0% 279.75 ± 25% interrupts.CPU188.NMI:Non-maskable_interrupts
1.25 ±173% +22280.0% 279.75 ± 25% interrupts.CPU188.PMI:Performance_monitoring_interrupts
257956 +135.0% 606126 interrupts.CPU189.LOC:Local_timer_interrupts
0.75 ±173% +41700.0% 313.50 ± 34% interrupts.CPU189.NMI:Non-maskable_interrupts
0.75 ±173% +41700.0% 313.50 ± 34% interrupts.CPU189.PMI:Performance_monitoring_interrupts
258147 +135.2% 607115 interrupts.CPU19.LOC:Local_timer_interrupts
1.00 ±122% +10250.0% 103.50 ± 86% interrupts.CPU19.RES:Rescheduling_interrupts
87.75 ± 47% -63.5% 32.00 ± 88% interrupts.CPU19.TLB:TLB_shootdowns
256298 +136.5% 606167 interrupts.CPU190.LOC:Local_timer_interrupts
257838 +135.1% 606204 interrupts.CPU191.LOC:Local_timer_interrupts
2.25 ± 72% +13733.3% 311.25 ± 26% interrupts.CPU191.NMI:Non-maskable_interrupts
2.25 ± 72% +13733.3% 311.25 ± 26% interrupts.CPU191.PMI:Performance_monitoring_interrupts
257840 +135.3% 606664 interrupts.CPU2.LOC:Local_timer_interrupts
258010 +135.1% 606461 interrupts.CPU20.LOC:Local_timer_interrupts
97.25 ± 31% -70.2% 29.00 ±109% interrupts.CPU20.TLB:TLB_shootdowns
258033 +134.7% 605617 interrupts.CPU21.LOC:Local_timer_interrupts
258053 +135.2% 607010 interrupts.CPU22.LOC:Local_timer_interrupts
258009 +135.1% 606487 interrupts.CPU23.LOC:Local_timer_interrupts
259088 +133.8% 605797 interrupts.CPU24.LOC:Local_timer_interrupts
29.00 ± 42% +427.6% 153.00 ± 32% interrupts.CPU24.RES:Rescheduling_interrupts
258265 +134.6% 605784 interrupts.CPU25.LOC:Local_timer_interrupts
258019 +134.8% 605798 interrupts.CPU26.LOC:Local_timer_interrupts
258509 +134.3% 605788 interrupts.CPU27.LOC:Local_timer_interrupts
0.25 ±173% +1.7e+05% 427.00 ± 73% interrupts.CPU27.NMI:Non-maskable_interrupts
0.25 ±173% +1.7e+05% 427.00 ± 73% interrupts.CPU27.PMI:Performance_monitoring_interrupts
258270 +134.6% 605794 interrupts.CPU28.LOC:Local_timer_interrupts
258015 +134.8% 605812 interrupts.CPU29.LOC:Local_timer_interrupts
0.25 ±173% +1e+05% 259.25 ± 23% interrupts.CPU29.NMI:Non-maskable_interrupts
0.25 ±173% +1e+05% 259.25 ± 23% interrupts.CPU29.PMI:Performance_monitoring_interrupts
258002 +135.0% 606425 interrupts.CPU3.LOC:Local_timer_interrupts
258053 +134.8% 605796 interrupts.CPU30.LOC:Local_timer_interrupts
0.25 ±173% +73500.0% 184.00 ± 32% interrupts.CPU30.NMI:Non-maskable_interrupts
0.25 ±173% +73500.0% 184.00 ± 32% interrupts.CPU30.PMI:Performance_monitoring_interrupts
258163 +134.7% 605804 interrupts.CPU31.LOC:Local_timer_interrupts
258086 +134.7% 605795 interrupts.CPU32.LOC:Local_timer_interrupts
258056 +134.7% 605784 interrupts.CPU33.LOC:Local_timer_interrupts
258071 +134.7% 605817 interrupts.CPU34.LOC:Local_timer_interrupts
258062 +134.8% 605813 interrupts.CPU35.LOC:Local_timer_interrupts
258033 +134.8% 605820 interrupts.CPU36.LOC:Local_timer_interrupts
258047 +134.8% 605794 interrupts.CPU37.LOC:Local_timer_interrupts
258035 +134.8% 605828 interrupts.CPU38.LOC:Local_timer_interrupts
0.25 ±173% +1.8e+05% 443.25 ± 57% interrupts.CPU38.NMI:Non-maskable_interrupts
0.25 ±173% +1.8e+05% 443.25 ± 57% interrupts.CPU38.PMI:Performance_monitoring_interrupts
258037 +134.8% 605808 interrupts.CPU39.LOC:Local_timer_interrupts
0.25 ±173% +1.8e+05% 448.50 ± 77% interrupts.CPU39.NMI:Non-maskable_interrupts
0.25 ±173% +1.8e+05% 448.50 ± 77% interrupts.CPU39.PMI:Performance_monitoring_interrupts
257903 +135.1% 606264 interrupts.CPU4.LOC:Local_timer_interrupts
0.25 ±173% +1e+05% 257.75 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts
0.25 ±173% +1e+05% 257.75 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts
257882 +134.9% 605785 interrupts.CPU40.LOC:Local_timer_interrupts
0.25 ±173% +2.3e+05% 574.25 ± 99% interrupts.CPU40.NMI:Non-maskable_interrupts
0.25 ±173% +2.3e+05% 574.25 ± 99% interrupts.CPU40.PMI:Performance_monitoring_interrupts
257998 +134.6% 605337 interrupts.CPU41.LOC:Local_timer_interrupts
48.75 ± 83% -86.2% 6.75 ±109% interrupts.CPU41.RES:Rescheduling_interrupts
258079 +134.7% 605611 interrupts.CPU42.LOC:Local_timer_interrupts
0.50 ±173% +52250.0% 261.75 ± 25% interrupts.CPU42.NMI:Non-maskable_interrupts
0.50 ±173% +52250.0% 261.75 ± 25% interrupts.CPU42.PMI:Performance_monitoring_interrupts
257555 +135.2% 605682 interrupts.CPU43.LOC:Local_timer_interrupts
258011 +134.8% 605819 interrupts.CPU44.LOC:Local_timer_interrupts
110.00 ± 58% -96.1% 4.25 ± 56% interrupts.CPU44.RES:Rescheduling_interrupts
258046 +134.7% 605702 interrupts.CPU45.LOC:Local_timer_interrupts
101.00 ± 89% -97.3% 2.75 ±110% interrupts.CPU45.RES:Rescheduling_interrupts
258036 +134.8% 605870 interrupts.CPU46.LOC:Local_timer_interrupts
258037 +134.7% 605737 interrupts.CPU47.LOC:Local_timer_interrupts
257967 +134.9% 605917 interrupts.CPU48.LOC:Local_timer_interrupts
257674 +135.2% 605935 interrupts.CPU49.LOC:Local_timer_interrupts
258062 +135.1% 606595 interrupts.CPU5.LOC:Local_timer_interrupts
258045 +134.9% 606025 interrupts.CPU50.LOC:Local_timer_interrupts
258020 +134.8% 605923 interrupts.CPU51.LOC:Local_timer_interrupts
257886 +135.0% 605918 interrupts.CPU52.LOC:Local_timer_interrupts
258018 +134.8% 605948 interrupts.CPU53.LOC:Local_timer_interrupts
258032 +134.8% 605884 interrupts.CPU54.LOC:Local_timer_interrupts
55.25 ± 76% -95.5% 2.50 ± 60% interrupts.CPU54.RES:Rescheduling_interrupts
257864 +135.0% 605919 interrupts.CPU55.LOC:Local_timer_interrupts
258008 +134.8% 605925 interrupts.CPU56.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 299.25 ± 4% interrupts.CPU56.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 299.25 ± 4% interrupts.CPU56.PMI:Performance_monitoring_interrupts
258009 +134.9% 605943 interrupts.CPU57.LOC:Local_timer_interrupts
257781 +135.1% 606003 interrupts.CPU58.LOC:Local_timer_interrupts
258019 +134.8% 605916 interrupts.CPU59.LOC:Local_timer_interrupts
257973 +135.1% 606373 interrupts.CPU6.LOC:Local_timer_interrupts
258077 +134.8% 605916 interrupts.CPU60.LOC:Local_timer_interrupts
257983 +134.8% 605737 interrupts.CPU61.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 293.00 ± 3% interrupts.CPU61.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 293.00 ± 3% interrupts.CPU61.PMI:Performance_monitoring_interrupts
260661 ± 2% +132.5% 605907 interrupts.CPU62.LOC:Local_timer_interrupts
258162 +134.7% 605888 interrupts.CPU63.LOC:Local_timer_interrupts
258056 +134.8% 605938 interrupts.CPU64.LOC:Local_timer_interrupts
257625 +135.2% 605921 interrupts.CPU65.LOC:Local_timer_interrupts
2005 ± 9% +51.1% 3030 ± 44% interrupts.CPU66.CAL:Function_call_interrupts
257972 +134.9% 605902 interrupts.CPU66.LOC:Local_timer_interrupts
257840 +135.0% 605920 interrupts.CPU67.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 291.25 ± 2% interrupts.CPU67.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 291.25 ± 2% interrupts.CPU67.PMI:Performance_monitoring_interrupts
255873 +136.8% 605914 interrupts.CPU68.LOC:Local_timer_interrupts
257918 +134.9% 605936 interrupts.CPU69.LOC:Local_timer_interrupts
257750 +135.5% 607125 interrupts.CPU7.LOC:Local_timer_interrupts
258004 +134.8% 605804 interrupts.CPU70.LOC:Local_timer_interrupts
1865 ± 11% +16.2% 2167 ± 7% interrupts.CPU71.CAL:Function_call_interrupts
260658 ± 2% +132.5% 605957 interrupts.CPU71.LOC:Local_timer_interrupts
257805 +135.0% 605950 interrupts.CPU72.LOC:Local_timer_interrupts
258043 +134.9% 606168 interrupts.CPU73.LOC:Local_timer_interrupts
0.25 ±173% +9.2e+05% 2310 ±153% interrupts.CPU73.NMI:Non-maskable_interrupts
0.25 ±173% +9.2e+05% 2310 ±153% interrupts.CPU73.PMI:Performance_monitoring_interrupts
258016 +134.9% 606153 interrupts.CPU74.LOC:Local_timer_interrupts
258012 +135.0% 606224 interrupts.CPU75.LOC:Local_timer_interrupts
258034 +134.9% 606203 interrupts.CPU76.LOC:Local_timer_interrupts
257998 +135.0% 606202 interrupts.CPU77.LOC:Local_timer_interrupts
0.25 ±173% +1.1e+05% 286.75 ± 26% interrupts.CPU77.NMI:Non-maskable_interrupts
0.25 ±173% +1.1e+05% 286.75 ± 26% interrupts.CPU77.PMI:Performance_monitoring_interrupts
257982 +135.0% 606209 interrupts.CPU78.LOC:Local_timer_interrupts
258005 +135.0% 606194 interrupts.CPU79.LOC:Local_timer_interrupts
258005 +135.1% 606517 interrupts.CPU8.LOC:Local_timer_interrupts
257963 +135.0% 606187 interrupts.CPU80.LOC:Local_timer_interrupts
258018 +134.9% 606189 interrupts.CPU81.LOC:Local_timer_interrupts
258015 +135.0% 606248 interrupts.CPU82.LOC:Local_timer_interrupts
258001 +135.0% 606206 interrupts.CPU83.LOC:Local_timer_interrupts
258067 +134.9% 606176 interrupts.CPU84.LOC:Local_timer_interrupts
258026 +134.9% 606002 interrupts.CPU85.LOC:Local_timer_interrupts
1941 ± 12% +51.5% 2940 ± 24% interrupts.CPU86.CAL:Function_call_interrupts
258049 +134.9% 606232 interrupts.CPU86.LOC:Local_timer_interrupts
258021 +134.9% 606195 interrupts.CPU87.LOC:Local_timer_interrupts
258025 +134.9% 606192 interrupts.CPU88.LOC:Local_timer_interrupts
258019 +135.0% 606238 interrupts.CPU89.LOC:Local_timer_interrupts
257891 +135.2% 606490 interrupts.CPU9.LOC:Local_timer_interrupts
258039 +134.9% 606176 interrupts.CPU90.LOC:Local_timer_interrupts
257929 +135.0% 606125 interrupts.CPU91.LOC:Local_timer_interrupts
258045 +134.9% 606114 interrupts.CPU92.LOC:Local_timer_interrupts
256784 +136.0% 606132 interrupts.CPU93.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 307.50 ± 32% interrupts.CPU93.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 307.50 ± 32% interrupts.CPU93.PMI:Performance_monitoring_interrupts
255472 +137.3% 606111 interrupts.CPU94.LOC:Local_timer_interrupts
257726 +135.1% 606036 interrupts.CPU95.LOC:Local_timer_interrupts
0.25 ±173% +1.2e+05% 305.25 ± 28% interrupts.CPU95.NMI:Non-maskable_interrupts
0.25 ±173% +1.2e+05% 305.25 ± 28% interrupts.CPU95.PMI:Performance_monitoring_interrupts
257710 +135.1% 605816 interrupts.CPU96.LOC:Local_timer_interrupts
18.25 ± 62% +352.1% 82.50 ±104% interrupts.CPU96.RES:Rescheduling_interrupts
257967 +135.2% 606640 interrupts.CPU97.LOC:Local_timer_interrupts
0.50 ±100% +60400.0% 302.50 ± 19% interrupts.CPU97.NMI:Non-maskable_interrupts
0.50 ±100% +60400.0% 302.50 ± 19% interrupts.CPU97.PMI:Performance_monitoring_interrupts
257967 +135.1% 606427 interrupts.CPU98.LOC:Local_timer_interrupts
258011 +135.0% 606237 interrupts.CPU99.LOC:Local_timer_interrupts
49504415 +135.1% 1.164e+08 interrupts.LOC:Local_timer_interrupts
0.00 +1.9e+104% 192.00 interrupts.MCP:Machine_check_polls
17.50 ± 24% +8.5e+05% 148487 ± 7% interrupts.NMI:Non-maskable_interrupts
17.50 ± 24% +8.5e+05% 148487 ± 7% interrupts.PMI:Performance_monitoring_interrupts
10597 ± 10% -46.8% 5641 ± 2% interrupts.RES:Rescheduling_interrupts
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.syscall_exit_to_user_mode
15.60 ± 87% -15.6 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare
13.10 ±116% -13.1 0.00 perf-profile.calltrace.cycles-pp.task_work_run.do_exit.do_group_exit.get_signal.arch_do_signal_or_restart
10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.seq_read_iter.proc_reg_read_iter.new_sync_read.vfs_read.ksys_read
10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.41 ± 93% -10.4 0.00 perf-profile.calltrace.cycles-pp.proc_reg_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
10.09 ±143% -10.1 0.00 perf-profile.calltrace.cycles-pp.__fput.task_work_run.do_exit.do_group_exit.get_signal
9.02 ±118% -9.0 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.02 ±118% -9.0 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.00 ±139% -9.0 0.00 perf-profile.calltrace.cycles-pp.perf_release.__fput.task_work_run.do_exit.do_group_exit
9.00 ±139% -9.0 0.00 perf-profile.calltrace.cycles-pp.perf_event_release_kernel.perf_release.__fput.task_work_run.do_exit
7.15 ± 68% -7.1 0.00 perf-profile.calltrace.cycles-pp.show_interrupts.seq_read_iter.proc_reg_read_iter.new_sync_read.vfs_read
6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.smp_call_function_single.event_function_call.perf_remove_from_context.perf_event_release_kernel.perf_release
6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.event_function_call.perf_remove_from_context.perf_event_release_kernel.perf_release.__fput
6.82 ±130% -6.8 0.00 perf-profile.calltrace.cycles-pp.perf_remove_from_context.perf_event_release_kernel.perf_release.__fput.task_work_run
6.39 ±127% -6.4 0.00 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
6.39 ±127% -6.4 0.00 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
6.34 ±109% -6.3 0.00 perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.34 ±109% -6.3 0.00 perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.__libc_start_main
4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.main.__libc_start_main
4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.run_builtin.main.__libc_start_main
4.10 ±100% -4.1 0.00 perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.__libc_start_main
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
3.89 ±107% -3.9 0.00 perf-profile.calltrace.cycles-pp.read
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat.do_syscall_64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.lookup_fast.walk_component.path_lookupat.filename_lookup.vfs_statx
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.vfs_statx.__do_sys_newlstat.do_syscall_64.entry_SYSCALL_64_after_hwframe.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.calltrace.cycles-pp.walk_component.path_lookupat.filename_lookup.vfs_statx.__do_sys_newlstat
0.00 +0.6 0.55 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu
0.00 +0.6 0.62 ± 6% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
0.00 +0.6 0.63 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp._find_next_bit.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes
0.00 +0.7 0.73 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
0.00 +0.7 0.74 ± 5% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.8 0.79 ± 6% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +0.8 0.81 ± 11% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary
0.00 +0.8 0.85 ± 12% perf-profile.calltrace.cycles-pp.pagecache_get_page.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.00 +0.9 0.87 ± 12% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack
0.00 +0.9 0.87 ± 12% perf-profile.calltrace.cycles-pp.prepare_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
0.00 +0.9 0.88 ± 8% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack
0.00 +0.9 0.93 ± 18% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write
0.00 +0.9 0.94 ± 18% perf-profile.calltrace.cycles-pp.copyin.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter
0.00 +0.9 0.95 ± 18% perf-profile.calltrace.cycles-pp.iov_iter_copy_from_user_atomic.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.00 +1.0 0.98 ± 17% perf-profile.calltrace.cycles-pp.btrfs_copy_from_user.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
0.00 +1.0 1.05 ± 10% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt
0.00 +1.1 1.07 ± 32% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
0.00 +1.2 1.15 ± 11% perf-profile.calltrace.cycles-pp.cpumask_next.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata
0.00 +1.2 1.22 ± 6% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt
0.00 +1.3 1.29 ± 13% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
0.00 +1.6 1.61 ± 32% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode
0.00 +2.1 2.05 ± 32% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work.worker_thread
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn.process_one_work
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb.wb_writeback.wb_workfn
0.00 +2.1 2.06 ± 32% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.__writeback_inodes_wb
0.00 +2.2 2.25 ± 28% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
0.00 +2.2 2.25 ± 28% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
0.00 +2.3 2.34 ± 28% perf-profile.calltrace.cycles-pp.ret_from_fork
0.00 +2.3 2.34 ± 28% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
0.00 +2.8 2.81 ± 7% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.8 2.83 ± 7% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +2.8 2.84 ± 7% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +4.0 4.00 ± 13% perf-profile.calltrace.cycles-pp.__percpu_counter_sum.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write
1.09 ±173% +4.1 5.15 ± 5% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
1.09 ±173% +8.1 9.19 ± 4% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.00 +11.6 11.61 ± 5% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +14.8 14.81 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata
0.00 +15.1 15.06 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write
0.00 +17.0 17.02 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write
0.00 +17.2 17.20 ± 12% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter
0.00 +17.3 17.31 ± 12% perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.00 +17.3 17.34 ± 12% perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
0.00 +19.6 19.56 ± 11% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
0.00 +19.6 19.56 ± 11% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
0.00 +19.9 19.86 ± 11% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
15.69 ± 61% +26.3 42.00 ± 12% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +41.7 41.71 ± 12% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write
0.00 +41.8 41.82 ± 12% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.00 +41.8 41.84 ± 12% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +41.9 41.93 ± 12% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +42.0 41.96 ± 12% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
19.48 ± 62% -19.5 0.00 perf-profile.children.cycles-pp.do_group_exit
19.48 ± 62% -19.5 0.00 perf-profile.children.cycles-pp.do_exit
16.98 ± 75% -17.0 0.00 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
16.98 ± 75% -17.0 0.00 perf-profile.children.cycles-pp.exit_to_user_mode_prepare
15.60 ± 87% -15.6 0.00 perf-profile.children.cycles-pp.arch_do_signal_or_restart
15.60 ± 87% -15.6 0.00 perf-profile.children.cycles-pp.get_signal
14.48 ±102% -14.5 0.00 perf-profile.children.cycles-pp.task_work_run
12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.ksys_read
12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.vfs_read
12.91 ± 81% -12.9 0.00 perf-profile.children.cycles-pp.seq_read_iter
11.47 ±123% -11.5 0.00 perf-profile.children.cycles-pp.__fput
10.41 ± 93% -10.4 0.00 perf-profile.children.cycles-pp.new_sync_read
10.41 ± 93% -10.4 0.00 perf-profile.children.cycles-pp.proc_reg_read_iter
9.00 ±139% -9.0 0.00 perf-profile.children.cycles-pp.perf_release
9.00 ±139% -9.0 0.00 perf-profile.children.cycles-pp.perf_event_release_kernel
7.15 ± 68% -7.1 0.00 perf-profile.children.cycles-pp.show_interrupts
6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.smp_call_function_single
6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.event_function_call
6.82 ±130% -6.8 0.00 perf-profile.children.cycles-pp.perf_remove_from_context
6.39 ±127% -6.4 0.00 perf-profile.children.cycles-pp.schedule_idle
6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_sys_open
6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_sys_openat2
6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.do_filp_open
6.34 ±109% -6.3 0.00 perf-profile.children.cycles-pp.path_openat
4.70 ±103% -4.7 0.00 perf-profile.children.cycles-pp.lookup_fast
4.70 ±103% -4.7 0.00 perf-profile.children.cycles-pp.walk_component
4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.__libc_start_main
4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.main
4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.run_builtin
4.10 ±100% -4.1 0.00 perf-profile.children.cycles-pp.cmd_record
3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.__x64_sys_exit_group
3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.mmput
3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.exit_mmap
3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.pick_next_task_fair
3.89 ±107% -3.9 0.00 perf-profile.children.cycles-pp.read
3.56 ±104% -3.6 0.00 perf-profile.children.cycles-pp.seq_printf
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__do_sys_newlstat
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__d_lookup
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.filename_lookup
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.path_lookupat
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.__lxstat64
3.31 ±102% -3.3 0.00 perf-profile.children.cycles-pp.vfs_statx
0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.dec_zone_page_state
0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.btrfs_write_check
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.rcu_idle_exit
0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.run_rebalance_domains
0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.free_extent_state
0.00 +0.1 0.07 ± 21% perf-profile.children.cycles-pp.merge_state
0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.__mod_lruvec_page_state
0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.__mod_node_page_state
0.00 +0.1 0.09 ± 26% perf-profile.children.cycles-pp.crc_24
0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.update_rq_clock
0.00 +0.1 0.09 ± 23% perf-profile.children.cycles-pp.find_get_entry
0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.btrfs_transaction_in_commit
0.00 +0.1 0.09 ± 20% perf-profile.children.cycles-pp.calc_global_load_tick
0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.__mod_memcg_state
0.00 +0.1 0.10 ± 23% perf-profile.children.cycles-pp.crc_32
0.00 +0.1 0.10 ± 23% perf-profile.children.cycles-pp.rmqueue
0.00 +0.1 0.10 ± 24% perf-profile.children.cycles-pp.kernel_fpu_begin
0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.__mod_lruvec_state
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.blk_mq_submit_bio
0.00 +0.1 0.10 ± 15% perf-profile.children.cycles-pp.account_process_tick
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.run_one_async_done
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.btrfs_map_bio
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.submit_bio
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.submit_bio_noacct
0.00 +0.1 0.11 ± 25% perf-profile.children.cycles-pp.node_dirty_ok
0.00 +0.1 0.11 ± 33% perf-profile.children.cycles-pp.crc_41
0.00 +0.1 0.11 ± 23% perf-profile.children.cycles-pp.__crc32c_pcl_intel_finup
0.00 +0.1 0.11 ± 11% perf-profile.children.cycles-pp.clear_state_bit
0.00 +0.1 0.12 ± 13% perf-profile.children.cycles-pp.native_sched_clock
0.00 +0.1 0.12 ± 11% perf-profile.children.cycles-pp.update_sd_lb_stats
0.00 +0.1 0.12 ± 14% perf-profile.children.cycles-pp.__pagevec_lru_add
0.00 +0.1 0.12 ± 15% perf-profile.children.cycles-pp.sched_clock
0.00 +0.1 0.12 ± 34% perf-profile.children.cycles-pp.btrfs_find_delalloc_range
0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.account_page_dirtied
0.00 +0.1 0.12 ± 12% perf-profile.children.cycles-pp.find_busiest_group
0.00 +0.1 0.13 ± 29% perf-profile.children.cycles-pp._raw_read_lock
0.00 +0.1 0.13 ± 35% perf-profile.children.cycles-pp.crc_96
0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.btrfs_run_delalloc_range
0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.run_delalloc_nocow
0.00 +0.1 0.13 ± 31% perf-profile.children.cycles-pp.find_get_pages_contig
0.00 +0.1 0.14 ± 6% perf-profile.children.cycles-pp.lock_extent_bits
0.00 +0.1 0.14 ± 11% perf-profile.children.cycles-pp.lru_cache_add
0.00 +0.1 0.14 ± 10% perf-profile.children.cycles-pp.sched_clock_cpu
0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.__lookup_extent_mapping
0.00 +0.2 0.15 ± 35% perf-profile.children.cycles-pp.lock_delalloc_pages
0.00 +0.2 0.15 ± 11% perf-profile.children.cycles-pp.load_balance
0.00 +0.2 0.15 ± 32% perf-profile.children.cycles-pp.btrfs_get_chunk_map
0.00 +0.2 0.15 ± 9% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.2 0.16 ± 17% perf-profile.children.cycles-pp.btrfs_drop_pages
0.00 +0.2 0.16 ± 14% perf-profile.children.cycles-pp.mem_cgroup_charge
0.00 +0.2 0.16 ± 23% perf-profile.children.cycles-pp.crc_72
0.00 +0.2 0.16 ± 6% perf-profile.children.cycles-pp.read_tsc
0.00 +0.2 0.16 ± 11% perf-profile.children.cycles-pp.find_next_bit
0.00 +0.2 0.17 ± 8% perf-profile.children.cycles-pp.lapic_next_deadline
0.00 +0.2 0.17 ± 13% perf-profile.children.cycles-pp.btrfs_work_helper
0.00 +0.2 0.18 ± 10% perf-profile.children.cycles-pp.xas_load
0.00 +0.2 0.19 ± 18% perf-profile.children.cycles-pp.btrfs_get_alloc_profile
0.00 +0.2 0.21 ± 29% perf-profile.children.cycles-pp.btrfs_get_io_geometry
0.00 +0.2 0.21 ± 17% perf-profile.children.cycles-pp.calc_available_free_space
0.00 +0.2 0.22 ± 29% perf-profile.children.cycles-pp.btrfs_bio_fits_in_stripe
0.00 +0.2 0.23 ± 20% perf-profile.children.cycles-pp.btrfs_get_extent
0.00 +0.2 0.23 ± 6% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.2 0.23 ± 11% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need
0.00 +0.2 0.25 ± 29% perf-profile.children.cycles-pp.crc_42
0.00 +0.3 0.26 ± 13% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.00 +0.3 0.27 ± 25% perf-profile.children.cycles-pp.get_page_from_freelist
0.00 +0.3 0.27 ± 29% perf-profile.children.cycles-pp.submit_extent_page
0.00 +0.3 0.28 ± 33% perf-profile.children.cycles-pp.__process_pages_contig
0.00 +0.3 0.29 ± 32% perf-profile.children.cycles-pp.find_lock_delalloc_range
0.00 +0.3 0.29 ± 22% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.00 +0.3 0.30 ± 35% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.00 +0.3 0.30 ± 37% perf-profile.children.cycles-pp.test_clear_page_writeback
0.00 +0.3 0.31 ± 24% perf-profile.children.cycles-pp.percpu_counter_add_batch
0.00 +0.3 0.32 ± 14% perf-profile.children.cycles-pp.kmem_cache_alloc
0.00 +0.3 0.33 ± 8% perf-profile.children.cycles-pp.scheduler_tick
0.00 +0.3 0.33 ± 38% perf-profile.children.cycles-pp.end_page_writeback
0.00 +0.3 0.34 ± 4% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +0.4 0.36 ± 13% perf-profile.children.cycles-pp.btrfs_set_delalloc_extent
0.00 +0.4 0.37 ± 18% perf-profile.children.cycles-pp.alloc_extent_state
0.00 +0.4 0.37 ± 13% perf-profile.children.cycles-pp.set_state_bits
0.00 +0.4 0.37 ± 11% perf-profile.children.cycles-pp.btrfs_reserve_data_bytes
0.00 +0.4 0.38 ± 11% perf-profile.children.cycles-pp.__add_to_page_cache_locked
0.00 +0.4 0.38 ± 10% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.00 +0.4 0.41 ± 8% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.00 +0.4 0.42 ± 35% perf-profile.children.cycles-pp.__test_set_page_writeback
0.00 +0.4 0.42 ± 26% perf-profile.children.cycles-pp.crc_128
0.00 +0.4 0.42 ± 10% perf-profile.children.cycles-pp.btrfs_check_data_free_space
0.00 +0.4 0.43 ± 32% perf-profile.children.cycles-pp.writepage_delalloc
0.00 +0.4 0.43 ± 16% perf-profile.children.cycles-pp.clear_extent_bit
0.00 +0.4 0.43 ± 11% perf-profile.children.cycles-pp.irqtime_account_irq
0.00 +0.5 0.50 ± 12% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +0.5 0.51 ± 14% perf-profile.children.cycles-pp.btrfs_set_extent_delalloc
0.00 +0.5 0.52 ± 13% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.00 +0.5 0.53 ± 10% perf-profile.children.cycles-pp.add_to_page_cache_lru
0.00 +0.5 0.53 ± 35% perf-profile.children.cycles-pp.btrfs_set_range_writeback
0.00 +0.6 0.55 ± 11% perf-profile.children.cycles-pp.set_extent_bit
0.00 +0.6 0.55 ± 4% perf-profile.children.cycles-pp.rebalance_domains
0.00 +0.6 0.62 ± 12% perf-profile.children.cycles-pp.__clear_extent_bit
0.00 +0.7 0.66 ±117% perf-profile.children.cycles-pp.start_kernel
0.00 +0.7 0.69 ± 6% perf-profile.children.cycles-pp.update_process_times
0.00 +0.7 0.70 ± 7% perf-profile.children.cycles-pp.tick_sched_handle
0.00 +0.7 0.73 ± 12% perf-profile.children.cycles-pp.tick_nohz_next_event
0.00 +0.8 0.79 ± 5% perf-profile.children.cycles-pp.tick_irq_enter
0.00 +0.8 0.80 ± 5% perf-profile.children.cycles-pp.irq_enter_rcu
0.00 +0.8 0.81 ± 11% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.9 0.87 ± 12% perf-profile.children.cycles-pp.prepare_pages
0.00 +0.9 0.93 ± 11% perf-profile.children.cycles-pp.pagecache_get_page
0.00 +0.9 0.94 ± 18% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.00 +0.9 0.95 ± 18% perf-profile.children.cycles-pp.copyin
0.00 +1.0 0.96 ± 18% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
0.00 +1.0 0.96 ± 8% perf-profile.children.cycles-pp.tick_sched_timer
0.00 +1.0 0.98 ± 17% perf-profile.children.cycles-pp.btrfs_copy_from_user
0.00 +1.1 1.08 ± 32% perf-profile.children.cycles-pp.__extent_writepage_io
0.00 +1.1 1.09 ± 10% perf-profile.children.cycles-pp.clockevents_program_event
0.00 +1.3 1.29 ± 13% perf-profile.children.cycles-pp.btrfs_dirty_pages
0.00 +1.3 1.33 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.00 +1.6 1.62 ± 32% perf-profile.children.cycles-pp.__extent_writepage
0.00 +2.1 2.05 ± 32% perf-profile.children.cycles-pp.extent_write_cache_pages
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.wb_workfn
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.wb_writeback
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.writeback_sb_inodes
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.__writeback_single_inode
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.extent_writepages
0.00 +2.1 2.06 ± 32% perf-profile.children.cycles-pp.do_writepages
0.00 +2.2 2.25 ± 28% perf-profile.children.cycles-pp.process_one_work
0.00 +2.2 2.25 ± 28% perf-profile.children.cycles-pp.worker_thread
0.00 +2.3 2.29 ± 10% perf-profile.children.cycles-pp.ktime_get
0.00 +2.3 2.34 ± 28% perf-profile.children.cycles-pp.kthread
0.00 +2.3 2.34 ± 28% perf-profile.children.cycles-pp.ret_from_fork
0.00 +3.0 2.97 ± 7% perf-profile.children.cycles-pp.hrtimer_interrupt
0.00 +3.0 3.01 ± 7% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.00 +4.0 4.02 ± 13% perf-profile.children.cycles-pp.__percpu_counter_sum
1.09 ±173% +4.3 5.36 ± 5% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.00 +11.7 11.67 ± 5% perf-profile.children.cycles-pp.menu_select
0.00 +17.3 17.33 ± 12% perf-profile.children.cycles-pp.btrfs_block_rsv_release
0.00 +17.3 17.34 ± 12% perf-profile.children.cycles-pp.btrfs_inode_rsv_release
0.00 +19.6 19.56 ± 11% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
0.00 +19.9 19.85 ± 11% perf-profile.children.cycles-pp.__reserve_bytes
0.00 +19.9 19.86 ± 11% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.00 +32.0 31.95 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.00 +33.4 33.40 ± 11% perf-profile.children.cycles-pp._raw_spin_lock
0.00 +41.7 41.73 ± 12% perf-profile.children.cycles-pp.btrfs_buffered_write
0.00 +41.8 41.82 ± 12% perf-profile.children.cycles-pp.btrfs_file_write_iter
0.00 +41.9 41.85 ± 12% perf-profile.children.cycles-pp.new_sync_write
0.00 +41.9 41.95 ± 12% perf-profile.children.cycles-pp.vfs_write
0.00 +42.0 41.98 ± 12% perf-profile.children.cycles-pp.ksys_write
6.82 ±130% -6.8 0.00 perf-profile.self.cycles-pp.smp_call_function_single
4.67 ±100% -4.7 0.00 perf-profile.self.cycles-pp.show_interrupts
3.31 ±102% -3.3 0.00 perf-profile.self.cycles-pp.__d_lookup
0.00 +0.1 0.05 perf-profile.self.cycles-pp.dec_zone_page_state
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rebalance_domains
0.00 +0.1 0.06 ± 15% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.free_extent_state
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.1 0.07 ± 10% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.00 +0.1 0.07 ± 19% perf-profile.self.cycles-pp.btrfs_get_alloc_profile
0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.btrfs_dirty_pages
0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.00 +0.1 0.08 ± 23% perf-profile.self.cycles-pp.test_clear_page_writeback
0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.btrfs_get_extent
0.00 +0.1 0.08 ± 15% perf-profile.self.cycles-pp.do_idle
0.00 +0.1 0.08 ± 15% perf-profile.self.cycles-pp.btrfs_reserve_data_bytes
0.00 +0.1 0.08 ± 21% perf-profile.self.cycles-pp.__mod_lruvec_page_state
0.00 +0.1 0.08 ± 10% perf-profile.self.cycles-pp.update_sd_lb_stats
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.__mod_node_page_state
0.00 +0.1 0.08 ± 26% perf-profile.self.cycles-pp.crc_24
0.00 +0.1 0.09 ± 17% perf-profile.self.cycles-pp.btrfs_file_write_iter
0.00 +0.1 0.09 ± 14% perf-profile.self.cycles-pp.__mod_memcg_state
0.00 +0.1 0.09 ± 20% perf-profile.self.cycles-pp.calc_global_load_tick
0.00 +0.1 0.10 ± 28% perf-profile.self.cycles-pp.crc_32
0.00 +0.1 0.10 ± 23% perf-profile.self.cycles-pp.kernel_fpu_begin
0.00 +0.1 0.10 ± 28% perf-profile.self.cycles-pp.clear_page_dirty_for_io
0.00 +0.1 0.10 ± 17% perf-profile.self.cycles-pp.btrfs_set_delalloc_extent
0.00 +0.1 0.10 ± 15% perf-profile.self.cycles-pp.account_process_tick
0.00 +0.1 0.11 ± 32% perf-profile.self.cycles-pp.__test_set_page_writeback
0.00 +0.1 0.11 ± 11% perf-profile.self.cycles-pp.native_sched_clock
0.00 +0.1 0.11 ± 32% perf-profile.self.cycles-pp.__process_pages_contig
0.00 +0.1 0.12 ± 19% perf-profile.self.cycles-pp.__clear_extent_bit
0.00 +0.1 0.12 ± 21% perf-profile.self.cycles-pp.tick_nohz_next_event
0.00 +0.1 0.13 ± 29% perf-profile.self.cycles-pp._raw_read_lock
0.00 +0.1 0.13 ± 35% perf-profile.self.cycles-pp.crc_96
0.00 +0.1 0.14 ± 8% perf-profile.self.cycles-pp.xas_load
0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.__lookup_extent_mapping
0.00 +0.1 0.14 ± 18% perf-profile.self.cycles-pp.update_process_times
0.00 +0.1 0.14 ± 15% perf-profile.self.cycles-pp.btrfs_drop_pages
0.00 +0.2 0.15 ± 8% perf-profile.self.cycles-pp.read_tsc
0.00 +0.2 0.16 ± 23% perf-profile.self.cycles-pp.crc_72
0.00 +0.2 0.16 ± 11% perf-profile.self.cycles-pp.find_next_bit
0.00 +0.2 0.17 ± 8% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.2 0.25 ± 28% perf-profile.self.cycles-pp.crc_42
0.00 +0.3 0.26 ± 17% perf-profile.self.cycles-pp.kmem_cache_alloc
0.00 +0.3 0.27 ± 26% perf-profile.self.cycles-pp.percpu_counter_add_batch
0.00 +0.3 0.30 ± 8% perf-profile.self.cycles-pp.__reserve_bytes
0.00 +0.3 0.34 ± 4% perf-profile.self.cycles-pp.native_irq_return_iret
0.00 +0.4 0.38 ± 13% perf-profile.self.cycles-pp.irqtime_account_irq
0.00 +0.4 0.38 ± 10% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.00 +0.4 0.41 ± 12% perf-profile.self.cycles-pp.cpumask_next
0.00 +0.4 0.42 ± 27% perf-profile.self.cycles-pp.crc_128
0.00 +0.5 0.47 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.5 0.52 ± 13% perf-profile.self.cycles-pp.timekeeping_max_deferment
0.00 +0.9 0.94 ± 18% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.00 +1.5 1.47 ± 10% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +2.2 2.17 ± 10% perf-profile.self.cycles-pp.ktime_get
0.00 +2.5 2.47 ± 15% perf-profile.self.cycles-pp.__percpu_counter_sum
0.00 +7.5 7.47 ± 9% perf-profile.self.cycles-pp.cpuidle_enter_state
0.00 +10.8 10.81 ± 5% perf-profile.self.cycles-pp.menu_select
0.00 +31.8 31.83 ± 11% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
***************************************************************************************************
lkp-cpl-4sp1: 144 threads Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/1BRD_48G/btrfs/x86_64-rhel-8.3/4/debian-10.4-x86_64-20200603.cgz/lkp-cpl-4sp1/creat-clo/aim7/0x700001e
commit:
5b964115b5 ("btrfs: rework btrfs_calc_reclaim_metadata_size")
53c5e5a1d8 ("btrfs: simplify the logic in need_preemptive_flushing")
5b964115b57bfe55 53c5e5a1d87e3341d06a00746c7
---------------- ---------------------------
%stddev %change %stddev
\ | \
2188 -3.5% 2112 aim7.jobs-per-min
10.99 +3.7% 11.40 aim7.time.elapsed_time
10.99 +3.7% 11.40 aim7.time.elapsed_time.max
2735 ± 3% +3.9% 2841 aim7.time.maximum_resident_set_size
240926 ± 5% -99.1% 2250 ± 3% aim7.time.voluntary_context_switches
1.99 +10.9% 2.20 ± 12% iostat.cpu.system
0.61 ± 2% +0.0 0.66 ± 4% mpstat.cpu.all.sys%
220726 ± 23% +69.2% 373544 ± 16% softirqs.RCU
37875 ± 4% -89.6% 3955 ± 2% vmstat.system.cs
249812 ± 8% -88.5% 28754 ± 11% cpuidle.POLL.time
196378 ± 6% -96.4% 6990 ± 3% cpuidle.POLL.usage
5390 ± 9% +21.0% 6521 ± 9% numa-meminfo.node1.KernelStack
70376 ± 17% -20.5% 55945 ± 12% numa-meminfo.node3.SUnreclaim
5412 ± 8% +20.4% 6514 ± 9% numa-vmstat.node1.nr_kernel_stack
17593 ± 17% -20.5% 13987 ± 12% numa-vmstat.node3.nr_slab_unreclaimable
13523 ± 6% -77.7% 3022 ± 3% perf-stat.i.context-switches
12463 ± 7% -77.7% 2783 ± 3% perf-stat.ps.context-switches
716.75 +5.0% 752.25 ± 3% proc-vmstat.nr_active_file
716.75 +5.0% 752.25 ± 3% proc-vmstat.nr_zone_active_file
21939 ± 44% -75.9% 5285 ± 29% sched_debug.cfs_rq:/.load_avg.max
2732 ± 25% -51.6% 1323 ± 12% sched_debug.cfs_rq:/.load_avg.stddev
6380 ± 74% +209.7% 19761 ± 28% sched_debug.cfs_rq:/.spread0.max
160664 ± 4% -14.1% 137974 ± 7% interrupts.CAL:Function_call_interrupts
1054 ± 24% +42.9% 1506 ± 25% interrupts.CPU1.CAL:Function_call_interrupts
911.50 +12.3% 1023 ± 6% interrupts.CPU104.CAL:Function_call_interrupts
1795 ± 29% -43.4% 1016 ± 10% interrupts.CPU2.CAL:Function_call_interrupts
776.50 ± 10% -58.9% 319.50 ± 34% interrupts.RES:Rescheduling_interrupts
0.01 ± 89% +365.1% 0.05 ± 41% perf-sched.sch_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
0.02 ±109% +354.5% 0.08 ± 10% perf-sched.sch_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
0.01 ± 23% +11226.9% 0.74 ±171% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.03 ± 65% +3965.6% 1.27 ±163% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.81 ± 25% -28.8% 0.58 ± 15% perf-sched.total_wait_time.average.ms
1681 ± 3% -8.0% 1547 ± 8% slabinfo.files_cache.active_objs
1681 ± 3% -8.0% 1547 ± 8% slabinfo.files_cache.num_objs
3846 ± 9% -16.7% 3203 ± 5% slabinfo.fsnotify_mark_connector.active_objs
3846 ± 9% -16.7% 3203 ± 5% slabinfo.fsnotify_mark_connector.num_objs
1903 ± 4% -19.5% 1532 ± 13% slabinfo.khugepaged_mm_slot.active_objs
1903 ± 4% -19.5% 1532 ± 13% slabinfo.khugepaged_mm_slot.num_objs
10.00 ±122% -8.6 1.35 ±173% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
6.25 ±103% -5.6 0.68 ±173% perf-profile.calltrace.cycles-pp.read
5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.new_sync_read.vfs_read
5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
5.00 ±100% -5.0 0.00 perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read
3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel
3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel
3.59 ±106% -1.1 2.50 ±173% perf-profile.calltrace.cycles-pp.__open64_nocancel
11.25 ±110% -9.2 2.03 ±173% perf-profile.children.cycles-pp.ksys_read
11.25 ±110% -9.2 2.03 ±173% perf-profile.children.cycles-pp.vfs_read
10.00 ±122% -8.6 1.35 ±173% perf-profile.children.cycles-pp.new_sync_read
6.25 ±103% -5.6 0.68 ±173% perf-profile.children.cycles-pp.read
5.00 ±100% -5.0 0.00 perf-profile.children.cycles-pp.pipe_read
5.00 ±100% -4.3 0.68 ±173% perf-profile.children.cycles-pp.schedule
5.00 ±100% -3.6 1.35 ±173% perf-profile.children.cycles-pp.__schedule
3.59 ±106% -1.1 2.50 ±173% perf-profile.children.cycles-pp.__open64_nocancel
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Oliver Sang
Attachments
- config-5.11.0-rc2-00040-g53c5e5a1d87e [text/plain] 172412 bytes · preview
- job-script [text/plain] 8119 bytes · preview
- job.yaml [text/plain] 5632 bytes · preview
- reproduce [text/plain] 497 bytes · preview