[powerpc/perf] WARN_ONCE arch/powerpc/include/asm/interrupt.h:365 with perf tests

2 messages, 1 author, 2022-03-24 · open the first message on its own page

[powerpc/perf] WARN_ONCE arch/powerpc/include/asm/interrupt.h:365 with perf tests

From: Sachin Sant <hidden>
Date: 2022-02-04 07:04:20

While running perftool [1] test against 5.17-rc2 booted on Power 9 LPAR
following warning is seen:

[  442.002150] ------------[ cut here ]------------
[  442.002164] WARNING: CPU: 7 PID: 76 at arch/powerpc/include/asm/interrupt.h:365 interrupt_nmi_exit_prepare+0x150/0x160
[  442.002175] Modules linked in: dm_mod nft_compat nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables nfnetlink pseries_rng uio_pdrv_genirq uio nfsd auth_rpcgss nfs_acl lockd drm grace sunrpc sch_fq_codel drm_panel_orientation_quirks i2c_core xfs libcrc32c sr_mod sd_mod cdrom t10_pi xts ibmvscsi ibmveth scsi_transport_srp vmx_crypto fuse
[  442.002216] CPU: 7 PID: 76 Comm: kworker/7:1 Kdump: loaded Not tainted 5.17.0-rc2-00167-gdcb85f85fa6f #1
[  442.002222] Workqueue: events perf_sched_delayed
[  442.002228] NIP:  c00000000002b840 LR: c00000000002b818 CTR: c000000000343240
[  442.002232] REGS: c00000002c08f460 TRAP: 0700   Not tainted  (5.17.0-rc2-00167-gdcb85f85fa6f)
[  442.002237] MSR:  8000000000021033 <SF,ME,IR,DR,RI,LE>  CR: 88002484  XER: 000000cf
[  442.002247] CFAR: c00000000002b7ac IRQMASK: 3
[  442.002247] GPR00: c00000000002b818 c00000002c08f700 c000000002a1fe00 0000000000000000
[  442.002247] GPR04: c00000002c08f758 0000000000000008 00000003fdcc0000 00000003fdcc0000
[  442.002247] GPR08: 0000000000000001 c000000007fc6280 0000000000000021 000000003b9aca00
[  442.002247] GPR12: 0000000000002000 c000000007fc6280 c0000000001925f8 c000000028ce0100
[  442.002247] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  442.002247] GPR20: 0000000000000000 c000000002a53a00 c000000002b0a8a0 0000000000000000
[  442.002247] GPR24: c000000002550de0 0000000000000001 c000000001059d60 0000000000000000
[  442.002247] GPR28: 0000000060000000 c008000000100000 c00000002c08f810 c00000002c08f758
[  442.002299] NIP [c00000000002b840] interrupt_nmi_exit_prepare+0x150/0x160
[  442.002305] LR [c00000000002b818] interrupt_nmi_exit_prepare+0x128/0x160
[  442.002310] Call Trace:
[  442.002312] [c00000002c08f700] [c00000000013bcac] perf_event_interrupt+0x3c/0x70 (unreliable)
[  442.002320] [c00000002c08f730] [c00000000002b8c4] performance_monitor_exception_nmi+0x74/0xb0
[  442.002327] [c00000002c08f780] [c00000000002bd44] performance_monitor_exception+0x44/0x60
[  442.002332] [c00000002c08f7a0] [c00000000000af28] performance_monitor_common_virt+0x208/0x210
[  442.002339] --- interrupt: f00 at __patch_instruction+0x10/0x60
[  442.002344] NIP:  c0000000000b0bc0 LR: c0000000000b1080 CTR: 000000000000ae84
[  442.002348] REGS: c00000002c08f810 TRAP: 0f00   Not tainted  (5.17.0-rc2-00167-gdcb85f85fa6f)
[  442.002352] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 44002484  XER: 000000cf
[  442.002365] CFAR: 0000000000000000 IRQMASK: 1
[  442.002365] GPR00: c0000000000b1114 c00000002c08fab0 c000000002a1fe00 c0000000001a59f8
[  442.002365] GPR04: 0000000060000000 c0080000001059f8 8e011a00000000c0 ffffffffffffffff
[  442.002365] GPR08: c0000000001a018e 0000000060000000 00000000000000c0 0000000000000040
[  442.002365] GPR12: 0000000084002484 c000000007fc6280 c0000000001925f8 c000000028ce0100
[  442.002365] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  442.002365] GPR20: 0000000000000000 c000000002a53a00 c000000002b0a8a0 0000000000000000
[  442.002365] GPR24: c000000002550de0 0000000000000001 c000000001059d60 0000000000000000
[  442.002365] GPR28: 0000000060000000 c008000000100000 0000000000000000 c0000000001a59f8
[  442.002414] NIP [c0000000000b0bc0] __patch_instruction+0x10/0x60
[  442.002419] LR [c0000000000b1080] patch_instruction+0xf0/0x1c0
[  442.002423] --- interrupt: f00
[  442.002426] [c00000002c08fab0] [c0000000000b1114] patch_instruction+0x184/0x1c0 (unreliable)
[  442.002432] [c00000002c08fb00] [c00000000005dc48] arch_jump_label_transform+0x38/0x78
[  442.002438] [c00000002c08fb20] [c00000000036a948] __jump_label_update+0x148/0x180
[  442.002444] [c00000002c08fbc0] [c00000000036afd8] static_key_disable_cpuslocked+0xe8/0x130
[  442.002450] [c00000002c08fc30] [c00000000036b050] static_key_disable+0x30/0x50
[  442.002454] [c00000002c08fc60] [c000000000343978] perf_sched_delayed+0x98/0xc0
[  442.002460] [c00000002c08fc90] [c0000000001849dc] process_one_work+0x27c/0x550
[  442.002466] [c00000002c08fd30] [c000000000184d58] worker_thread+0xa8/0x620
[  442.002472] [c00000002c08fdc0] [c000000000192710] kthread+0x120/0x130
[  442.002477] [c00000002c08fe10] [c00000000000ce64] ret_from_kernel_thread+0x5c/0x64
[  442.002482] Instruction dump:
[  442.002486] e87e0100 480893a5 60000000 2c230000 4182ff58 39400000 f87e0100 b14d0930
[  442.002494] 4bffff48 60000000 60000000 60420000 <0fe00000> 60000000 60000000 60420000
[  442.002503] ---[ end trace 0000000000000000 ]---
[  442.002511] Can't find PMC that caused IRQ

Since this warning is printed only once I can’t recreate it easily. I have seen this only
on P9 (hash MMU) and only with 5.17-rc1 onwards.

- Sachin

[1] - https://github.com/sacsant/avocado-misc-tests/blob/ci/perf/perftool.py

Re: [powerpc/perf] WARN_ONCE arch/powerpc/include/asm/interrupt.h:365 with perf tests

From: Sachin Sant <hidden>
Date: 2022-03-24 06:21:39

On 04-Feb-2022, at 12:33 PM, Sachin Sant [off-list ref] wrote:

While running perftool [1] test against 5.17-rc2 booted on Power 9 LPAR
following warning is seen:

[  442.002150] ------------[ cut here ]------------
[  442.002164] WARNING: CPU: 7 PID: 76 at arch/powerpc/include/asm/interrupt.h:365 interrupt_nmi_exit_prepare+0x150/0x160
I continue to see this warning. 

[ 2989.536335] ------------[ cut here ]------------
[ 2989.536346] WARNING: CPU: 15 PID: 1153191 at arch/powerpc/include/asm/interrupt.h:362 performance_monitor_exception_nmi+0x2a0/0x2b0
[ 2989.536356] Modules linked in: dm_mod ip_set bonding rfkill nf_tables libcrc32c nfnetlink sunrpc pseries_rng xts uio_pdrv_genirq vmx_crypto uio sch_fq_codel ext4 mbcache jbd2 sd_mod t10_pi crc64_rocksoft crc64 sg ibmvscsi ibmveth scsi_transport_srp ipmi_devintf ipmi_msghandler fuse
[ 2989.536387] CPU: 15 PID: 1153191 Comm: kworker/15:3 Not tainted 5.17.0-next-20220323 #1
[ 2989.536393] Workqueue: events perf_sched_delayed
[ 2989.536398] NIP:  c00000000002b870 LR: c00000000002b838 CTR: c0000000003342f0
[ 2989.536402] REGS: c00000001339f490 TRAP: 0700   Not tainted  (5.17.0-next-20220323)
[ 2989.536406] MSR:  8000000000021033 <SF,ME,IR,DR,RI,LE>  CR: 28002828  XER: 00000000
[ 2989.536416] CFAR: c00000000002b794 IRQMASK: 3 
[ 2989.536416] GPR00: c00000000002b838 c00000001339f730 c000000002a03700 0000000000000000 
[ 2989.536416] GPR04: 0000000006008010 c000000002a43524 c000000001150140 c0000000000b2f30 
[ 2989.536416] GPR08: c000000000016cec 0000000000000001 c00000001ec3a280 7265677368657265 
[ 2989.536416] GPR12: 0000000000002000 c00000001ec3a280 c00000000018f058 c00000000963a340 
[ 2989.536416] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[ 2989.536416] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[ 2989.536416] GPR24: fffffffffffffef7 0000000000000001 c000000002a3dfd8 0000000000000001 
[ 2989.536416] GPR28: 0000000000002001 0000000000000000 0000000000000021 c00000001339f810 
[ 2989.536468] NIP [c00000000002b870] performance_monitor_exception_nmi+0x2a0/0x2b0
[ 2989.536474] LR [c00000000002b838] performance_monitor_exception_nmi+0x268/0x2b0
[ 2989.536480] Call Trace:
[ 2989.536482] [c00000001339f730] [c00000000002b70c] performance_monitor_exception_nmi+0x13c/0x2b0 (unreliable)
[ 2989.536490] [c00000001339f780] [c00000000002c624] performance_monitor_exception+0x44/0x50
[ 2989.536496] [c00000001339f7a0] [c00000000000af28] performance_monitor_common_virt+0x208/0x210
[ 2989.536503] --- interrupt: f00 at __patch_instruction+0x10/0x60
[ 2989.536509] NIP:  c0000000000b2f30 LR: c0000000000b3604 CTR: c000000000335020
[ 2989.536513] REGS: c00000001339f810 TRAP: 0f00   Not tainted  (5.17.0-next-20220323)
[ 2989.536516] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 24002224  XER: 00000000
[ 2989.536529] CFAR: 0000000000000000 IRQMASK: 1 
[ 2989.536529] GPR00: c0000000000b3578 c00000001339fab0 c000000002a03700 c0000000001a1068 
[ 2989.536529] GPR04: 0000000060000000 c008000000201068 8e011a00000000c0 0000000000103907 
[ 2989.536529] GPR08: ffffffffffffffff 0000000000000018 0000000060000000 0000000000000006 
[ 2989.536529] GPR12: 0000000000002000 c00000001ec3a280 c00000000018f058 c00000000963a340 
[ 2989.536529] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[ 2989.536529] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[ 2989.536529] GPR24: fffffffffffffef7 0000000000000001 c000000001000918 0000000000000000 
[ 2989.536529] GPR28: c008000000200000 0000000000000000 0000000060000000 c0000000001a1068 
[ 2989.536577] NIP [c0000000000b2f30] __patch_instruction+0x10/0x60
[ 2989.536583] LR [c0000000000b3604] patch_instruction+0x124/0x160
[ 2989.536588] --- interrupt: f00
[ 2989.536590] [c00000001339fab0] [c0000000000b3578] patch_instruction+0x98/0x160 (unreliable)
[ 2989.536597] [c00000001339fb00] [c00000000005fa28] arch_jump_label_transform+0x38/0x78
[ 2989.536603] [c00000001339fb20] [c000000000359854] __jump_label_update+0x144/0x180
[ 2989.536609] [c00000001339fbc0] [c000000000359e38] static_key_disable_cpuslocked+0xe8/0x130
[ 2989.536615] [c00000001339fc30] [c000000000359eb0] static_key_disable+0x30/0x50
[ 2989.536620] [c00000001339fc60] [c0000000003350b8] perf_sched_delayed+0x98/0xc0
[ 2989.536626] [c00000001339fc90] [c000000000182738] process_one_work+0x288/0x560
[ 2989.536632] [c00000001339fd30] [c000000000182a88] worker_thread+0x78/0x620
[ 2989.536637] [c00000001339fdc0] [c00000000018f174] kthread+0x124/0x130
[ 2989.536642] [c00000001339fe10] [c00000000000ce64] ret_from_kernel_thread+0x5c/0x64
[ 2989.536647] Instruction dump:
[ 2989.536650] 4182ff2c 39400000 f87f0100 b14d0930 4bffff1c 60000000 60000000 60000000 
[ 2989.536659] 38600001 482cf905 60000000 4bfffe6c <0fe00000> 60000000 60000000 60000000 
[ 2989.536668] ---[ end trace 0000000000000000 ]—

- Sachin
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help