panic in cpufreq_online() in 6.14-rc1 on PowerNV

Hi,

I am getting a kernel panic on my Raptor Talos Power9 system after
updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
time, but I haven't start bisecting yet. Does it sound familiar to
anyone?

...
[    4.226443] powernv-cpufreq: cpufreq pstate min 0x63 nominal 0x30 max 0x0
[    4.226464] powernv-cpufreq: Workload Optimized Frequency is enabled in the platform
[    4.226662] BUG: Unable to handle kernel instruction fetch (NULL pointer?)
[    4.226687] Faulting instruction address: 0x00000000
[    4.226700] Oops: Kernel access of bad area, sig: 7 [#1]
[    4.226734] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA PowerNV
[    4.226770] Modules linked in:
[    4.226795] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.14.0-0.rc1.15.fc42.ppc64le #1
[    4.226854] Hardware name: T2P9D01 REV 1.00 POWER9 0x4e1202 opal:skiboot-bc106a0 PowerNV
[    4.226912] NIP:  0000000000000000 LR: c0000000012a57f4 CTR: 0000000000000000
[    4.226970] REGS: c00000000cdcf610 TRAP: 0400   Not tainted  (6.14.0-0.rc1.15.fc42.ppc64le)
[    4.227007] MSR:  9000000002089033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE>  CR: 84008848  XER: 00000000
[    4.227063] CFAR: c0000000012a57f0 IRQMASK: 0
[    4.227063] GPR00: c0000000012a57a0 c00000000cdcf8b0 c000000002474900 c000000023447000
[    4.227063] GPR04: 0000000000000001 0000000000000000 0000000000000080 0000000000000000
[    4.227063] GPR08: 0000000000000000 c000000003cf56c8 0000000000000001 0000000000008000
[    4.227063] GPR12: 0000000000000000 c000000003f70000 c000000003cb39a8 c000000001d80988
[    4.227063] GPR16: 0000000000000001 c000000001c6a4d8 c000000001c4f828 c0000000012adf70
[    4.227063] GPR20: c000000003c30ed8 0000000000000001 0000000000000000 c0000000234474b8
[    4.227063] GPR24: c000000023447448 c000000003c30000 c000000003f492a0 0000000000000000
[    4.227063] GPR28: 0000000000000000 c000000003d56d60 c0000000039d4bc0 c000000023447000
[    4.227407] NIP [0000000000000000] 0x0
[    4.227440] LR [c0000000012a57f4] cpufreq_online+0x474/0x1250
[    4.227478] Call Trace:
[    4.227506] [c00000000cdcf8b0] [c0000000012a57a0] cpufreq_online+0x420/0x1250 (unreliable)
[    4.227559] [c00000000cdcf980] [c0000000012a6710] cpufreq_add_dev+0x110/0x170
[    4.227597] [c00000000cdcfa00] [c00000000101f858] subsys_interface_register+0x188/0x1d0
[    4.227647] [c00000000cdcfa70] [c0000000012a0cfc] cpufreq_register_driver+0x23c/0x470
[    4.227690] [c00000000cdcfb00] [c00000000308e894] powernv_cpufreq_init+0x910/0xa10
[    4.227728] [c00000000cdcfc40] [c000000000010c3c] do_one_initcall+0x7c/0x3ac
[    4.227766] [c00000000cdcfd20] [c00000000300e574] kernel_init_freeable+0x3d0/0x480
[    4.227798] [c00000000cdcfdf0] [c000000000011320] kernel_init+0x2c/0x1bc
[    4.227815] [c00000000cdcfe50] [c00000000000debc] ret_from_kernel_user_thread+0x14/0x1c
[    4.227834] --- interrupt: 0 at 0x0
[    4.227845] Code: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
[    4.227919] ---[ end trace 0000000000000000 ]---
[    4.430870] ata5: SATA link down (SStatus 0 SControl 300)
[    4.430912] ata3: SATA link down (SStatus 0 SControl 300)
[    4.430944] ata7: SATA link down (SStatus 0 SControl 300)
[    4.440855] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    4.440900] ata6: SATA link down (SStatus 0 SControl 300)
[    4.440943] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    4.440985] ata4: SATA link down (SStatus 0 SControl 300)
[    4.552442] ata8.00: ATAPI: MARVELL VIRTUALL, 1.09, max UDMA/66
[    4.552596] ata8.00: configured for UDMA/66
[    4.553499] ata2.00: ATAPI: DRW-24D5MT, 1.00, max UDMA/133
[    4.554908] ata2.00: configured for UDMA/133
[    4.569631] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    4.569884] ata1.00: Model 'Samsung SSD 860 EVO 500GB', rev 'RVT02B6Q', applying quirks: noncqtrim zeroaftertrim noncqonati
[    4.569986] ata1.00: supports DRM functions and may not be fully accessible
[    4.570020] ata1.00: ATA-11: Samsung SSD 860 EVO 500GB, RVT02B6Q, max UDMA/133
[    4.570798] ata1.00: 976773168 sectors, multi 1: LBA48 NCQ (depth 32), AA
[    4.576809] ata1.00: Features: Trust Dev-Sleep NCQ-sndrcv
[    4.577253] ata1.00: supports DRM functions and may not be fully accessible
[    4.584748] ata1.00: configured for UDMA/133
[    4.585278] scsi 0:0:0:0: Direct-Access     ATA      Samsung SSD 860  2B6Q PQ: 0 ANSI: 5
[    4.586021] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    4.586556] ata1.00: Enabling discard_zeroes_data
[    4.586602] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[    4.586650] sd 0:0:0:0: [sda] Write Protect is off
[    4.586698] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    4.586742] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    4.586854] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[    4.587066] scsi 1:0:0:0: CD-ROM            ASUS     DRW-24D5MT       1.00 PQ: 0 ANSI: 5
[    4.588265] ata1.00: Enabling discard_zeroes_data
[    6.037014] pstore: backend (nvram) writing error (-1)
[    6.037032]
[    7.037040] note: swapper/0[1] exited with irqs disabled
[    7.037097] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000007


	Thanks,

		Dan

Hi Dan,

The fix for this was pulled yesterday[1], you can apply it

[1] - https://lore.kernel.org/all/20250205181347.2079272-1-aboorvad@linux.ibm.com

Thanks,
Gautam

On Thu, 6 Feb 2025 14:30:40 +0530
Gautam Menghani [off-list ref] wrote:

Hi Dan,

The fix for this was pulled yesterday[1], you can apply it

[1] - https://lore.kernel.org/all/20250205181347.2079272-1-aboorvad@linux.ibm.com

thanks for the pointer, will give it a try


		Dan

On Thu Feb 6, 2025 at 6:41 PM AEST, Dan Horák wrote:

Hi,

I am getting a kernel panic on my Raptor Talos Power9 system after
updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
time, but I haven't start bisecting yet. Does it sound familiar to
anyone?

No, but it's possible it could be skiboot changes in PM code.

Thanks,
Nick

On Tue, 18 Feb 2025 11:35:08 +1000
"Nicholas Piggin" [off-list ref] wrote:

On Thu Feb 6, 2025 at 6:41 PM AEST, Dan Horák wrote:

quoted

Hi,

I am getting a kernel panic on my Raptor Talos Power9 system after
updating to the 6.14-rc1 kernel from 6.13. Seems reproducable every
time, but I haven't start bisecting yet. Does it sound familiar to
anyone?

No, but it's possible it could be skiboot changes in PM code.

it was the issue Gautam pointed me to, fixed in 6.14-rc2


		Dan