Re: [BUG][2.6.23-rc6] Badness at arch/powerpc/kernel/smp.c:202
From: Satyam Sharma <hidden>
Date: 2007-09-14 10:37:47
Also in:
lkml
Hi Kamalesh, There's two things at work here ... On 9/14/07, Kamalesh Babulal [off-list ref] wrote:
quoted hunk ↗ jump to hunk
Hi, With 2.6.23-rc6 running on the ppc64 box, following oops is hit Oops: Machine check, sig: 7 [#1] SMP NR_CPUS=128 pSeries Modules linked in: binfmt_misc ipv6 dm_mod ehci_hcd ohci_hcd usbcore NIP: c0000000000ed560 LR: c0000000000efc7c CTR: c0000000000ed504 REGS: c00000000ffef680 TRAP: 0200 Not tainted (2.6.23-rc6-autokern1) MSR: 8000000000109032 <EE,ME,IR,DR> CR: 28002042 XER: 00000010 TASK = c0000000ecf9f000[0] 'swapper' THREAD: c00000000ff8c000 CPU: 2 GPR00: 0000000000000000 c00000000ffef900 c0000000006fe598 c0000000d7a8f200 GPR04: 0000000000001000 0000000000000000 0000000000001000 8000000000c26393 GPR08: c0000000006b43d0 0000000000000001 0000000000001000 0000000000000000 GPR12: 0000000048000048 c0000000005f1700 0000000000000000 0000000007a8dcd0 GPR16: 0000000000000002 0000000000000000 0000000000000000 0000000000000000 GPR20: 0000000000000000 0000000000001000 0000000000001000 0000000000000000 GPR24: 0000000000000000 0000000000000000 0000000000001000 c0000000063234e8 GPR28: 0000000000001000 0000000000000000 c000000000689c08 c00000000ff3a480 NIP [c0000000000ed560] .end_bio_bh_io_sync+0x5c/0xac LR [c0000000000efc7c] .bio_endio+0xb4/0xd4 Call Trace: [c00000000ffef900] [c00000000ffef990] 0xc00000000ffef990 (unreliable) [c00000000ffef980] [c0000000000efc7c] .bio_endio+0xb4/0xd4 [c00000000ffefa10] [c000000000290060] .__end_that_request_first+0x154/0x548 [c00000000ffefae0] [c00000000035af10] .scsi_end_request+0x40/0x138 [c00000000ffefb80] [c00000000035b234] .scsi_io_completion+0x188/0x454 [c00000000ffefc60] [c000000000372a24] .sd_rw_intr+0x2e4/0x338 [c00000000ffefd30] [c000000000354548] .scsi_finish_command+0xbc/0xe0 [c00000000ffefdc0] [c00000000035bdf0] .scsi_softirq_done+0x140/0x188 [c00000000ffefe60] [c000000000293184] .blk_done_softirq+0xa0/0xd0 [c00000000ffefef0] [c000000000055e1c] .__do_softirq+0xa8/0x164 [c00000000ffeff90] [c000000000023f14] .call_do_softirq+0x14/0x24 [c00000000ff8f960] [c00000000000bd30] .do_softirq+0x68/0xac [c00000000ff8f9f0] [c000000000055f70] .irq_exit+0x54/0x6c [c00000000ff8fa70] [c00000000000c358] .do_IRQ+0x170/0x1ac [c00000000ff8fb00] [c000000000004780] hardware_interrupt_entry+0x18/0x98--- Exception: 501 at .pseries_dedicated_idle_sleep+0xe0/0x194 LR = .pseries_dedicated_idle_sleep+0xd0/0x194[c00000000ff8fdf0] [0000000000000000] .__start+0x4000000000000000/0x8 (unreliable) [c00000000ff8fe80] [c000000000010bd4] .cpu_idle+0x104/0x1d8 [c00000000ff8ff00] [c00000000002672c] .start_secondary+0x160/0x184 [c00000000ff8ff90] [c000000000008364] .start_secondary_prolog+0xc/0x10 Instruction dump: 409a0030 393f0018 38000080 7d6048a8 7d6b0378 7d6049ad 40a2fff4 38002000 7d2018a8 7d290378 7d2019ad 40a2fff4 <e9230038> e89f0018 e9690000 f8410028 Kernel panic - not syncing: Fatal exception in interrupt
This oops is the real bug here, but is that a machine check exception? If so, it could be a hardware failure what you saw there instead, and not really a kernel bug ...
------------[ cut here ]------------ Badness at arch/powerpc/kernel/smp.c:202
This one is not the real issue at all -- it's just a sad problem in the powerpc panic() -> smp_send_stop() -> smp_call_function() -> smp_call_function_map() call chain. The thing is, smp_call_function() cannot be allowed from interrupt disabled contexts, hence the WARN_ON() there. But panic() is a special case, and so we must *not* complain when called from panic -- doing so will only scroll away the real call trace / oops log from the screen (thankfully you had a serial cable there?) and distract from the real bug, like what happened here ... The fix is to define an alternative __smp_call_function(), that calls into smp_call_function_map(), and is itself called from smp_call_function(), and then: 1. Use the inner __smp_call_function() in smp_send_stop(), and, 2. Move the WARN_ON() to the smp_call_function() wrapper instead. I'd be back at my desk only by Tuesday, so don't expect a patch before then, unless Paul fixes this up by himself before that. Cheers, Satyam