[next-20190530] Boot failure on PowerPC

Latest next fails to boot with a kernel panic on POWER9.

[   33.689332] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: write_irq_affinity.isra.5+0x15c/0x160
[   33.689346] CPU: 35 PID: 4907 Comm: irqbalance Not tainted 5.2.0-rc2-next-20190530-autotest-autotest #1
[   33.689352] Call Trace:
[   33.689356] [c0000018d974bab0] [c000000000b5328c] dump_stack+0xb0/0xf4 (unreliable)
[   33.689364] [c0000018d974baf0] [c000000000120694] panic+0x16c/0x408
[   33.689370] [c0000018d974bb80] [c00000000012010c] __stack_chk_fail+0x2c/0x30
[   33.689376] [c0000018d974bbe0] [c0000000001b859c] write_irq_affinity.isra.5+0x15c/0x160
[   33.689383] [c0000018d974bd30] [c0000000004d6f30] proc_reg_write+0x90/0x110
[   33.689388] [c0000018d974bd60] [c00000000041453c] __vfs_write+0x3c/0x70
[   33.689394] [c0000018d974bd80] [c000000000418650] vfs_write+0xd0/0x250
[   33.689399] [c0000018d974bdd0] [c000000000418a2c] ksys_write+0x7c/0x130
[   33.689405] [c0000018d974be20] [c00000000000b688] system_call+0x5c/0x70

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 30 to
narrow down this problem.

Thanks
-Sachin

Sachin Sant [off-list ref] writes:

Latest next fails to boot with a kernel panic on POWER9.

[   33.689332] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: write_irq_affinity.isra.5+0x15c/0x160
[   33.689346] CPU: 35 PID: 4907 Comm: irqbalance Not tainted 5.2.0-rc2-next-20190530-autotest-autotest #1
[   33.689352] Call Trace:
[   33.689356] [c0000018d974bab0] [c000000000b5328c] dump_stack+0xb0/0xf4 (unreliable)
[   33.689364] [c0000018d974baf0] [c000000000120694] panic+0x16c/0x408
[   33.689370] [c0000018d974bb80] [c00000000012010c] __stack_chk_fail+0x2c/0x30
[   33.689376] [c0000018d974bbe0] [c0000000001b859c] write_irq_affinity.isra.5+0x15c/0x160
[   33.689383] [c0000018d974bd30] [c0000000004d6f30] proc_reg_write+0x90/0x110
[   33.689388] [c0000018d974bd60] [c00000000041453c] __vfs_write+0x3c/0x70
[   33.689394] [c0000018d974bd80] [c000000000418650] vfs_write+0xd0/0x250
[   33.689399] [c0000018d974bdd0] [c000000000418a2c] ksys_write+0x7c/0x130
[   33.689405] [c0000018d974be20] [c00000000000b688] system_call+0x5c/0x70

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 30 to
narrow down this problem.

My CI was fine with next-20190529 (9a15d2e3fd03e3).

cheers

On Fri, May 31, 2019 at 6:52 AM Michael Ellerman [off-list ref] wrote:

Sachin Sant [off-list ref] writes:

quoted

Latest next fails to boot with a kernel panic on POWER9.

[   33.689332] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: write_irq_affinity.isra.5+0x15c/0x160
[   33.689346] CPU: 35 PID: 4907 Comm: irqbalance Not tainted 5.2.0-rc2-next-20190530-autotest-autotest #1
[   33.689352] Call Trace:
[   33.689356] [c0000018d974bab0] [c000000000b5328c] dump_stack+0xb0/0xf4 (unreliable)
[   33.689364] [c0000018d974baf0] [c000000000120694] panic+0x16c/0x408
[   33.689370] [c0000018d974bb80] [c00000000012010c] __stack_chk_fail+0x2c/0x30
[   33.689376] [c0000018d974bbe0] [c0000000001b859c] write_irq_affinity.isra.5+0x15c/0x160
[   33.689383] [c0000018d974bd30] [c0000000004d6f30] proc_reg_write+0x90/0x110
[   33.689388] [c0000018d974bd60] [c00000000041453c] __vfs_write+0x3c/0x70
[   33.689394] [c0000018d974bd80] [c000000000418650] vfs_write+0xd0/0x250
[   33.689399] [c0000018d974bdd0] [c000000000418a2c] ksys_write+0x7c/0x130
[   33.689405] [c0000018d974be20] [c00000000000b688] system_call+0x5c/0x70

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 30 to
narrow down this problem.

My CI was fine with next-20190529 (9a15d2e3fd03e3).

cheers

Hi Sachin,
It looks this patch may fix the issue:
https://lkml.org/lkml/2019/5/30/1630 , but I'm not sure.

Thanks
-- Dexuan

On 31-May-2019, at 11:43 PM, Dexuan-Linux Cui [off-list ref] wrote:

On Fri, May 31, 2019 at 6:52 AM Michael Ellerman [off-list ref] wrote:

quoted

quoted

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 30 to
narrow down this problem.

My CI was fine with next-20190529 (9a15d2e3fd03e3).

cheers

Hi Sachin,
It looks this patch may fix the issue:
https://lkml.org/lkml/2019/5/30/1630 , but I'm not sure.

It does not help fix the kernel panic issue, but it fixes the get_swap_device warning
messages during the boot.

Thanks
-Sachin

Hi Sachin,

I verified below patch against Ubuntu 18.04, didn't hit the kernel panic any more, could you please let know how did you verify?

Thanks,
Lili
-----Original Message-----
From: Sachin Sant <redacted> 
Sent: Monday, June 3, 2019 11:12 AM
To: Dexuan-Linux Cui <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>; linuxppc-dev@lists.ozlabs.org; linux-next@vger.kernel.org; Dexuan Cui <decui@microsoft.com>; Lili Deng (Wicresoft North America Ltd) <redacted>
Subject: Re: [next-20190530] Boot failure on PowerPC

On 31-May-2019, at 11:43 PM, Dexuan-Linux Cui [off-list ref] wrote:

On Fri, May 31, 2019 at 6:52 AM Michael Ellerman [off-list ref] wrote:

quoted

quoted

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 
30 to narrow down this problem.

My CI was fine with next-20190529 (9a15d2e3fd03e3).

cheers

Hi Sachin,
It looks this patch may fix the issue:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.org%2Flkml%2F2019%2F5%2F30%2F1630&data=02%7C01%7Cv-lide%40microsoft.com%7C66e1ef6017aa461703f808d6e7d148cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636951283393233385&sdata=IJFhtvL2Bd87HCoMZ7oWL%2Bar6NY%2FfPbmdCZMT%2BJz5t4%3D&reserved=0 , but I'm not sure.

It does not help fix the kernel panic issue, but it fixes the get_swap_device warning messages during the boot.

Thanks
-Sachin

ACK for ' but it fixes the get_swap_device warning messages during the boot.'
Double check for kernel panic issue, without this patch, it seems not repro in my local manual environment, so please ignore my previous mail for 'the patch fixes the kernel panic'.
Sorry for the confusion. 

-----Original Message-----
From: Lili Deng (Wicresoft North America Ltd) 
Sent: Monday, June 3, 2019 11:24 AM
To: Sachin Sant <redacted>; Dexuan-Linux Cui <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>; linuxppc-dev@lists.ozlabs.org; linux-next@vger.kernel.org; Dexuan Cui <decui@microsoft.com>
Subject: RE: [next-20190530] Boot failure on PowerPC

Hi Sachin,

I verified below patch against Ubuntu 18.04, didn't hit the kernel panic any more, could you please let know how did you verify?

Thanks,
Lili
-----Original Message-----
From: Sachin Sant <redacted> 
Sent: Monday, June 3, 2019 11:12 AM
To: Dexuan-Linux Cui <redacted>
Cc: Michael Ellerman <mpe@ellerman.id.au>; linuxppc-dev@lists.ozlabs.org; linux-next@vger.kernel.org; Dexuan Cui <decui@microsoft.com>; Lili Deng (Wicresoft North America Ltd) <redacted>
Subject: Re: [next-20190530] Boot failure on PowerPC

On 31-May-2019, at 11:43 PM, Dexuan-Linux Cui [off-list ref] wrote:

On Fri, May 31, 2019 at 6:52 AM Michael Ellerman [off-list ref] wrote:

quoted

quoted

Machine boots till login prompt and then panics few seconds later.

Last known next build was May 24th. Will attempt few builds till May 
30 to narrow down this problem.

My CI was fine with next-20190529 (9a15d2e3fd03e3).

cheers

Hi Sachin,
It looks this patch may fix the issue:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.org%2Flkml%2F2019%2F5%2F30%2F1630&data=02%7C01%7Cv-lide%40microsoft.com%7C66e1ef6017aa461703f808d6e7d148cd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636951283393233385&sdata=IJFhtvL2Bd87HCoMZ7oWL%2Bar6NY%2FfPbmdCZMT%2BJz5t4%3D&reserved=0 , but I'm not sure.

It does not help fix the kernel panic issue, but it fixes the get_swap_device warning messages during the boot.

Thanks
-Sachin