3.7-rc7: BUG: MAX_STACK_TRACE_ENTRIES too low!

Hi,

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html
-- 
BOFH excuse #235:

The new frame relay network hasn't bedded down the software loop transmitter yet.

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Christian.

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html

-- 
BOFH excuse #191:

Just type 'mv * /dev/null'.

On Tue, 2012-11-27 at 19:22 -0800, Christian Kujau wrote:

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

quoted

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Hi Christian, 

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

Thanks, Zhong

==============================================

diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9499385..cadebfd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S

@@ -439,6 +439,8 @@ ret_from_fork:
 ret_from_kernel_thread:
 	REST_NVGPRS(r1)
 	bl	schedule_tail
+	li	r3,0
+	stw	r3,0(r1)
 	mtlr	r14
 	mr	r3,r15
 	PPC440EP_ERR42

==============================================

Christian.

quoted

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html

On Wed, 28 Nov 2012 at 16:41, Li Zhong wrote:

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

I've just applied this to 3.7-rc7 and booted the machine. I don't know how 
to trigger this bug, so it might take a while until it happens again - or 
not, now with your patch applied.

It happened only 2 times so far, after ~8h and after ~20h:

Nov  5 13:28:20 alice kernel: [    0.000000] Linux version 3.7.0-rc4 
Nov  5 21:00:26 alice kernel: [27148.965634] BUG: MAX_STACK_TRACE_ENTRIES too low!
Nov 26 21:53:43 alice kernel: [    0.000000] Linux version 3.7.0-rc7 
Nov 27 17:15:29 alice kernel: [69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!

Thanks,
Christian.
-- 
BOFH excuse #86:

Runt packets

On Wed, 28 Nov 2012 at 16:41, Li Zhong wrote:

On Tue, 2012-11-27 at 19:22 -0800, Christian Kujau wrote:

quoted

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

quoted

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Hi Christian, 

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

After 3 days of uptime with your patch applied (and "normal" usage, just 
as before) no such BUG message occured. So, from my point of view, feel
free to add:

   Tested-by: Christian Kujau [off-list ref]

Thanks!
Christian.

-- 
BOFH excuse #267:

The UPS is on strike.

On Sat, 2012-12-01 at 12:32 -0800, Christian Kujau wrote:

On Wed, 28 Nov 2012 at 16:41, Li Zhong wrote:

quoted

On Tue, 2012-11-27 at 19:22 -0800, Christian Kujau wrote:

quoted

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

quoted

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Hi Christian, 

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

After 3 days of uptime with your patch applied (and "normal" usage, just 
as before) no such BUG message occured. So, from my point of view, feel
free to add:

   Tested-by: Christian Kujau [off-list ref]

Thank you, Christian. 

I'll try to submit a patch for it.

Thanks, Zhong

Thanks!
Christian.

quoted

==============================================

diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9499385..cadebfd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S

@@ -439,6 +439,8 @@ ret_from_fork:
 ret_from_kernel_thread:
 	REST_NVGPRS(r1)
 	bl	schedule_tail
+	li	r3,0
+	stw	r3,0(r1)
 	mtlr	r14
 	mr	r3,r15
 	PPC440EP_ERR42

==============================================

quoted

Christian.

quoted

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html

On Wed, 28 Nov 2012 at 16:41, Li Zhong wrote:

On Tue, 2012-11-27 at 19:22 -0800, Christian Kujau wrote:

quoted

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

quoted

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Hi Christian, 

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

The patch hasn't made it into mainline yet. Any chance that this will be 
included in 3.8?

Thanks,
Christian.

-- 
BOFH excuse #258:

That's easy to fix, but I can't be bothered.

On Sat, 2013-01-12 at 12:43 -0800, Christian Kujau wrote:

On Wed, 28 Nov 2012 at 16:41, Li Zhong wrote:

quoted

On Tue, 2012-11-27 at 19:22 -0800, Christian Kujau wrote:

quoted

On Tue, 27 Nov 2012 at 19:06, Christian Kujau wrote:

quoted

the same thing[0] happened again in 3.7-rc7, after ~20h uptime:

I found the following on patchwork, but this seems to deal with powerpc64 
only, while this PowerBook G4 of mine is powerpc32:

  http://patchwork.ozlabs.org/patch/193414/

It looks related, but then again, I fail to parse assember...

Hi Christian, 

Would you please help to try the following fix? I don't have a powerpc32
machine for test...

The patch hasn't made it into mainline yet. Any chance that this will be 
included in 3.8?

I don't know...

FYI, it is already in the next of ppc tree 
http://git.kernel.org/?p=linux/kernel/git/benh/powerpc.git;a=shortlog;h=refs/heads/next

I guess it would get into 3.9, at least. 

Thanks, Zhong

Thanks,
Christian.

quoted

==============================================

diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9499385..cadebfd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S

@@ -439,6 +439,8 @@ ret_from_fork:
 ret_from_kernel_thread:
 	REST_NVGPRS(r1)
 	bl	schedule_tail
+	li	r3,0
+	stw	r3,0(r1)
 	mtlr	r14
 	mr	r3,r15
 	PPC440EP_ERR42

==============================================

quoted

Christian.

quoted

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html

On Tue, 15 Jan 2013 at 14:59, Li Zhong wrote:

FYI, it is already in the next of ppc tree 
http://git.kernel.org/?p=linux/kernel/git/benh/powerpc.git;a=shortlog;h=refs/heads/next

I guess it would get into 3.9, at least. 

Hm, is there no chance to get this into 3.8? I've been running with this 
patch applied since 3.7-rc7 and it got rid of this 
"MAX_STACK_TRACE_ENTRIES too low" message. I've just upgraded to 3.8-rc5 
and it's still not in mainline :-\

Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.3/01836.html

quoted

quoted

==============================================

diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index 9499385..cadebfd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S

@@ -439,6 +439,8 @@ ret_from_fork:
 ret_from_kernel_thread:
 	REST_NVGPRS(r1)
 	bl	schedule_tail
+	li	r3,0
+	stw	r3,0(r1)
 	mtlr	r14
 	mr	r3,r15
 	PPC440EP_ERR42

==============================================

quoted

Christian.

quoted

[40007.339487] [sched_delayed] sched: RT throttling activated
[69731.388717] BUG: MAX_STACK_TRACE_ENTRIES too low!
[69731.390371] turning off the locking correctness validator.
[69731.391942] Call Trace:
[69731.393525] [c9a61c10] [c0009064] show_stack+0x70/0x1bc (unreliable)
[69731.395152] [c9a61c50] [c0077460] save_trace+0xfc/0x114
[69731.396735] [c9a61c60] [c007be20] __lock_acquire+0x1568/0x19b8
[69731.398296] [c9a61d00] [c007c2c0] lock_acquire+0x50/0x70
[69731.399857] [c9a61d20] [c0550e28] _raw_spin_lock_irq+0x5c/0x78
[69731.401419] [c9a61d40] [c054fb58] __schedule+0xd8/0x534
[69731.402972] [c9a61da0] [c0550094] _cond_resched+0x50/0x68
[69731.404527] [c9a61db0] [c0479908] dst_gc_task+0xbc/0x258
[69731.406070] [c9a61e40] [c004eeb8] process_one_work+0x1f4/0x49c
[69731.407585] [c9a61e80] [c004f644] worker_thread+0x14c/0x400
[69731.409075] [c9a61eb0] [c0057634] kthread+0xbc/0xc0
[69731.410521] [c9a61f40] [c0011ad4] ret_from_kernel_thread+0x5c/0x64
[...repeated 54 times...]

Anyone knows what this is about?

Thanks,
Christian.

[0] http://lkml.indiana.edu/hypermail/linux/kernel/1211.0/03025.html

-- 
BOFH excuse #238:

You did wha... oh _dear_....

On Sun, 27 Jan 2013 at 14:56, Christian Kujau wrote:

Hm, is there no chance to get this into 3.8? I've been running with this 
patch applied since 3.7-rc7 and it got rid of this 
"MAX_STACK_TRACE_ENTRIES too low" message. I've just upgraded to 3.8-rc5 
and it's still not in mainline :-\

Hah! I just noticed that it got merged the next day - Thanks!

Christian.
-- 
BOFH excuse #80:

That's a great computer you have there; have you considered how it would work as a BSD machine?