12 messages, 4 authors, 2016-06-21 · open the first message on its own page
From: Denis Kirjanov <hidden>
Date: 2016-06-06 07:54:47
Update: now with the proper mailing address. Hi, Posting if anyone knows how to quickly fix it :) Enabling CONFIG_PPC_EARLY_DEBUG on my quad G5 causes kernel to die with the unrecoverable exception 4100. I haven't bisected this yet. Photo attached. Thanks!
From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2016-06-06 12:27:06
On Mon, 2016-06-06 at 10:52 +0300, Denis Kirjanov wrote:
Update: now with the proper mailing address. Hi, Posting if anyone knows how to quickly fix it :)
Sorry but no.
Enabling CONFIG_PPC_EARLY_DEBUG on my quad G5 causes kernel to die with the unrecoverable exception 4100.
That means you took a SLB miss when MSR_RI was off. What's at c000000000ace328 ?
I haven't bisected this yet. Photo attached.
A bisect would be awesome. cheers
From: Denis Kirjanov <hidden>
Date: 2016-06-16 08:12:21
On 6/6/16, Michael Ellerman [off-list ref] wrote:
On Mon, 2016-06-06 at 10:52 +0300, Denis Kirjanov wrote:quoted
Update: now with the proper mailing address. Hi, Posting if anyone knows how to quickly fix it :)Sorry but no.quoted
Enabling CONFIG_PPC_EARLY_DEBUG on my quad G5 causes kernel to die with the unrecoverable exception 4100.That means you took a SLB miss when MSR_RI was off. What's at c000000000ace328 ?quoted
I haven't bisected this yet. Photo attached.A bisect would be awesome.
Ok, the issue was introduced by the commit
caca285e5ab4a7a19fede51688106ceed6fc45dd (powerpc/mm/radix: Use
STD_MMU_64 to properly isolate hash related code) and 970mp dies on
the following condition:
+BEGIN_MMU_FTR_SECTION
bl slb_allocate_realmode
-
+END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)
In my config I've enabled radix MMU lookup but in my understanding
only recent IBM machines support this feature and we have to call SLB
entry through slb_allocate_realmode on HPTE machine?
Thanks!
cheers
From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2016-06-16 12:07:16
On Thu, 2016-06-16 at 11:12 +0300, Denis Kirjanov wrote:
On 6/6/16, Michael Ellerman [off-list ref] wrote:quoted
On Mon, 2016-06-06 at 10:52 +0300, Denis Kirjanov wrote:quoted
I haven't bisected this yet. Photo attached.A bisect would be awesome.Ok, the issue was introduced by the commit caca285e5ab4a7a19fede51688106ceed6fc45dd (powerpc/mm/radix: Use STD_MMU_64 to properly isolate hash related code) and 970mp dies on the following condition:
+BEGIN_MMU_FTR_SECTION
bl slb_allocate_realmode
-
+END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)Thanks. How did you work that out? That FTR section says to call slb_allocate_realmode() when MMU_FTR_RADIX is clear (ie. zero). That should always be the case on your machine, unless you've plugged a Power9 into your G5 ;)
In my config I've enabled radix MMU lookup but in my understanding only recent IBM machines support this feature and we have to call SLB entry through slb_allocate_realmode on HPTE machine?
That's right, and that's what the code should be doing.
My G5 is booting OK:
michael@cogito:~$ uname -a
Linux cogito 4.7.0-rc3-gdb06d75 #1 SMP Thu Jun 16 05:16:31 AEST 2016 ppc64 GNU/Linux
michael@cogito:~$ cat /proc/cpuinfo
processor : 0
cpu : PPC970FX, altivec supported
clock : 1599.999000MHz
revision : 3.0 (pvr 003c 0300)
timebase : 33333333
platform : PowerMac
model : PowerMac8,1
machine : PowerMac8,1
motherboard : PowerMac8,1 MacRISC4 Power Macintosh
detected as : 338 (iMac G5)
pmac flags : 00000000
L2 cache : 512K unified
pmac-generation : NewWorld
michael@cogito:~$ zgrep RADIX /proc/config.gz
CONFIG_PPC_RADIX_MMU=y
michael@cogito:~$ dmesg|grep _features
[ 0.000000] cpu_features = 0x0804806318100448
[ 0.000000] cpu_user_features = 0xdc080000 0x00000000
[ 0.000000] mmu_features = 0x0c000001
[ 0.000000] firmware_features = 0x0000000000000000
In particular notice:
#define MMU_FTR_RADIX ASM_CONST(0x80000000)
mmu_features = 0x0c000001
ie. MMU_FTR_RADIX is clear.
Do you see a different value for mmu_features?
cheers
From: Denis Kirjanov <hidden>
Date: 2016-06-16 19:49:10
quoted
Ok, the issue was introduced by the commit caca285e5ab4a7a19fede51688106ceed6fc45dd (powerpc/mm/radix: Use STD_MMU_64 to properly isolate hash related code) and 970mp dies on the following condition:quoted
+BEGIN_MMU_FTR_SECTION bl slb_allocate_realmode - +END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)Thanks. How did you work that out? That FTR section says to call slb_allocate_realmode() when MMU_FTR_RADIX is clear (ie. zero). That should always be the case on your machine, unless you've plugged a Power9 into your G5 ;)quoted
In my config I've enabled radix MMU lookup but in my understanding only recent IBM machines support this feature and we have to call SLB entry through slb_allocate_realmode on HPTE machine?That's right, and that's what the code should be doing. My G5 is booting OK: michael@cogito:~$ uname -a Linux cogito 4.7.0-rc3-gdb06d75 #1 SMP Thu Jun 16 05:16:31 AEST 2016 ppc64 GNU/Linux michael@cogito:~$ cat /proc/cpuinfo processor : 0 cpu : PPC970FX, altivec supported clock : 1599.999000MHz revision : 3.0 (pvr 003c 0300) timebase : 33333333 platform : PowerMac model : PowerMac8,1 machine : PowerMac8,1 motherboard : PowerMac8,1 MacRISC4 Power Macintosh detected as : 338 (iMac G5) pmac flags : 00000000 L2 cache : 512K unified pmac-generation : NewWorld michael@cogito:~$ zgrep RADIX /proc/config.gz CONFIG_PPC_RADIX_MMU=y michael@cogito:~$ dmesg|grep _features [ 0.000000] cpu_features = 0x0804806318100448 [ 0.000000] cpu_user_features = 0xdc080000 0x00000000 [ 0.000000] mmu_features = 0x0c000001 [ 0.000000] firmware_features = 0x0000000000000000 In particular notice: #define MMU_FTR_RADIX ASM_CONST(0x80000000) mmu_features = 0x0c000001 ie. MMU_FTR_RADIX is clear. Do you see a different value for mmu_features?
Oops, I messed things up, sorry :/
The reason is here:
/* All done -- return from exception. */
ld r10,PACA_EXSLB+EX_LR(r13)@@ -1384,7 +1399,9 @@ slb_miss_realmode: lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */ mtlr r10 - +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2f
So with the patch we got the direct jump to label 2 : c000000000006edc <slb_miss_realmode>: c000000000006edc: 7d 48 02 a6 mflr r10 c000000000006ee0: 91 2d 01 8c stw r9,396(r13) c000000000006ee4: f9 4d 01 98 std r10,408(r13) c000000000006ee8: 48 03 28 09 bl c0000000000396f0 <.slb_allocate_realmode> c000000000006eec: e9 4d 01 98 ld r10,408(r13) c000000000006ef0: e8 6d 01 90 ld r3,400(r13) c000000000006ef4: 81 2d 01 8c lwz r9,396(r13) c000000000006ef8: 7d 48 03 a6 mtlr r10 c000000000006efc: 48 00 00 38 b c000000000006f34 <slb_miss_realmode+0x58> c000000000006f00: 71 8a 00 02 andi. r10,r12,2 c000000000006f04: 41 82 00 30 beq c000000000006f34 <slb_miss_realmode+0x58> c000000000006f08: 7d 38 01 20 mtocrf 128,r9 c000000000006f0c: 7d 30 11 20 mtocrf 1,r9 c000000000006f10: e9 2d 01 a8 ld r9,424(r13) c000000000006f14: 7d 20 e3 a6 mtppr r9 c000000000006f18: e9 2d 01 50 ld r9,336(r13) c000000000006f1c: e9 4d 01 58 ld r10,344(r13) c000000000006f20: e9 6d 01 60 ld r11,352(r13) c000000000006f24: e9 8d 01 68 ld r12,360(r13) c000000000006f28: e9 ad 01 70 ld r13,368(r13) c000000000006f2c: 4c 00 00 24 rfid c000000000006f30: 48 00 00 00 b c000000000006f30 <slb_miss_realmode+0x54> c000000000006f34: 7d 7a 02 a6 mfsrr0 r11 c000000000006f38: e9 4d 00 18 ld r10,24(r13) c000000000006f3c: 61 4a 6f 54 ori r10,r10,28500 c000000000006f40: 7d 5a 03 a6 mtsrr0 r10 c000000000006f44: e9 4d 00 20 ld r10,32(r13) c000000000006f48: 7d 5b 03 a6 mtsrr1 r10 c000000000006f4c: 4c 00 00 24 rfid c000000000006f50: 48 00 00 00 b c000000000006f50 <slb_miss_realmode+0x74> Without the ftr conditional section everything looks fine: c000000000006edc <slb_miss_realmode>: c000000000006edc: 7d 48 02 a6 mflr r10 c000000000006ee0: 91 2d 01 8c stw r9,396(r13) c000000000006ee4: f9 4d 01 98 std r10,408(r13) c000000000006ee8: 48 03 28 09 bl c0000000000396f0 <.slb_allocate_realmode> c000000000006eec: e9 4d 01 98 ld r10,408(r13) c000000000006ef0: e8 6d 01 90 ld r3,400(r13) c000000000006ef4: 81 2d 01 8c lwz r9,396(r13) c000000000006ef8: 7d 48 03 a6 mtlr r10 c000000000006efc: 71 8a 00 02 andi. r10,r12,2 c000000000006f00: 41 82 00 30 beq c000000000006f30 <slb_miss_realmode+0x54> c000000000006f04: 7d 38 01 20 mtocrf 128,r9 c000000000006f08: 7d 30 11 20 mtocrf 1,r9 c000000000006f0c: e9 2d 01 a8 ld r9,424(r13) c000000000006f10: 7d 20 e3 a6 mtppr r9 c000000000006f14: e9 2d 01 50 ld r9,336(r13) c000000000006f18: e9 4d 01 58 ld r10,344(r13) c000000000006f1c: e9 6d 01 60 ld r11,352(r13) c000000000006f20: e9 8d 01 68 ld r12,360(r13) c000000000006f24: e9 ad 01 70 ld r13,368(r13) c000000000006f28: 4c 00 00 24 rfid c000000000006f2c: 48 00 00 00 b c000000000006f2c <slb_miss_realmode+0x50> c000000000006f30: 7d 7a 02 a6 mfsrr0 r11 c000000000006f34: e9 4d 00 18 ld r10,24(r13) c000000000006f38: 61 4a 6f 50 ori r10,r10,28496 c000000000006f3c: 7d 5a 03 a6 mtsrr0 r10 c000000000006f40: e9 4d 00 20 ld r10,32(r13) c000000000006f44: 7d 5b 03 a6 mtsrr1 r10 c000000000006f48: 4c 00 00 24 rfid c000000000006f4c: 48 00 00 00 b c000000000006f4c <slb_miss_realmode+0x70>
cheers
From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: 2016-06-16 22:33:30
On Thu, 2016-06-16 at 22:49 +0300, Denis Kirjanov wrote:
- +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2f
Are we taking an SLB miss before we do the fixup maybe ? Cheers, Ben.
From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2016-06-17 01:46:04
On Fri, 2016-06-17 at 08:33 +1000, Benjamin Herrenschmidt wrote:
On Thu, 2016-06-16 at 22:49 +0300, Denis Kirjanov wrote:quoted
- +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2fAre we taking an SLB miss before we do the fixup maybe ?
Yeah that's the only explanation that makes any sense. I think instead of patching down this low we should instead be redirecting SLB misses to unknown_exception() when radix is enabled. Aneesh? cheers
From: Aneesh Kumar K.V <hidden>
Date: 2016-06-18 17:17:29
Michael Ellerman [off-list ref] writes:
On Fri, 2016-06-17 at 08:33 +1000, Benjamin Herrenschmidt wrote:quoted
On Thu, 2016-06-16 at 22:49 +0300, Denis Kirjanov wrote:quoted
- +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2fAre we taking an SLB miss before we do the fixup maybe ?Yeah that's the only explanation that makes any sense. I think instead of patching down this low we should instead be redirecting SLB misses to unknown_exception() when radix is enabled. Aneesh?
The 2f branch ends up doing unrecoverable exception. Or are you suggesting something else ? -aneesh
From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2016-06-20 09:23:50
On Sat, 2016-06-18 at 22:47 +0530, Aneesh Kumar K.V wrote:
Michael Ellerman [off-list ref] writes:quoted
On Fri, 2016-06-17 at 08:33 +1000, Benjamin Herrenschmidt wrote:quoted
On Thu, 2016-06-16 at 22:49 +0300, Denis Kirjanov wrote:quoted
- +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2fAre we taking an SLB miss before we do the fixup maybe ?Yeah that's the only explanation that makes any sense. I think instead of patching down this low we should instead be redirecting SLB misses to unknown_exception() when radix is enabled. Aneesh?The 2f branch ends up doing unrecoverable exception. Or are you suggesting something else ?
I meant more like diverting to unknown_exception() higher up the call stack, but that's complicated. How about this? Denis does this work? cheers
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 4c9440629128..8bcc1b457115 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S@@ -1399,11 +1399,12 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX) lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */ mtlr r10 -BEGIN_MMU_FTR_SECTION - b 2f -END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ +BEGIN_MMU_FTR_SECTION beq- 2f +FTR_SECTION_ELSE + b 2f +ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX) .machine push .machine "power4"
From: Aneesh Kumar K.V <hidden>
Date: 2016-06-20 10:21:26
Michael Ellerman [off-list ref] writes:
On Sat, 2016-06-18 at 22:47 +0530, Aneesh Kumar K.V wrote:quoted
Michael Ellerman [off-list ref] writes:quoted
On Fri, 2016-06-17 at 08:33 +1000, Benjamin Herrenschmidt wrote:quoted
On Thu, 2016-06-16 at 22:49 +0300, Denis Kirjanov wrote:quoted
- +BEGIN_MMU_FTR_SECTION + b 2f +END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ beq- 2fAre we taking an SLB miss before we do the fixup maybe ?Yeah that's the only explanation that makes any sense. I think instead of patching down this low we should instead be redirecting SLB misses to unknown_exception() when radix is enabled. Aneesh?The 2f branch ends up doing unrecoverable exception. Or are you suggesting something else ?I meant more like diverting to unknown_exception() higher up the call stack, but that's complicated. How about this? Denis does this work? cheersdiff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 4c9440629128..8bcc1b457115 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S@@ -1399,11 +1399,12 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX) lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */ mtlr r10 -BEGIN_MMU_FTR_SECTION - b 2f -END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ +BEGIN_MMU_FTR_SECTION beq- 2f +FTR_SECTION_ELSE + b 2f +ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX) .machine push .machine "power4"
I sent a patch which should get this problem fixed. http://mid.gmane.org/1466274479-5650-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com -aneesh
From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2016-06-21 00:59:45
On Mon, 2016-06-20 at 15:51 +0530, Aneesh Kumar K.V wrote:
Michael Ellerman [off-list ref] writes:quoted
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 4c9440629128..8bcc1b457115 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S@@ -1399,11 +1399,12 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX) lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */ mtlr r10 -BEGIN_MMU_FTR_SECTION - b 2f -END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ +BEGIN_MMU_FTR_SECTION beq- 2f +FTR_SECTION_ELSE + b 2f +ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX) .machine push .machine "power4"I sent a patch which should get this problem fixed. http://mid.gmane.org/1466274479-5650-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com
Well s/fixed/avoided/. I'd rather we fixed the root cause, which is that the SLB miss handler is broken until code patching happens. When possible we should write feature sections so that the unpatched code is functional, to avoid problems like this. cheers
From: Denis Kirjanov <hidden>
Date: 2016-06-21 04:23:48
How about this? Denis does this work? cheersdiff --git a/arch/powerpc/kernel/exceptions-64s.Sb/arch/powerpc/kernel/exceptions-64s.S index 4c9440629128..8bcc1b457115 100644--- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S@@ -1399,11 +1399,12 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX) lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */ mtlr r10 -BEGIN_MMU_FTR_SECTION - b 2f -END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX) andi. r10,r12,MSR_RI /* check for unrecoverable exception */ +BEGIN_MMU_FTR_SECTION beq- 2f +FTR_SECTION_ELSE + b 2f +ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX) .machine push .machine "power4"
Yeah, it works. Thanks