This is the latest batch of mu 8xx MMU/TLB rework.
I think this is complete now and will relax with
other work the next few days. I hope I can get some
testing from Scott and Rex during this time.
Joakim Tjernlund (8):
8xx: invalidate non present TLBs
8xx: Update TLB asm so it behaves as linux mm expects.
8xx: Tag DAR with 0x00f0 to catch buggy instructions.
8xx: Fixup DAR from buggy dcbX instructions.
8xx: dcbst sets store bit in DTLB error, workaround.
8xx: Add missing Guarded setting in DTLB Error.
8xx: Restore _PAGE_WRITETHRU
8xx: start using dcbX instructions in various copy routines
arch/powerpc/include/asm/pte-8xx.h | 14 +-
arch/powerpc/kernel/head_8xx.S | 307 ++++++++++++++++++++++++++++++------
arch/powerpc/kernel/misc_32.S | 18 --
arch/powerpc/lib/copy_32.S | 24 ---
arch/powerpc/mm/fault.c | 8 +-
5 files changed, 269 insertions(+), 102 deletions(-)
8xx sometimes need to load a invalid/non-present TLBs in
it DTLB asm handler.
These must be invalidated separaly as linux mm don't.
---
arch/powerpc/mm/fault.c | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)
@@ -243,6 +243,12 @@ good_area:gotobad_area;#endif /* CONFIG_6xx */#if defined(CONFIG_8xx)+/* 8xx sometimes need to load a invalid/non-present TLBs.+*Thesemustbeinvalidatedseparatelyaslinuxmmdon't.+*/+if(error_code&0x40000000)/* no translation? */+_tlbil_va(address);+/* The MPC8xx seems to always set 0x80000000, which is*"undefined".Ofthosethatcanbeset,thisistheonly*onewhichseemsbad.
dcbz, dcbf, dcbi, dcbst and icbi do not set DAR when they
cause a DTLB Error. Dectect this by tagging DAR with 0x00f0
at every exception exit that modifies DAR.
Test for DAR=0x00f0 in DataTLBError and bail
to handle_page_fault().
---
arch/powerpc/kernel/head_8xx.S | 15 ++++++++++++++-
1 files changed, 14 insertions(+), 1 deletions(-)
Update the TLB asm to make proper use of _PAGE_DIRY and _PAGE_ACCESSED.
Get rid of _PAGE_HWWRITE too.
Pros:
- I/D TLB Miss never needs to write to the linux pte.
- _PAGE_ACCESSED is only set on TLB Error fixing accounting
- _PAGE_DIRTY is mapped to 0x100, the changed bit, and is set directly
when a page has been made dirty.
- Proper RO/RW mapping of user space.
- Free up 2 SW TLB bits in the linux pte(add back _PAGE_WRITETHRU ?)
- Less instructions in I/D TLB Miss.
- kernel RO/user NA support.
Cons:
- A few more instructions in the TLB Miss routines.
---
arch/powerpc/include/asm/pte-8xx.h | 13 ++---
arch/powerpc/kernel/head_8xx.S | 99 ++++++++++++++++++-----------------
2 files changed, 57 insertions(+), 55 deletions(-)
@@ -32,22 +32,21 @@#define _PAGE_FILE 0x0002 /* when !present: nonlinear file mapping */#define _PAGE_NO_CACHE 0x0002 /* I: cache inhibit */#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */+#define _PAGE_DIRTY 0x0100 /* C: page changed */-/* These five software bits must be masked out when the entry is loaded-*intotheTLB.+/* These 3 software bits must be masked out when the entry is loaded+*intotheTLB,2SWbitsleft.*/#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */#define _PAGE_GUARDED 0x0010 /* software: guarded access */-#define _PAGE_DIRTY 0x0020 /* software: page changed */-#define _PAGE_RW 0x0040 /* software: user write access allowed */-#define _PAGE_ACCESSED 0x0080 /* software: page referenced */+#define _PAGE_ACCESSED 0x0020 /* software: page referenced *//* Setting any bits in the nibble with the follow two controls will*requireaTLBexceptionhandlerchange.Itisassumedunusedbits*arealwayszero.*/-#define _PAGE_HWWRITE 0x0100 /* h/w write enable: never set in Linux PTE */-#define _PAGE_USER 0x0800 /* One of the PP bits, the other is USER&~RW */+#define _PAGE_RW 0x0400 /* lsb PP bits, inverted in HW */+#define _PAGE_USER 0x0800 /* msb PP bits */#define _PMD_PRESENT 0x0001#define _PMD_BAD 0x0ff0
@@ -333,26 +333,20 @@ InstructionTLBMiss:mfsprr11,SPRN_MD_TWC/*....andgetthepteaddress*/lwzr10,0(r11)/*Getthepte*/-#ifdef CONFIG_SWAP-/*donotsetthe_PAGE_ACCESSEDbitofanon-presentpage*/-andi.r11,r10,_PAGE_PRESENT-beq4f-orir10,r10,_PAGE_ACCESSED-mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/-stwr10,0(r11)-4:-#else-orir10,r10,_PAGE_ACCESSED-stwr10,0(r11)-#endif+andi.r11,r10,_PAGE_ACCESSED|_PAGE_PRESENT+cmpwicr0,r11,_PAGE_ACCESSED|_PAGE_PRESENT+bne-cr0,2f++/*ClearPPlsb,0x400*/+rlwinmr10,r10,0,22,20/*TheLinuxPTEwon't go exactly into the MMU TLB.-*Softwareindicatorbits21,22and28mustbeclear.+*Softwareindicatorbits22and28mustbeclear.*Softwareindicatorbits24,25,26,and27mustbe*set.AllotherLinuxPTEbitscontrolthebehavior*oftheMMU.*/-2:lir11,0x00f0+lir11,0x00f0rlwimir10,r11,0,24,28/*Set24-27,clear28*/DO_8xx_CPU6(0x2d80,r3)mtsprSPRN_MI_RPN,r10/*UpdateTLBentry*/
@@ -409,21 +419,27 @@ DataStoreTLBMiss:DO_8xx_CPU6(0x3b80,r3)mtsprSPRN_MD_TWC,r11-#ifdef CONFIG_SWAP-/*donotsetthe_PAGE_ACCESSEDbitofanon-presentpage*/-andi.r11,r10,_PAGE_PRESENT-beq4f-orir10,r10,_PAGE_ACCESSED-4:-/*andupdatepteintable*/-#else-orir10,r10,_PAGE_ACCESSED-#endif-mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/-stwr10,0(r11)+/*Both_PAGE_ACCESSEDand_PAGE_PRESENThastobeset.+*Wealsoneedtoknowiftheinsnisaload/store,so:+*Clear_PAGE_PRESENTandloadthatwhichwill+*trapintoDTLBErrorwithstorebitsetaccordinly.+*/+/*PRESENT=0x1,ACCESSED=0x20+*r11=((r10&PRESENT)&((r10&ACCESSED)>>5));+*r10=(r10&~PRESENT)|r11;+*/+rlwinmr11,r10,32-5,31,31+andr11,r11,r10+rlwimir10,r11,0,31,31++/*HonourkernelRO,UserNA*/+andi.r11,r10,_PAGE_USER|_PAGE_RW+bne-cr0,5f+orir10,r10,0x200/*Extendedencoding,bit22*/+5:xorir10,r10,_PAGE_RW/*invertRWbit*//*TheLinuxPTEwon't go exactly into the MMU TLB.-*Softwareindicatorbits21,22and28mustbeclear.+*Softwareindicatorbits22and28mustbeclear.*Softwareindicatorbits24,25,26,and27mustbe*set.AllotherLinuxPTEbitscontrolthebehavior*oftheMMU.
@@ -522,26 +539,12 @@ DataTLBError:mfsprr11,SPRN_MD_TWC/*....andgetthepteaddress*/lwzr10,0(r11)/*Getthepte*/-andi.r11,r10,_PAGE_RW/*Isitwriteable?*/-beq2f/*Bailoutifnot*/--/*Update'changed',amongothers.-*/-#ifdef CONFIG_SWAP-orir10,r10,_PAGE_DIRTY|_PAGE_HWWRITE-/*donotsetthe_PAGE_ACCESSEDbitofanon-presentpage*/-andi.r11,r10,_PAGE_PRESENT-beq4f-orir10,r10,_PAGE_ACCESSED-4:-#else-orir10,r10,_PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE-#endif-mfsprr11,SPRN_MD_TWC/*Getpteaddressagain*/+orir10,r10,_PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITEstwr10,0(r11)/*andupdatepteintable*/+xorir10,r10,_PAGE_RW/*RWbitisinverted*//*TheLinuxPTEwon't go exactly into the MMU TLB.-*Softwareindicatorbits21,22and28mustbeclear.+*Softwareindicatorbits22and28mustbeclear.*Softwareindicatorbits24,25,26,and27mustbe*set.AllotherLinuxPTEbitscontrolthebehavior*oftheMMU.
This is an assembler version to fixup DAR not being set
by dcbX, icbi instructions. There are two versions, one
uses selfmodifing code, the other uses a
jump table but is much bigger(default).
---
arch/powerpc/kernel/head_8xx.S | 146 +++++++++++++++++++++++++++++++++++++++-
1 files changed, 145 insertions(+), 1 deletions(-)
@@ -604,6 +605,149 @@ DataTLBError:.=0x2000+/*ThisistheproceduretocalculatethedataEAforbuggydcbx,dcbiinstructions+*bydecodingtheregistersusedbythedcbxinstructionandaddingthem.+*DARissettothecalculatedaddressandr10alsoholdstheEAonexit.+*/+#define NO_SELF_MODIFYING_CODE /* define if you don't want to use self modifying code */+nop/*Afewnopstomakethemodified_instr:spacebelowcachelinealigned*/+nop+139:/*fetchinstructionfromuserspacememory*/+DO_8xx_CPU6(0x3780,r3)+mtsprSPRN_MD_EPN,r10+mfsprr11,SPRN_M_TWB/*Getlevel1tableentryaddress*/+lwzr11,0(r11)/*Getthelevel1entry*/+tophys (r11,r11)+DO_8xx_CPU6(0x3b80,r3)+mtsprSPRN_MD_TWC,r11/*Loadptetablebaseaddress*/+mfsprr11,SPRN_MD_TWC/*....andgetthepteaddress*/+lwzr11,0(r11)/*Getthepte*/+/*concatphysicalpageaddress(r11)andpageoffset(r10)*/+rlwimir11,r10,0,20,31+b140f+FixDAR:/*Entrypointfordcbxworkaround.*/+/*fetchinstructionfrommemory.*/+mfsprr10,SPRN_SRR0+andis.r11,r10,0x8000+tophys (r11,r10)+beq-139b/*Branchifuserspaceaddress*/+140:lwzr11,0(r11)+#ifdef CONFIG_8xx_CPU6+lwzr3,8(r0)/*restorer3frommemory*/+#endif+#ifndef NO_SELF_MODIFYING_CODE+andis.r10,r11,0x1f/*testifregRAisr0*/+lir10,modified_instr@l+dcbtstr0,r10/*touchforstore*/+rlwinmr11,r11,0,0,20/*Zerolower10bits*/+orisr11,r11,640/*Transforminstr.toa"add r10,RA,RB"*/+orir11,r11,532+stwr11,0(r10)/*storeadd/andinstruction*/+dcbf0,r10/*flushnewinstr.tomemory.*/+icbi0,r10/*invalidateinstr.cacheline*/+lwzr11,4(r0)/*restorer11frommemory*/+mfsprr10,SPRN_M_TW/*restorer10fromM_TW*/+isync/*Waituntilnewinstrisloadedfrommemory*/+modified_instr:+.space4/*thisiswheretheadd/andinstr.isstored*/+bne+143f+subfr10,r0,r10/*r10=r10-r0,onlyifregRAisr0*/+143:mtdarr10/*storefaultingEAinDAR*/+bDARFix/*GobacktonormalTLBhandling*/+#else+mfctrr10+mtdarr10/*savectrreginDAR*/+rlwinmr10,r11,24,24,28/*offsetintojumptableforregRB*/+addir10,r10,150f@l/*addstartoftable*/+mtctrr10/*loadctrwithjumpaddress*/+xorr10,r10,r10/*sumstartsatzero*/+bctr/*jumpintotable*/+150:+addr10,r10,r0+b151f+addr10,r10,r1+b151f+addr10,r10,r2+b151f+addr10,r10,r3+b151f+addr10,r10,r4+b151f+addr10,r10,r5+b151f+addr10,r10,r6+b151f+addr10,r10,r7+b151f+addr10,r10,r8+b151f+addr10,r10,r9+b151f+addr10,r10,r10+b151f+addr10,r10,r11+b151f+addr10,r10,r12+b151f+addr10,r10,r13+b151f+addr10,r10,r14+b151f+addr10,r10,r15+b151f+addr10,r10,r16+b151f+addr10,r10,r17+b151f+addr10,r10,r18+b151f+addr10,r10,r19+b151f+mtctrr11/*r10needsspecialhandling*/+b154f+mtctrr11/*r11needsspecialhandling*/+b153f+addr10,r10,r22+b151f+addr10,r10,r23+b151f+addr10,r10,r24+b151f+addr10,r10,r25+b151f+addr10,r10,r25+b151f+addr10,r10,r27+b151f+addr10,r10,r28+b151f+addr10,r10,r29+b151f+addr10,r10,r30+b151f+addr10,r10,r31+151:+rlwinm.r11,r11,19,24,28/*offsetintojumptableforregRA*/+beq152f/*ifregRAiszero,don't add it */ +addir11,r11,150b@l/*addstartoftable*/+mtctrr11/*loadctrwithjumpaddress*/+rlwinmr11,r11,0,16,10/*makesurewedon't execute this more than once */+bctr/*jumpintotable*/+152:+mfdarr11+mtctrr11/*restorectrregfromDAR*/+mtdarr10/*savefaultEAtoDAR*/+bDARFix/*GobacktonormalTLBhandling*/++/*specialhandlingforr10,r11sincethesearemodifiedalready*/+153:lwzr11,4(r0)/*loadr11frommemory*/+b155f+154:mfsprr11,SPRN_M_TW/*loadr10fromM_TW*/+155:addr10,r10,r11/*addit*/+mfctrr11/*restorer11*/+b151b+#endif+.globlgiveup_fpugiveup_fpu:blr
8xx has not had WRITETHRU due to lack of bits in the pte.
After the recent rewrite of the 8xx TLB code, there are
two bits left. Use one of them to WRITETHRU.
Perhaps use the last SW bit to PAGE_SPECIAL or PAGE_FILE?
---
arch/powerpc/include/asm/pte-8xx.h | 5 +++--
arch/powerpc/kernel/head_8xx.S | 8 ++++++++
2 files changed, 11 insertions(+), 2 deletions(-)
@@ -34,12 +34,13 @@#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */#define _PAGE_DIRTY 0x0100 /* C: page changed */-/* These 3 software bits must be masked out when the entry is loaded-*intotheTLB,2SWbitsleft.+/* These 4 software bits must be masked out when the entry is loaded+*intotheTLB,1SWbitleft(0x0080).*/#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */#define _PAGE_GUARDED 0x0010 /* software: guarded access */#define _PAGE_ACCESSED 0x0020 /* software: page referenced */+#define _PAGE_WRITETHRU 0x0040 /* software: caching is write through *//* Setting any bits in the nibble with the follow two controls will*requireaTLBexceptionhandlerchange.Itisassumedunusedbits
@@ -559,6 +563,10 @@ DARFix: /* Return from dcbx instruction bug workaround, r10 holds value of DAR **Itisbit27ofboththeLinuxPTEandtheTWC*/rlwimir11,r10,0,27,27+/*InserttheWriteThruflagintotheTWCfromtheLinuxPTE.+*Itisbit25intheLinuxPTEandbit30intheTWC+*/+rlwimir11,r10,32-5,30,30DO_8xx_CPU6(0x3b80,r3)mtsprSPRN_MD_TWC,r11mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/
Now that 8xx can fixup dcbX instructions, start using them
where possible like every other PowerPc arch do.
---
arch/powerpc/kernel/misc_32.S | 18 ------------------
arch/powerpc/lib/copy_32.S | 24 ------------------------
2 files changed, 0 insertions(+), 42 deletions(-)
@@ -528,15 +520,6 @@ _GLOBAL(copy_page)addir3,r3,-4addir4,r4,-4-#ifdef CONFIG_8xx-/*don't use prefetch on 8xx */-lir0,4096/L1_CACHE_BYTES-mtctrr0-1:COPY_16_BYTES-bdnz1b-blr--#else /* not 8xx, we can prefetch */lir5,4#if MAX_COPY_PREFETCH > 1
@@ -356,14 +341,6 @@ _GLOBAL(__copy_tofrom_user)lir11,4beq63f-#ifdef CONFIG_8xx-/*Don't use prefetch on 8xx */-mtctrr0-lir0,0-53:COPY_16_BYTES_WITHEX(0)-bdnz53b--#else /* not CONFIG_8xx *//*Herewedecidehowfaraheadtoprefetchthesource*/lir3,4cmpwir0,1
only DTLB Miss did set this bit, DTLB Error needs too otherwise
the setting is lost when the page becomes dirty.
---
arch/powerpc/kernel/head_8xx.S | 13 ++++++++++---
1 files changed, 10 insertions(+), 3 deletions(-)
@@ -552,9 +552,16 @@ DARFix: /* Return from dcbx instruction bug workaround, r10 holds value of DAR **/orir11,r11,1/*SetvalidbitinphysicalL2page*/DO_8xx_CPU6(0x3b80,r3)-mtsprSPRN_MD_TWC,r11/*Loadptetablebaseaddress*/-mfsprr11,SPRN_MD_TWC/*....andgetthepteaddress*/-lwzr10,0(r11)/*Getthepte*/+mtsprSPRN_MD_TWC,r11/*Loadptetablebaseaddress*/+mfsprr10,SPRN_MD_TWC/*....andgetthepteaddress*/+lwzr10,0(r10)/*Getthepte*/+/*InserttheGuardedflagintotheTWCfromtheLinuxPTE.+*Itisbit27ofboththeLinuxPTEandtheTWC+*/+rlwimir11,r10,0,27,27+DO_8xx_CPU6(0x3b80,r3)+mtsprSPRN_MD_TWC,r11+mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/orir10,r10,_PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITEstwr10,0(r11)/*andupdatepteintable*/
dcbst should not set the store bit(bit 6, DSISR) when
trapping into a DTLB Error. Clear this bit while doing
the dcbX missing DAR workaround.
---
arch/powerpc/kernel/head_8xx.S | 34 +++++++++++++++++++++++++++++++---
1 files changed, 31 insertions(+), 3 deletions(-)
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-11 21:25:18
On Sun, 2009-10-11 at 18:35 +0200, Joakim Tjernlund wrote:
quoted hunk
@@ -522,26 +539,12 @@ DataTLBError:
I still think you should remove everything from DataTLBError, it's
just plain useless :-)
Ben.
mfspr r11, SPRN_MD_TWC /* ....and get the pte
address */
lwz r10, 0(r11) /* Get the pte */
- andi. r11, r10, _PAGE_RW /* Is it writeable? */
- beq 2f /* Bail out if not */
-
- /* Update 'changed', among others.
- */
-#ifdef CONFIG_SWAP
- ori r10, r10, _PAGE_DIRTY|_PAGE_HWWRITE
- /* do not set the _PAGE_ACCESSED bit of a non-present page */
- andi. r11, r10, _PAGE_PRESENT
- beq 4f
- ori r10, r10, _PAGE_ACCESSED
-4:
-#else
- ori r10, r10, _PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE
-#endif
- mfspr r11, SPRN_MD_TWC /* Get pte address
again */
+ ori r10, r10, _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_HWWRITE
stw r10, 0(r11) /* and update pte in table */
+ xori r10, r10, _PAGE_RW /* RW bit is inverted */
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 21, 22 and 28 must be clear.
+ * Software indicator bits 22 and 28 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
--
1.6.4.4
@@ -552,9 +552,16 @@ DARFix: /* Return from dcbx instruction bug workaround, r10 holds value of DAR **/orir11,r11,1/*SetvalidbitinphysicalL2page*/DO_8xx_CPU6(0x3b80,r3)-mtsprSPRN_MD_TWC,r11/*Loadptetablebaseaddress*/-mfsprr11,SPRN_MD_TWC/*....andgetthepteaddress*/-lwzr10,0(r11)/*Getthepte*/+mtsprSPRN_MD_TWC,r11/*Loadptetablebaseaddress*/+mfsprr10,SPRN_MD_TWC/*....andgetthepteaddress*/+lwzr10,0(r10)/*Getthepte*/+/*InserttheGuardedflagintotheTWCfromtheLinuxPTE.+*Itisbit27ofboththeLinuxPTEandtheTWC+*/+rlwimir11,r10,0,27,27+DO_8xx_CPU6(0x3b80,r3)+mtsprSPRN_MD_TWC,r11+mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/orir10,r10,_PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITEstwr10,0(r11)/*andupdatepteintable*/
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-11 21:26:36
On Sun, 2009-10-11 at 18:35 +0200, Joakim Tjernlund wrote:
8xx has not had WRITETHRU due to lack of bits in the pte.
After the recent rewrite of the 8xx TLB code, there are
two bits left. Use one of them to WRITETHRU.
Perhaps use the last SW bit to PAGE_SPECIAL or PAGE_FILE?
_PAGE_FILE can already overwrite other bits as it's only set
when !present, and should pretty much always be 0x2
I think I've replaced _PAGE_EXEC with _PAGE_SPECIAL already
upstream since _PAGE_EXEC is unused on 8xx.
Cheers,
Ben.
@@ -34,12 +34,13 @@#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */#define _PAGE_DIRTY 0x0100 /* C: page changed */-/* These 3 software bits must be masked out when the entry is loaded-*intotheTLB,2SWbitsleft.+/* These 4 software bits must be masked out when the entry is loaded+*intotheTLB,1SWbitleft(0x0080).*/#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */#define _PAGE_GUARDED 0x0010 /* software: guarded access */#define _PAGE_ACCESSED 0x0020 /* software: page referenced */+#define _PAGE_WRITETHRU 0x0040 /* software: caching is write through *//* Setting any bits in the nibble with the follow two controls will*requireaTLBexceptionhandlerchange.Itisassumedunusedbits
@@ -559,6 +563,10 @@ DARFix: /* Return from dcbx instruction bug workaround, r10 holds value of DAR **Itisbit27ofboththeLinuxPTEandtheTWC*/rlwimir11,r10,0,27,27+/*InserttheWriteThruflagintotheTWCfromtheLinuxPTE.+*Itisbit25intheLinuxPTEandbit30intheTWC+*/+rlwimir11,r10,32-5,30,30DO_8xx_CPU6(0x3b80,r3)mtsprSPRN_MD_TWC,r11mfsprr11,SPRN_MD_TWC/*getthepteaddressagain*/
Benjamin Herrenschmidt [off-list ref] wrote on 11/10/2009 23:25:30:
On Sun, 2009-10-11 at 18:35 +0200, Joakim Tjernlund wrote:
quoted
only DTLB Miss did set this bit, DTLB Error needs too otherwise
the setting is lost when the page becomes dirty.
Easier fix: Stop doing thing in DTLB Error
Ben.
I hear you, I can remove DTLB error with an add on patch later if that is OK?
I cannot remove the DARFix though, when I move that to do_page_fault(), I get
duplicate TLB hits on the same insn. It is like when transfer_to_handler()
executes rfi, the cpu restarts the the faulting insn instead of jumping
to the page fault handler, not always but often.
Jocke
Benjamin Herrenschmidt [off-list ref] wrote on 11/10/2009 23:26:27:
On Sun, 2009-10-11 at 18:35 +0200, Joakim Tjernlund wrote:
quoted
8xx has not had WRITETHRU due to lack of bits in the pte.
After the recent rewrite of the 8xx TLB code, there are
two bits left. Use one of them to WRITETHRU.
Perhaps use the last SW bit to PAGE_SPECIAL or PAGE_FILE?
_PAGE_FILE can already overwrite other bits as it's only set
when !present, and should pretty much always be 0x2
OK.
I think I've replaced _PAGE_EXEC with _PAGE_SPECIAL already
upstream since _PAGE_EXEC is unused on 8xx.
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-11 22:45:08
On Mon, 2009-10-12 at 00:19 +0200, Joakim Tjernlund wrote:
I hear you, I can remove DTLB error with an add on patch later if that is OK?
I cannot remove the DARFix though, when I move that to do_page_fault(), I get
duplicate TLB hits on the same insn. It is like when transfer_to_handler()
executes rfi, the cpu restarts the the faulting insn instead of jumping
to the page fault handler, not always but often.
Benjamin Herrenschmidt [off-list ref] wrote on 12/10/2009 00:44:56:
On Mon, 2009-10-12 at 00:19 +0200, Joakim Tjernlund wrote:
quoted
I hear you, I can remove DTLB error with an add on patch later if that is OK?
I cannot remove the DARFix though, when I move that to do_page_fault(), I get
duplicate TLB hits on the same insn. It is like when transfer_to_handler()
executes rfi, the cpu restarts the the faulting insn instead of jumping
to the page fault handler, not always but often.
I'm not sure what you mean here ...
Just that I need to keep the DAR fix for dcbX insn in the DTLB handler. If I try
to move it to do_page_fault() I get a lot more DTLB errors for dcbX insn.
Jocke
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-12 05:46:23
On Mon, 2009-10-12 at 07:36 +0200, Joakim Tjernlund wrote:
Benjamin Herrenschmidt [off-list ref] wrote on 12/10/2009 00:44:56:
quoted
On Mon, 2009-10-12 at 00:19 +0200, Joakim Tjernlund wrote:
quoted
I hear you, I can remove DTLB error with an add on patch later if that is OK?
I cannot remove the DARFix though, when I move that to do_page_fault(), I get
duplicate TLB hits on the same insn. It is like when transfer_to_handler()
executes rfi, the cpu restarts the the faulting insn instead of jumping
to the page fault handler, not always but often.
I'm not sure what you mean here ...
Just that I need to keep the DAR fix for dcbX insn in the DTLB handler. If I try
to move it to do_page_fault() I get a lot more DTLB errors for dcbX insn.
I'm not sure why (ie, I didn't get your explanation about rfi and
restarting the faulting insn etc...) but ok, I don't mind having
the DAR fixup remain in the asm. It's the whole logic that looks
at the PTE and does things with it that I feel has no room in there :-)
BTW. Maybe the do_page_fault() thing comes from the fact that we
also go there via ITLB Error which doesn't set the DAR and
that's normal ?
Cheers,
Ben.
Benjamin Herrenschmidt [off-list ref] wrote on 12/10/2009 07:46:14:
On Mon, 2009-10-12 at 07:36 +0200, Joakim Tjernlund wrote:
quoted
Benjamin Herrenschmidt [off-list ref] wrote on 12/10/2009 00:44:56:
quoted
On Mon, 2009-10-12 at 00:19 +0200, Joakim Tjernlund wrote:
quoted
I hear you, I can remove DTLB error with an add on patch later if that is OK?
I cannot remove the DARFix though, when I move that to do_page_fault(), I get
duplicate TLB hits on the same insn. It is like when transfer_to_handler()
executes rfi, the cpu restarts the the faulting insn instead of jumping
to the page fault handler, not always but often.
I'm not sure what you mean here ...
Just that I need to keep the DAR fix for dcbX insn in the DTLB handler. If I try
to move it to do_page_fault() I get a lot more DTLB errors for dcbX insn.
I'm not sure why (ie, I didn't get your explanation about rfi and
restarting the faulting insn etc...) but ok, I don't mind having
the DAR fixup remain in the asm. It's the whole logic that looks
at the PTE and does things with it that I feel has no room in there :-)
OK, I will send a removal patch too.
BTW. Maybe the do_page_fault() thing comes from the fact that we
also go there via ITLB Error which doesn't set the DAR and
that's normal ?
I had that idea to, but no. These have different traps numbers and
address is set to SRR0 for ITLB and to DAR for DTLB. I test for DTLB
trap number before doing anything.
From: Scott Wood <hidden> Date: 2009-10-14 16:56:45
On Sun, Oct 11, 2009 at 06:35:05PM +0200, Joakim Tjernlund wrote:
quoted hunk
8xx sometimes need to load a invalid/non-present TLBs in
it DTLB asm handler.
These must be invalidated separaly as linux mm don't.
---
arch/powerpc/mm/fault.c | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)
@@ -243,6 +243,12 @@ good_area:gotobad_area;#endif /* CONFIG_6xx */#if defined(CONFIG_8xx)+/* 8xx sometimes need to load a invalid/non-present TLBs.+*Thesemustbeinvalidatedseparatelyaslinuxmmdon't.+*/+if(error_code&0x40000000)/* no translation? */+_tlbil_va(address);
arch/powerpc/mm/fault.c:253: error: too few arguments to function ‘_tlbil_va’
-Scott
From: Scott Wood <hidden> Date: 2009-10-14 17:19:41
On Sun, Oct 11, 2009 at 06:35:08PM +0200, Joakim Tjernlund wrote:
quoted hunk
This is an assembler version to fixup DAR not being set
by dcbX, icbi instructions. There are two versions, one
uses selfmodifing code, the other uses a
jump table but is much bigger(default).
---
arch/powerpc/kernel/head_8xx.S | 146 +++++++++++++++++++++++++++++++++++++++-
1 files changed, 145 insertions(+), 1 deletions(-)
Both FixDAR and DARFix? Could we make the labels a little clearer?
+/* This is the procedure to calculate the data EA for buggy dcbx,dcbi instructions
+ * by decoding the registers used by the dcbx instruction and adding them.
+ * DAR is set to the calculated address and r10 also holds the EA on exit.
+ */
How often does this happen? Could we just do it in C code after saving all
the registers, and avoid the self modifying stuff (or the big switch
statement equivalent)?
-Scott
From: Scott Wood <hidden> Date: 2009-10-14 17:23:28
On Sun, Oct 11, 2009 at 06:35:04PM +0200, Joakim Tjernlund wrote:
This is the latest batch of mu 8xx MMU/TLB rework.
I think this is complete now and will relax with
other work the next few days. I hope I can get some
testing from Scott and Rex during this time.
I applied this stack plus "Remove DIRTY pte handling in DTLB Error" (fixing
up conflicts again, as well as the noted build errors), and got this:
INIT: version 2.85 booting
Mounting /proc and /sys
Starting the hotplug events dispatcher udevd
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:4
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:10
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:14
Synthesizing initial hotplug events
/etc/rc.d/init.d/udev: line 41: 187 Segmentation fault udevsettle --timeout=300
Setting the hostname to 8xx
Running depmod
WARNING: Couldn't open directory /lib/modules/2.6.31-08384-g2cb4b47-dirty: No such file or directory
FATAL: Could not open /lib/modules/2.6.31-08384-g2cb4b47-dirty/modules.dep.tempfor writing: No such file or directory
Mounting filesystems
BUG: soft lockup - CPU#0 stuck for 61s! [cat:265]
NIP: c000f160 LR: c000f160 CTR: 00000007
REGS: c3919c70 TRAP: 0901 Not tainted (2.6.31-08384-g2cb4b47-dirty)
MSR: 00009032 <EE,ME,IR,DR> CR: 88008424 XER: 00000000
TASK = c3944bd0[265] 'cat' THREAD: c3918000
GPR00: c000f160 c3919d20 c3944bd0 00000000 100180fc 00000000 00000000 00000001
GPR08: c393e700 00000000 03ca9d21 00000001 48000428
NIP [c000f160] do_page_fault+0x188/0x49c
LR [c000f160] do_page_fault+0x188/0x49c
Call Trace:
[c3919d20] [c000f160] do_page_fault+0x188/0x49c (unreliable)
[c3919dd0] [c000e3f0] handle_page_fault+0xc/0x80
[c3919e90] [c008910c] seq_read+0x2a4/0x558
[c3919ee0] [c00ab7c0] proc_reg_read+0x4c/0x70
[c3919ef0] [c006f47c] vfs_read+0xb4/0x158
[c3919f10] [c006f78c] sys_read+0x4c/0x90
[c3919f40] [c000dfc0] ret_from_syscall+0x0/0x38
Instruction dump:
2f990000 419e01f0 801f0014 700a0002 418201f8 7c1900d0 541e0ffe 7fe4fb78
7f85e378 7fc6f378 7f63db78 4804c31d <70690003> 7c601b78 40820230 70690004
-Scott
Scott Wood [off-list ref] wrote on 14/10/2009 19:23:51:
On Sun, Oct 11, 2009 at 06:35:04PM +0200, Joakim Tjernlund wrote:
quoted
This is the latest batch of mu 8xx MMU/TLB rework.
I think this is complete now and will relax with
other work the next few days. I hope I can get some
testing from Scott and Rex during this time.
I applied this stack plus "Remove DIRTY pte handling in DTLB Error" (fixing
up conflicts again, as well as the noted build errors), and got this:
Sorry about the build problems, will fixup.
Don't know what is causing the error though, works just fine on 2.4 :(
Assuming you fixup this error:
arch/powerpc/kernel/head_8xx.S:577: undefined reference to `DARfix'
with a "b DARFix" I can only guess and I don't have a good guess even :(
You could back out
8xx: start using dcbX instructions in various copy routines
to see if it is the dcbX insn that is causing the error.
INIT: version 2.85 booting
Mounting /proc and /sys
Starting the hotplug events dispatcher udevd
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:4
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:10
udevd[177]: add_to_rules: unknown key 'MODALIAS' in /etc/udev/rules.d/60-pcmcia.rules:14
Synthesizing initial hotplug events
/etc/rc.d/init.d/udev: line 41: 187 Segmentation fault udevsettle --timeout=300
This looks like the first error?
Setting the hostname to 8xx
Running depmod
WARNING: Couldn't open directory /lib/modules/2.6.31-08384-g2cb4b47-dirty: No
such file or directory
FATAL: Could not open /lib/modules/2.6.31-08384-g2cb4b47-dirty/
modules.dep.tempfor writing: No such file or directory
Mounting filesystems
BUG: soft lockup - CPU#0 stuck for 61s! [cat:265]
Perhaps it is looping on the same TLB Error, caused by dcbX insn?
Scott Wood [off-list ref] wrote on 14/10/2009 19:20:03:
On Sun, Oct 11, 2009 at 06:35:08PM +0200, Joakim Tjernlund wrote:
quoted
This is an assembler version to fixup DAR not being set
by dcbX, icbi instructions. There are two versions, one
uses selfmodifing code, the other uses a
jump table but is much bigger(default).
---
arch/powerpc/kernel/head_8xx.S | 146 +++++++++++++++++++++++++++++++++++++++-
1 files changed, 145 insertions(+), 1 deletions(-)
Both FixDAR and DARFix? Could we make the labels a little clearer?
Yes, need to come up with better names :)
quoted
+/* This is the procedure to calculate the data EA for buggy dcbx,dcbi instructions
+ * by decoding the registers used by the dcbx instruction and adding them.
+ * DAR is set to the calculated address and r10 also holds the EA on exit.
+ */
How often does this happen? Could we just do it in C code after saving all
the registers, and avoid the self modifying stuff (or the big switch
statement equivalent)?
I had some problems with the C-version. I got lots of extra TLB errors for the same address
so I am not confident it will work in the long run.
BTW, you could add a test and printk in do_page_fault on address 0x000000f0.
if that ever hits there is a problem with dcbX fixup.
Scott Wood [off-list ref] wrote on 14/10/2009 21:23:02:
Joakim Tjernlund wrote:
quoted
BTW, you could add a test and printk in do_page_fault on address 0x000000f0.
if that ever hits there is a problem with dcbX fixup.
It doesn't get any 0xf0 faults.
FWIW, I'm not seeing the segfault any more, but I still get the lockup.
Have you reverted
8xx: start using dcbX instructions in various copy routines ?
After that you could stick a
b DataAccess
directly in the DTLB error handler to skip and dcbX fixups.
From: Scott Wood <hidden> Date: 2009-10-14 20:22:04
Joakim Tjernlund wrote:
Scott Wood [off-list ref] wrote on 14/10/2009 21:23:02:
quoted
Joakim Tjernlund wrote:
quoted
BTW, you could add a test and printk in do_page_fault on address 0x000000f0.
if that ever hits there is a problem with dcbX fixup.
It doesn't get any 0xf0 faults.
FWIW, I'm not seeing the segfault any more, but I still get the lockup.
Have you reverted
8xx: start using dcbX instructions in various copy routines ?
After that you could stick a
b DataAccess
directly in the DTLB error handler to skip and dcbX fixups.
With that, I don't see the hard lockup, but things get stuck during
bootup with everything idle. I see this even if I revert everything but
the "invalidate non present TLBs" patch, and I was seeing similar things
sometimes with the other tlbil_va hacks.
I think there's something else going on in the 2.6 8xx code that needs
to be fixed before we can tell what the impact of these patches is.
I'll look into it.
-Scott
Scott Wood [off-list ref] wrote on 14/10/2009 22:22:25:
Joakim Tjernlund wrote:
quoted
Scott Wood [off-list ref] wrote on 14/10/2009 21:23:02:
quoted
Joakim Tjernlund wrote:
quoted
BTW, you could add a test and printk in do_page_fault on address 0x000000f0.
if that ever hits there is a problem with dcbX fixup.
It doesn't get any 0xf0 faults.
FWIW, I'm not seeing the segfault any more, but I still get the lockup.
Have you reverted
8xx: start using dcbX instructions in various copy routines ?
After that you could stick a
b DataAccess
directly in the DTLB error handler to skip and dcbX fixups.
With that, I don't see the hard lockup, but things get stuck during
You needed both to loose the hard lockup? I would think
it should be enough to revert the "various copy routines" stuff?
I figure that these routines aren't working in 8xx for other reasons
since they haven't been used on 8xx since at least early 2.4.
bootup with everything idle. I see this even if I revert everything but
the "invalidate non present TLBs" patch, and I was seeing similar things
sometimes with the other tlbil_va hacks.
OK, something else is up.
I think there's something else going on in the 2.6 8xx code that needs
to be fixed before we can tell what the impact of these patches is.
I'll look into it.
Great because I am really out of ideas. Perhaps back down to 2.6.30 and test
from there?
From: Scott Wood <hidden> Date: 2009-10-14 21:14:58
Joakim Tjernlund wrote:
quoted
With that, I don't see the hard lockup, but things get stuck during
You needed both to loose the hard lockup? I would think
it should be enough to revert the "various copy routines" stuff?
No, but when I just reverted the patch and didn't change the TLB error handler,
I got some other weirdness (assertion failure in some userspace program). It
may have been coincidental, though.
quoted
I think there's something else going on in the 2.6 8xx code that needs
to be fixed before we can tell what the impact of these patches is.
I'll look into it.
Great because I am really out of ideas. Perhaps back down to 2.6.30 and test
from there?
I think the last working version was a little older than that -- and it's quite
possible that there was underlying badness even earlier that just recently got
exposed. I think I want to just debug it and find out what's really going on.
-Scott
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-14 21:17:20
On Wed, 2009-10-14 at 16:14 -0500, Scott Wood wrote:
I think the last working version was a little older than that -- and it's quite
possible that there was underlying badness even earlier that just recently got
exposed. I think I want to just debug it and find out what's really going on.
That would be good :-)
I've been itching to do that but without HW it's not trivial :-)
Cheers,
Ben.
Benjamin Herrenschmidt [off-list ref] wrote on 14/10/2009 23:17:09:
On Wed, 2009-10-14 at 16:14 -0500, Scott Wood wrote:
quoted
I think the last working version was a little older than that -- and it's quite
possible that there was underlying badness even earlier that just recently got
exposed. I think I want to just debug it and find out what's really going on.
That would be good :-)
I've been itching to do that but without HW it's not trivial :-)
Meanwhile, how about the tlb asm you promised me? :)
It will be a challenge I think since you only have 2 GPRs
I guess it would be possible to stash yet another reg since it
will fit in the cache line already used by the TLB handlers.
Jocke
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2009-10-14 21:52:18
On Wed, 2009-10-14 at 23:41 +0200, Joakim Tjernlund wrote:
Benjamin Herrenschmidt [off-list ref] wrote on 14/10/2009 23:17:09:
quoted
On Wed, 2009-10-14 at 16:14 -0500, Scott Wood wrote:
quoted
I think the last working version was a little older than that -- and it's quite
possible that there was underlying badness even earlier that just recently got
exposed. I think I want to just debug it and find out what's really going on.
That would be good :-)
I've been itching to do that but without HW it's not trivial :-)
Meanwhile, how about the tlb asm you promised me? :)
It will be a challenge I think since you only have 2 GPRs
I guess it would be possible to stash yet another reg since it
will fit in the cache line already used by the TLB handlers.
Benjamin Herrenschmidt [off-list ref] wrote on 14/10/2009 23:52:10:
On Wed, 2009-10-14 at 23:41 +0200, Joakim Tjernlund wrote:
quoted
Benjamin Herrenschmidt [off-list ref] wrote on 14/10/2009 23:17:09:
quoted
On Wed, 2009-10-14 at 16:14 -0500, Scott Wood wrote:
quoted
I think the last working version was a little older than that -- and it's quite
possible that there was underlying badness even earlier that just recently got
exposed. I think I want to just debug it and find out what's really going on.
That would be good :-)
I've been itching to do that but without HW it's not trivial :-)
Meanwhile, how about the tlb asm you promised me? :)
It will be a challenge I think since you only have 2 GPRs
I guess it would be possible to stash yet another reg since it
will fit in the cache line already used by the TLB handlers.