From: Simon Guo <redacted>
It is a simple patch just for moving kvmppc_save_tm/kvmppc_restore_tm()
functionalities to tm.S. There is no logic change. The reconstruct of
those APIs will be done in later patches to improve readability.
It is for preparation of reusing those APIs on both HV/PR PPC KVM.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/Makefile | 3 +
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 239 ----------------------------
arch/powerpc/kvm/tm.S | 267 ++++++++++++++++++++++++++++++++
3 files changed, 270 insertions(+), 239 deletions(-)
create mode 100644 arch/powerpc/kvm/tm.S
@@ -2951,243 +2949,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)mrr4,r31blr-#ifdef CONFIG_PPC_TRANSACTIONAL_MEM-/*-*SavetransactionalstateandTM-relatedregisters.-*Calledwithr9pointingtothevcpustruct.-*Thiscanmodifyallcheckpointedregisters,but-*restoresr1,r2andr9 (vcpupointer)beforeexit.-*/-kvmppc_save_tm:-mflrr0-stdr0,PPC_LR_STKOFF(r1)--/*TurnonTM.*/-mfmsrr8-lir0,1-rldimir8,r0,MSR_TM_LG,63-MSR_TM_LG-mtmsrdr8--ldr5,VCPU_MSR(r9)-rldicl.r5,r5,64-MSR_TS_S_LG,62-beq1f/*TMnotactiveinguest.*/--stdr1,HSTATE_HOST_R1(r13)-lir3,TM_CAUSE_KVM_RESCHED--/*CleartheMSRRIsincer1,r13areallgoingtobefoobar.*/-lir5,0-mtmsrdr5,1--/*AllGPRsarevolatileatthispoint.*/-TRECLAIM(R3)--/*Temporarilystorer13andr9sowehavesomeregstoplaywith*/-SET_SCRATCH0(r13)-GET_PACA(r13)-stdr9,PACATMSCRATCH(r13)-ldr9,HSTATE_KVM_VCPU(r13)--/*GetafewmoreGPRsfree.*/-stdr29,VCPU_GPRS_TM(29)(r9)-stdr30,VCPU_GPRS_TM(30)(r9)-stdr31,VCPU_GPRS_TM(31)(r9)--/*SaveawayPPRandDSCRsoonsodon't run with user values. */-mfsprr31,SPRN_PPR-HMT_MEDIUM-mfsprr30,SPRN_DSCR-ldr29,HSTATE_DSCR(r13)-mtsprSPRN_DSCR,r29--/*Saveallbutr9,r13&r29-r31*/-reg=0-.rept29-.if (reg!=9)&&(reg!=13)-stdreg,VCPU_GPRS_TM(reg)(r9)-.endif-reg=reg+1-.endr-/*...nowsaver13*/-GET_SCRATCH0(r4)-stdr4,VCPU_GPRS_TM(13)(r9)-/*...andsaver9*/-ldr4,PACATMSCRATCH(r13)-stdr4,VCPU_GPRS_TM(9)(r9)--/*ReloadstackpointerandTOC.*/-ldr1,HSTATE_HOST_R1(r13)-ldr2,PACATOC(r13)--/*SetMSRRInowwehaver1andr13back.*/-lir5,MSR_RI-mtmsrdr5,1--/*SaveawaycheckpintedSPRs.*/-stdr31,VCPU_PPR_TM(r9)-stdr30,VCPU_DSCR_TM(r9)-mflrr5-mfcrr6-mfctrr7-mfsprr8,SPRN_AMR-mfsprr10,SPRN_TAR-mfxerr11-stdr5,VCPU_LR_TM(r9)-stwr6,VCPU_CR_TM(r9)-stdr7,VCPU_CTR_TM(r9)-stdr8,VCPU_AMR_TM(r9)-stdr10,VCPU_TAR_TM(r9)-stdr11,VCPU_XER_TM(r9)--/*Restorer12astrapnumber.*/-lwzr12,VCPU_TRAP(r9)--/*SaveFP/VSX.*/-addir3,r9,VCPU_FPRS_TM-blstore_fp_state-addir3,r9,VCPU_VRS_TM-blstore_vr_state-mfsprr6,SPRN_VRSAVE-stwr6,VCPU_VRSAVE_TM(r9)-1:-/*-*WeneedtosavetheseSPRsafterthetreclaimsothatthesoftware-*errorcodeisrecordedcorrectlyintheTEXASR.Alsotheusermay-*changetheseoutsideofatransaction,sotheymustalwaysbe-*contextswitched.-*/-mfsprr5,SPRN_TFHAR-mfsprr6,SPRN_TFIAR-mfsprr7,SPRN_TEXASR-stdr5,VCPU_TFHAR(r9)-stdr6,VCPU_TFIAR(r9)-stdr7,VCPU_TEXASR(r9)--ldr0,PPC_LR_STKOFF(r1)-mtlrr0-blr--/*-*RestoretransactionalstateandTM-relatedregisters.-*Calledwithr4pointingtothevcpustruct.-*Thispotentiallymodifiesallcheckpointedregisters.-*Itrestoresr1,r2,r4fromthePACA.-*/-kvmppc_restore_tm:-mflrr0-stdr0,PPC_LR_STKOFF(r1)--/*TurnonTM/FP/VSX/VMXsowecanrestorethem.*/-mfmsrr5-lir6,MSR_TM>>32-sldir6,r6,32-orr5,r5,r6-orir5,r5,MSR_FP-orisr5,r5,(MSR_VEC|MSR_VSX)@h-mtmsrdr5--/*-*Theusermaychangetheseoutsideofatransaction,sotheymust-*alwaysbecontextswitched.-*/-ldr5,VCPU_TFHAR(r4)-ldr6,VCPU_TFIAR(r4)-ldr7,VCPU_TEXASR(r4)-mtsprSPRN_TFHAR,r5-mtsprSPRN_TFIAR,r6-mtsprSPRN_TEXASR,r7--ldr5,VCPU_MSR(r4)-rldicl.r5,r5,64-MSR_TS_S_LG,62-beqlr/*TMnotactiveinguest*/-stdr1,HSTATE_HOST_R1(r13)--/*Makesurethefailuresummaryisset,otherwisewe'll program check-*whenwetrechkpt.It's possible that this might have been not set-*onakvmppc_set_one_reg()callbutweshouldn't let this crash the-*host.-*/-orisr7,r7,(TEXASR_FS)@h-mtsprSPRN_TEXASR,r7--/*-*Weneedtoloadupthecheckpointedstatefortheguest.-*WeneedtodothisearlyasitwillblowawayanyGPRs,VSRsand-*someSPRs.-*/--mrr31,r4-addir3,r31,VCPU_FPRS_TM-blload_fp_state-addir3,r31,VCPU_VRS_TM-blload_vr_state-mrr4,r31-lwzr7,VCPU_VRSAVE_TM(r4)-mtsprSPRN_VRSAVE,r7--ldr5,VCPU_LR_TM(r4)-lwzr6,VCPU_CR_TM(r4)-ldr7,VCPU_CTR_TM(r4)-ldr8,VCPU_AMR_TM(r4)-ldr9,VCPU_TAR_TM(r4)-ldr10,VCPU_XER_TM(r4)-mtlrr5-mtcrr6-mtctrr7-mtsprSPRN_AMR,r8-mtsprSPRN_TAR,r9-mtxerr10--/*-*LoadupPPRandDSCRvaluesbutdon't put them in the actual SPRs-*tillthelastmomenttoavoidrunningwithuserspacePPRandDSCRfor-*toolong.-*/-ldr29,VCPU_DSCR_TM(r4)-ldr30,VCPU_PPR_TM(r4)--stdr2,PACATMSCRATCH(r13)/*SaveTOC*/--/*CleartheMSRRIsincer1,r13areallgoingtobefoobar.*/-lir5,0-mtmsrdr5,1--/*LoadGPRsr0-r28*/-reg=0-.rept29-ldreg,VCPU_GPRS_TM(reg)(r31)-reg=reg+1-.endr--mtsprSPRN_DSCR,r29-mtsprSPRN_PPR,r30--/*LoadfinalGPRs*/-ld29,VCPU_GPRS_TM(29)(r31)-ld30,VCPU_GPRS_TM(30)(r31)-ld31,VCPU_GPRS_TM(31)(r31)--/*TMcheckpointedstateisnowsetup.AllGPRsarenowvolatile.*/-TRECHKPT--/*Nowlet's get back the state we need. */-HMT_MEDIUM-GET_PACA(r13)-ldr29,HSTATE_DSCR(r13)-mtsprSPRN_DSCR,r29-ldr4,HSTATE_KVM_VCPU(r13)-ldr1,HSTATE_HOST_R1(r13)-ldr2,PACATMSCRATCH(r13)--/*SettheMSRRIsincewehaveourregistersback.*/-lir5,MSR_RI-mtmsrdr5,1--ldr0,PPC_LR_STKOFF(r1)-mtlrr0-blr-#endif-/**Wecomehereifwegetanyexceptionorinterruptwhileweare*executinghostrealmodecodewhileinguestMMUcontext.
@@ -0,0 +1,267 @@+/*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicense,version2,as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*Derivedfrombook3s_hv_rmhandlers.S,whichare:+*+*Copyright2011PaulMackerras,IBMCorp.<paulus@au1.ibm.com>+*+*/++#include <asm/reg.h>+#include <asm/ppc_asm.h>+#include <asm/asm-offsets.h>+#include <asm/export.h>+#include <asm/tm.h>+#include <asm/cputable.h>++#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+#define VCPU_GPRS_TM(reg) (((reg) * ULONG_SIZE) + VCPU_GPR_TM)+#endif++#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/*+*SavetransactionalstateandTM-relatedregisters.+*Calledwithr9pointingtothevcpustruct.+*Thiscanmodifyallcheckpointedregisters,but+*restoresr1,r2andr9 (vcpupointer)beforeexit.+*/+_GLOBAL(kvmppc_save_tm)+mflrr0+stdr0,PPC_LR_STKOFF(r1)++/*TurnonTM.*/+mfmsrr8+lir0,1+rldimir8,r0,MSR_TM_LG,63-MSR_TM_LG+mtmsrdr8++ldr5,VCPU_MSR(r9)+rldicl.r5,r5,64-MSR_TS_S_LG,62+beq1f/*TMnotactiveinguest.*/++stdr1,HSTATE_HOST_R1(r13)+lir3,TM_CAUSE_KVM_RESCHED++/*CleartheMSRRIsincer1,r13areallgoingtobefoobar.*/+lir5,0+mtmsrdr5,1++/*AllGPRsarevolatileatthispoint.*/+TRECLAIM(R3)++/*Temporarilystorer13andr9sowehavesomeregstoplaywith*/+SET_SCRATCH0(r13)+GET_PACA(r13)+stdr9,PACATMSCRATCH(r13)+ldr9,HSTATE_KVM_VCPU(r13)++/*GetafewmoreGPRsfree.*/+stdr29,VCPU_GPRS_TM(29)(r9)+stdr30,VCPU_GPRS_TM(30)(r9)+stdr31,VCPU_GPRS_TM(31)(r9)++/*SaveawayPPRandDSCRsoonsodon't run with user values. */+mfsprr31,SPRN_PPR+HMT_MEDIUM+mfsprr30,SPRN_DSCR+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE+ldr29,HSTATE_DSCR(r13)+mtsprSPRN_DSCR,r29+#endif++/*Saveallbutr9,r13&r29-r31*/+reg=0+.rept29+.if (reg!=9)&&(reg!=13)+stdreg,VCPU_GPRS_TM(reg)(r9)+.endif+reg=reg+1+.endr+/*...nowsaver13*/+GET_SCRATCH0(r4)+stdr4,VCPU_GPRS_TM(13)(r9)+/*...andsaver9*/+ldr4,PACATMSCRATCH(r13)+stdr4,VCPU_GPRS_TM(9)(r9)++/*ReloadstackpointerandTOC.*/+ldr1,HSTATE_HOST_R1(r13)+ldr2,PACATOC(r13)++/*SetMSRRInowwehaver1andr13back.*/+lir5,MSR_RI+mtmsrdr5,1++/*SaveawaycheckpintedSPRs.*/+stdr31,VCPU_PPR_TM(r9)+stdr30,VCPU_DSCR_TM(r9)+mflrr5+mfcrr6+mfctrr7+mfsprr8,SPRN_AMR+mfsprr10,SPRN_TAR+mfxerr11+stdr5,VCPU_LR_TM(r9)+stwr6,VCPU_CR_TM(r9)+stdr7,VCPU_CTR_TM(r9)+stdr8,VCPU_AMR_TM(r9)+stdr10,VCPU_TAR_TM(r9)+stdr11,VCPU_XER_TM(r9)++/*Restorer12astrapnumber.*/+lwzr12,VCPU_TRAP(r9)++/*SaveFP/VSX.*/+addir3,r9,VCPU_FPRS_TM+blstore_fp_state+addir3,r9,VCPU_VRS_TM+blstore_vr_state+mfsprr6,SPRN_VRSAVE+stwr6,VCPU_VRSAVE_TM(r9)+1:+/*+*WeneedtosavetheseSPRsafterthetreclaimsothatthesoftware+*errorcodeisrecordedcorrectlyintheTEXASR.Alsotheusermay+*changetheseoutsideofatransaction,sotheymustalwaysbe+*contextswitched.+*/+mfsprr5,SPRN_TFHAR+mfsprr6,SPRN_TFIAR+mfsprr7,SPRN_TEXASR+stdr5,VCPU_TFHAR(r9)+stdr6,VCPU_TFIAR(r9)+stdr7,VCPU_TEXASR(r9)++ldr0,PPC_LR_STKOFF(r1)+mtlrr0+blr++/*+*RestoretransactionalstateandTM-relatedregisters.+*Calledwithr4pointingtothevcpustruct.+*Thispotentiallymodifiesallcheckpointedregisters.+*Itrestoresr1,r2,r4fromthePACA.+*/+_GLOBAL(kvmppc_restore_tm)+mflrr0+stdr0,PPC_LR_STKOFF(r1)++/*TurnonTM/FP/VSX/VMXsowecanrestorethem.*/+mfmsrr5+lir6,MSR_TM>>32+sldir6,r6,32+orr5,r5,r6+orir5,r5,MSR_FP+orisr5,r5,(MSR_VEC|MSR_VSX)@h+mtmsrdr5++/*+*Theusermaychangetheseoutsideofatransaction,sotheymust+*alwaysbecontextswitched.+*/+ldr5,VCPU_TFHAR(r4)+ldr6,VCPU_TFIAR(r4)+ldr7,VCPU_TEXASR(r4)+mtsprSPRN_TFHAR,r5+mtsprSPRN_TFIAR,r6+mtsprSPRN_TEXASR,r7++ldr5,VCPU_MSR(r4)+rldicl.r5,r5,64-MSR_TS_S_LG,62+beqlr/*TMnotactiveinguest*/+stdr1,HSTATE_HOST_R1(r13)++/*Makesurethefailuresummaryisset,otherwisewe'll program check+*whenwetrechkpt.It's possible that this might have been not set+*onakvmppc_set_one_reg()callbutweshouldn't let this crash the+*host.+*/+orisr7,r7,(TEXASR_FS)@h+mtsprSPRN_TEXASR,r7++/*+*Weneedtoloadupthecheckpointedstatefortheguest.+*WeneedtodothisearlyasitwillblowawayanyGPRs,VSRsand+*someSPRs.+*/++mrr31,r4+addir3,r31,VCPU_FPRS_TM+blload_fp_state+addir3,r31,VCPU_VRS_TM+blload_vr_state+mrr4,r31+lwzr7,VCPU_VRSAVE_TM(r4)+mtsprSPRN_VRSAVE,r7++ldr5,VCPU_LR_TM(r4)+lwzr6,VCPU_CR_TM(r4)+ldr7,VCPU_CTR_TM(r4)+ldr8,VCPU_AMR_TM(r4)+ldr9,VCPU_TAR_TM(r4)+ldr10,VCPU_XER_TM(r4)+mtlrr5+mtcrr6+mtctrr7+mtsprSPRN_AMR,r8+mtsprSPRN_TAR,r9+mtxerr10++/*+*LoadupPPRandDSCRvaluesbutdon't put them in the actual SPRs+*tillthelastmomenttoavoidrunningwithuserspacePPRandDSCRfor+*toolong.+*/+ldr29,VCPU_DSCR_TM(r4)+ldr30,VCPU_PPR_TM(r4)++stdr2,PACATMSCRATCH(r13)/*SaveTOC*/++/*CleartheMSRRIsincer1,r13areallgoingtobefoobar.*/+lir5,0+mtmsrdr5,1++/*LoadGPRsr0-r28*/+reg=0+.rept29+ldreg,VCPU_GPRS_TM(reg)(r31)+reg=reg+1+.endr++mtsprSPRN_DSCR,r29+mtsprSPRN_PPR,r30++/*LoadfinalGPRs*/+ld29,VCPU_GPRS_TM(29)(r31)+ld30,VCPU_GPRS_TM(30)(r31)+ld31,VCPU_GPRS_TM(31)(r31)++/*TMcheckpointedstateisnowsetup.AllGPRsarenowvolatile.*/+TRECHKPT++/*Nowlet's get back the state we need. */+HMT_MEDIUM+GET_PACA(r13)+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE+ldr29,HSTATE_DSCR(r13)+mtsprSPRN_DSCR,r29+#endif+ldr4,HSTATE_KVM_VCPU(r13)+ldr1,HSTATE_HOST_R1(r13)+ldr2,PACATMSCRATCH(r13)++/*SettheMSRRIsincewehaveourregistersback.*/+lir5,MSR_RI+mtmsrdr5,1++ldr0,PPC_LR_STKOFF(r1)+mtlrr0+blr+#endif
From: Simon Guo <redacted>
HV KVM and PR KVM need different MSR source to indicate whether
treclaim. or trecheckpoint. is necessary.
This patch add new parameter (guest MSR) for these kvmppc_save_tm/
kvmppc_restore_tm() APIs:
- For HV KVM, it is VCPU_MSR
- For PR KVM, it is current host MSR or VCPU_SHADOW_SRR1
This enhancement enables these 2 APIs to be reused by PR KVM later.
And the patch keeps HV KVM logic unchanged.
This patch also reworks kvmppc_save_tm()/kvmppc_restore_tm() to
have a clean ABI: r3 for vcpu and r4 for guest_msr.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 12 ++++++-
arch/powerpc/kvm/tm.S | 61 ++++++++++++++++++---------------
2 files changed, 45 insertions(+), 28 deletions(-)
@@ -166,17 +173,17 @@ _GLOBAL(kvmppc_restore_tm)*Theusermaychangetheseoutsideofatransaction,sotheymust*alwaysbecontextswitched.*/-ldr5,VCPU_TFHAR(r4)-ldr6,VCPU_TFIAR(r4)-ldr7,VCPU_TEXASR(r4)+ldr5,VCPU_TFHAR(r3)+ldr6,VCPU_TFIAR(r3)+ldr7,VCPU_TEXASR(r3)mtsprSPRN_TFHAR,r5mtsprSPRN_TFIAR,r6mtsprSPRN_TEXASR,r7-ldr5,VCPU_MSR(r4)+mrr5,r4rldicl.r5,r5,64-MSR_TS_S_LG,62beqlr/*TMnotactiveinguest*/-stdr1,HSTATE_HOST_R1(r13)+stdr1,HSTATE_SCRATCH2(r13)/*Makesurethefailuresummaryisset,otherwisewe'll program check*whenwetrechkpt.It's possible that this might have been not set
From: Simon Guo <redacted>
kvmppc_save_tm() invokes store_fp_state/store_vr_state(). So it is
mandatory to turn on FP/VSX/VMX MSR bits for its execution, just
like what kvmppc_restore_tm() did.
Previsouly HV KVM has turned the bits on outside of function
kvmppc_save_tm(). Now we include this bit change in kvmppc_save_tm()
so that the logic is more clean. And PR KVM can reuse it later.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/tm.S | 2 ++
1 file changed, 2 insertions(+)
From: Simon Guo <redacted>
Currently _kvmppc_save/restore_tm() APIs can only be invoked from
assembly function. This patch adds C function wrappers for them so
that they can be safely called from C function.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/asm-prototypes.h | 7 ++
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 8 +--
arch/powerpc/kvm/tm.S | 107 +++++++++++++++++++++++++++++-
3 files changed, 116 insertions(+), 6 deletions(-)
@@ -186,6 +238,7 @@ _GLOBAL(kvmppc_restore_tm)rldicl.r5,r5,64-MSR_TS_S_LG,62beqlr/*TMnotactiveinguest*/stdr1,HSTATE_SCRATCH2(r13)+stdr3,HSTATE_SCRATCH1(r13)/*Makesurethefailuresummaryisset,otherwisewe'll program check*whenwetrechkpt.It's possible that this might have been not set
From: Simon Guo <redacted>
This patch simulates interrupt behavior per Power ISA while injecting
interrupt in PR KVM:
- When interrupt happens, transactional state should be suspended.
kvmppc_mmu_book3s_64_reset_msr() will be invoked when injecting an
interrupt. This patch performs this ISA logic in
kvmppc_mmu_book3s_64_reset_msr().
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_64_mmu.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
@@ -38,7 +38,16 @@staticvoidkvmppc_mmu_book3s_64_reset_msr(structkvm_vcpu*vcpu){-kvmppc_set_msr(vcpu,vcpu->arch.intr_msr);+unsignedlongmsr=vcpu->arch.intr_msr;+unsignedlongcur_msr=kvmppc_get_msr(vcpu);++/* If transactional, change to suspend mode on IRQ delivery */+if(MSR_TM_TRANSACTIONAL(cur_msr))+msr|=MSR_TS_S;+else+msr|=cur_msr&MSR_TS_MASK;++kvmppc_set_msr(vcpu,msr);}staticstructkvmppc_slb*kvmppc_mmu_book3s_64_find_slbe(
From: Simon Guo <redacted>
PowerPC TM functionality needs MSR TM/TS bits support in hardware level.
Guest TM functionality can not be emulated with "fake" MSR (msr in magic
page) TS bits.
This patch syncs TM/TS bits in shadow_msr with the MSR value in magic
page, so that the MSR TS value which guest sees is consistent with actual
MSR bits running in guest.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_pr.c | 5 +++++
1 file changed, 5 insertions(+)
From: Simon Guo <redacted>
This patches add some macros for CR0/TEXASR bits so that PR KVM TM
logic(tbegin./treclaim./tabort.) can make use of them later.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/reg.h | 21 ++++++++++++++++++++-
arch/powerpc/platforms/powernv/copy-paste.h | 3 +--
2 files changed, 21 insertions(+), 3 deletions(-)
From: Simon Guo <redacted>
MSR TS bits can be modified with non-privileged instruction like
tbegin./tend. That means guest can change MSR value "silently" without
notifying host.
It is necessary to sync the TM bits to host so that host can calculate
shadow msr correctly.
note privilege guest will always fail transactions so we only take
care of problem state guest.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_pr.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
From: Simon Guo <redacted>
Accordingly to ISA specification for RFID, in MSR TM disabled and TS
suspended state(S0), if the target MSR is TM disabled and TS state is
inactive(N0), rfid should suppress this update.
This patch make RFID emulation of PR KVM to be consistent with this.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
From: Simon Guo <redacted>
Apple G5 machines(PPC970/FX/GX/MP) have supervisor mode disabled and
MSR HV bit is forced into 1. We should follow this in PR KVM guest.
This patch set MSR HV=1 for G5 machines and HV=0 for others on PR
KVM guest.
Signed-off-by: Simon Guo <redacted>
Suggested-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_pr.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
@@ -334,7 +335,16 @@ static void kvmppc_recalc_shadow_msr(struct kvm_vcpu *vcpu)smsr|=(guest_msr&vcpu->arch.guest_owned_ext);/* 64-bit Process MSR values */#ifdef CONFIG_PPC_BOOK3S_64-smsr|=MSR_ISF|MSR_HV;+smsr|=MSR_ISF;++/* for PPC970 chip, its HV bit is hard-wired to 1. For others,+*weshouldclearHVbit.+*/+if((PVR_VER(guest_pvr)==PVR_970)||+(PVR_VER(guest_pvr)==PVR_970FX)||+(PVR_VER(guest_pvr)==PVR_970MP)||+(PVR_VER(guest_pvr)==PVR_970GX))+smsr|=MSR_HV;#endifvcpu->arch.shadow_msr=smsr;}
From: Simon Guo <redacted>
PR KVM host usually equipped with enabled TM in its host MSR value, and
with non-transactional TS value.
When a guest with TM active traps into PR KVM host, the rfid at the
tail of kvmppc_interrupt_pr() will try to switch TS bits from
S0 (Suspended & TM disabled) to N1 (Non-transactional & TM enabled).
That will leads to TM Bad Thing interrupt.
This patch manually sets target TS bits unchanged to avoid this
exception.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_segment.S | 13 +++++++++++++
1 file changed, 13 insertions(+)
From: Simon Guo <redacted>
PR KVM will need to reuse msr_check_and_set().
This patch exports this API for reuse.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kernel/process.c | 1 +
1 file changed, 1 insertion(+)
@@ -152,6 +152,7 @@ unsigned long msr_check_and_set(unsigned long bits)returnnewmsr;}+EXPORT_SYMBOL_GPL(msr_check_and_set);void__msr_check_and_clear(unsignedlongbits){
From: Simon Guo <redacted>
This patch adds 2 new APIs: kvmppc_copyto_vcpu_tm() and
kvmppc_copyfrom_vcpu_tm(). These 2 APIs will be used to copy from/to TM
data between VCPU_TM/VCPU area.
PR KVM will use these APIs for treclaim. or trchkpt. emulation.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
From: Simon Guo <redacted>
This patch adds 2 new APIs kvmppc_save_tm_sprs()/kvmppc_restore_tm_sprs()
for the purpose of TEXASR/TFIAR/TFHAR save/restore.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
---
arch/powerpc/kvm/book3s_pr.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
@@ -235,6 +236,27 @@ void kvmppc_copy_from_svcpu(struct kvm_vcpu *vcpu,preempt_enable();}+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+staticinlinevoidkvmppc_save_tm_sprs(structkvm_vcpu*vcpu)+{+tm_enable();+vcpu->arch.tfhar=mfspr(SPRN_TFHAR);+vcpu->arch.texasr=mfspr(SPRN_TEXASR);+vcpu->arch.tfiar=mfspr(SPRN_TFIAR);+tm_disable();+}++staticinlinevoidkvmppc_restore_tm_sprs(structkvm_vcpu*vcpu)+{+tm_enable();+mtspr(SPRN_TFHAR,vcpu->arch.tfhar);+mtspr(SPRN_TEXASR,vcpu->arch.texasr);+mtspr(SPRN_TFIAR,vcpu->arch.tfiar);+tm_disable();+}++#endif+staticintkvmppc_core_check_requests_pr(structkvm_vcpu*vcpu){intr=1;/* Indicate we want to get back into the guest */
From: Simon Guo <redacted>
The transaction memory checkpoint area save/restore behavior is
triggered when VCPU qemu process is switching out/into CPU. ie.
at kvmppc_core_vcpu_put_pr() and kvmppc_core_vcpu_load_pr().
MSR TM active state is determined by TS bits:
active: 10(transactional) or 01 (suspended)
inactive: 00 (non-transactional)
We don't "fake" TM functionality for guest. We "sync" guest virtual
MSR TM active state(10 or 01) with shadow MSR. That is to say,
we don't emulate a transactional guest with a TM inactive MSR.
TM SPR support(TFIAR/TFAR/TEXASR) has already been supported by
commit 9916d57e64a4 ("KVM: PPC: Book3S PR: Expose TM registers").
Math register support (FPR/VMX/VSX) will be done at subsequent
patch.
- TM save:
When kvmppc_save_tm_pr() is invoked, whether TM context need to
be saved can be determined by current host MSR state:
* TM active - save TM context
* TM inactive - no need to do so and only save TM SPRs.
- TM restore:
However when kvmppc_restore_tm_pr() is invoked, there is an
issue to determine whether TM restore should be performed.
The TM active host MSR val saved in kernel stack is not loaded yet.
We don't know whether there is a transaction to be restored from
current host MSR TM status at kvmppc_restore_tm_pr(). To solve this
issue, we save current MSR into vcpu->arch.save_msr_tm at
kvmppc_save_tm_pr(), and kvmppc_restore_tm_pr() check TS bits of
vcpu->arch.save_msr_tm to decide whether to do TM restore.
Signed-off-by: Simon Guo <redacted>
Suggested-by: Paul Mackerras <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 6 +++++
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/kvm/book3s_pr.c | 41 +++++++++++++++++++++++++++++++++++
3 files changed, 48 insertions(+)
From: Simon Guo <redacted>
The math registers will be saved into vcpu->arch.fp/vr and corresponding
vcpu->arch.fp_tm/vr_tm area.
We flush or giveup the math regs into vcpu->arch.fp/vr before saving
transaction. After transaction is restored, the math regs will be loaded
back into regs.
If there is a FP/VEC/VSX unavailable exception during transaction active
state, the math checkpoint content might be incorrect and we need to do
treclaim./load the correct checkpoint val/trechkpt. sequence to retry the
transaction.
If transaction is active, and the qemu process is switching out of CPU,
we need to keep the "guest_owned_ext" bits unchanged after qemu process
is switched back. The reason is that if we allow guest_owned_ext change
freely during a transaction, there will lack information to handle
FP/VEC/VSX unavailable exception during transaction active state.
Detail is as follows:
Assume we allow math bits to be given up freely during transaction:
- If it is the first FP unavailable exception after tbegin., vcpu->arch.fp/
vr need to be loaded for trechkpt.
- If it is the 2nd or subsequent FP unavailable exception after tbegin.,
vcpu->arch.fp_tm/vr_tm need to be loaded for trechkpt.
It will bring much additional complexity to cover both cases.
That is why we always save guest_owned_ext into vcpu->arch.save_msr_tm at
kvmppc_save_tm_pr(), then check those bits in vcpu->arch.save_msr_tm at
kvmppc_restore_tm_pr() to determine what math contents will be loaded.
With this, we will always load vcpu->arch.fp/vr in math unavailable
exception during active transaction.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_host.h | 4 +-
arch/powerpc/kvm/book3s_pr.c | 114 +++++++++++++++++++++++++++++-------
2 files changed, 95 insertions(+), 23 deletions(-)
@@ -55,6 +55,7 @@staticintkvmppc_handle_ext(structkvm_vcpu*vcpu,unsignedintexit_nr,ulongmsr);+staticintkvmppc_load_ext(structkvm_vcpu*vcpu,ulongmsr);staticvoidkvmppc_giveup_fac(structkvm_vcpu*vcpu,ulongfac);/* Some compatibility defines */
@@ -280,6 +281,33 @@ void kvmppc_save_tm_pr(struct kvm_vcpu *vcpu)return;}+/* when we are in transaction active state and switch out of CPU,+*weneedtobecarefultonot"change"guest_owned_extbitsafter+*kvmppc_save_tm_pr()/kvmppc_restore_tm_pr()pair.Thereasonis+*thatweneedtodistinguishfollowing2FP/VEC/VSXunavailable+*exceptioncasesinTMactivestate:+*1)tbegin.isexecutedwithguest_owned_extFP/VEC/VSXoff.Then+*therecomesaFP/VEC/VSXunavailableexceptionduringtransaction.+*Inthiscase,thevcpu->arch.fp/vrcontentsneedtobeloadedas+*checkpointcontents.+*2)tbegin.isexecutedwithguest_owned_extFP/VEC/VSXon.Then+*thereistaskswitchduringsuspendedstate.Ifwegiveupextand+*updateguest_owned_extasnoFP/VEC/VSXbitsduringcontextswitch,+*weneedtoloadvcpu->arch.fp_tm/vr_tmcontentsascheckpoint+*content.+*+*Asaresult,wedon'tchangeguest_owned_extbitsduring+*kvmppc_save/restore_tm_pr()pair.Sothatwecanonlyuse+*vcpu->arch.fp/vrcontentsascheckpointcontents.+*Andweneedto"save"theguest_owned_extbitsherewhoindicates+*whichmathbitsneedtobe"restored"inkvmppc_restore_tm_pr().+*/+vcpu->arch.save_msr_tm&=~(MSR_FP|MSR_VEC|MSR_VSX);+vcpu->arch.save_msr_tm|=(vcpu->arch.guest_owned_ext&+(MSR_FP|MSR_VEC|MSR_VSX));++kvmppc_giveup_ext(vcpu,MSR_VSX);+preempt_disable();_kvmppc_save_tm_pr(vcpu,mfmsr());preempt_enable();
@@ -788,12 +826,41 @@ static void kvmppc_giveup_fac(struct kvm_vcpu *vcpu, ulong fac)#endif}+staticintkvmppc_load_ext(structkvm_vcpu*vcpu,ulongmsr)+{+structthread_struct*t=¤t->thread;++if(msr&MSR_FP){+preempt_disable();+enable_kernel_fp();+load_fp_state(&vcpu->arch.fp);+disable_kernel_fp();+t->fp_save_area=&vcpu->arch.fp;+preempt_enable();+}++if(msr&MSR_VEC){+#ifdef CONFIG_ALTIVEC+preempt_disable();+enable_kernel_altivec();+load_vr_state(&vcpu->arch.vr);+disable_kernel_altivec();+t->vr_save_area=&vcpu->arch.vr;+preempt_enable();+#endif+}++t->regs->msr|=msr;+vcpu->arch.guest_owned_ext|=msr;+kvmppc_recalc_shadow_msr(vcpu);++returnRESUME_GUEST;+}+/* Handle external providers (FPU, Altivec, VSX) */staticintkvmppc_handle_ext(structkvm_vcpu*vcpu,unsignedintexit_nr,ulongmsr){-structthread_struct*t=¤t->thread;-/* When we have paired singles, we emulate in software */if(vcpu->arch.hflags&BOOK3S_HFLAG_PAIRED_SINGLE)returnRESUME_GUEST;
@@ -829,31 +896,34 @@ static int kvmppc_handle_ext(struct kvm_vcpu *vcpu, unsigned int exit_nr,printk(KERN_INFO"Loading up ext 0x%lx\n",msr);#endif-if(msr&MSR_FP){+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/* if TM is active, the checkpointed math content+*mightbeinvalid.Weneedtoreclaimcurrent+*transaction,loadthecorrectmath,andperform+*rechkpoint.+*/+if(MSR_TM_ACTIVE(mfmsr())){preempt_disable();-enable_kernel_fp();-load_fp_state(&vcpu->arch.fp);-disable_kernel_fp();-t->fp_save_area=&vcpu->arch.fp;-preempt_enable();-}+kvmppc_save_tm_pr(vcpu);+/* need update the chkpt math reg saving content,+*sothatwecancheckpointwithdesiredfpvalue.+*/+if(msr&MSR_FP)+memcpy(&vcpu->arch.fp_tm,&vcpu->arch.fp,+sizeof(structthread_fp_state));++if(msr&MSR_VEC){+memcpy(&vcpu->arch.vr_tm,&vcpu->arch.vr,+sizeof(structthread_vr_state));+vcpu->arch.vrsave_tm=vcpu->arch.vrsave;+}-if(msr&MSR_VEC){-#ifdef CONFIG_ALTIVEC-preempt_disable();-enable_kernel_altivec();-load_vr_state(&vcpu->arch.vr);-disable_kernel_altivec();-t->vr_save_area=&vcpu->arch.vr;+kvmppc_restore_tm_pr(vcpu);preempt_enable();-#endif}+#endif-t->regs->msr|=msr;-vcpu->arch.guest_owned_ext|=msr;-kvmppc_recalc_shadow_msr(vcpu);--returnRESUME_GUEST;+returnkvmppc_load_ext(vcpu,msr);}/*
From: Simon Guo <redacted>
The mfspr/mtspr on TM SPRs(TEXASR/TFIAR/TFHAR) are non-privileged
instructions and can be executed at PR KVM guest without trapping
into host in problem state. We only emulate mtspr/mfspr
texasr/tfiar/tfhar at guest PR=0 state.
When we are emulating mtspr tm sprs at guest PR=0 state, the emulation
result need to be visible to guest PR=1 state. That is, the actual TM
SPR val should be loaded into actual registers.
We already flush TM SPRs into vcpu when switching out of CPU, and load
TM SPRs when switching back.
This patch corrects mfspr()/mtspr() emulation for TM SPRs to make the
actual source/dest based on actual TM SPRs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 35 +++++++++++++++++++++++++++--------
1 file changed, 27 insertions(+), 8 deletions(-)
@@ -521,13 +521,26 @@ int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)break;#ifdef CONFIG_PPC_TRANSACTIONAL_MEMcaseSPRN_TFHAR:-vcpu->arch.tfhar=spr_val;-break;caseSPRN_TEXASR:-vcpu->arch.texasr=spr_val;-break;caseSPRN_TFIAR:-vcpu->arch.tfiar=spr_val;+if(MSR_TM_ACTIVE(kvmppc_get_msr(vcpu))){+/* it is illegal to mtspr() TM regs in+*otherthannon-transactionalstate.+*/+kvmppc_core_queue_program(vcpu,SRR1_PROGTM);+emulated=EMULATE_AGAIN;+break;+}++tm_enable();+if(sprn==SPRN_TFHAR)+mtspr(SPRN_TFHAR,spr_val);+elseif(sprn==SPRN_TEXASR)+mtspr(SPRN_TEXASR,spr_val);+else+mtspr(SPRN_TFIAR,spr_val);+tm_disable();+break;#endif#endif
@@ -674,13 +687,19 @@ int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_valbreak;#ifdef CONFIG_PPC_TRANSACTIONAL_MEMcaseSPRN_TFHAR:-*spr_val=vcpu->arch.tfhar;+tm_enable();+*spr_val=mfspr(SPRN_TFHAR);+tm_disable();break;caseSPRN_TEXASR:-*spr_val=vcpu->arch.texasr;+tm_enable();+*spr_val=mfspr(SPRN_TEXASR);+tm_disable();break;caseSPRN_TFIAR:-*spr_val=vcpu->arch.tfiar;+tm_enable();+*spr_val=mfspr(SPRN_TFIAR);+tm_disable();break;#endif#endif
From: Simon Guo <redacted>
Currently kernel doesn't use transaction memory.
And there is an issue for privilege guest that:
tbegin/tsuspend/tresume/tabort TM instructions can impact MSR TM bits
without trap into PR host. So following code will lead to a false mfmsr
result:
tbegin <- MSR bits update to Transaction active.
beq <- failover handler branch
mfmsr <- still read MSR bits from magic page with
transaction inactive.
It is not an issue for non-privilege guest since its mfmsr is not patched
with magic page and will always trap into PR host.
This patch will always fail tbegin attempt for privilege guest, so that
the above issue is prevented. It is benign since currently (guest) kernel
doesn't initiate a transaction.
Test case:
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.c
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 34 ++++++++++++++++++++++++++++++++++
arch/powerpc/kvm/book3s_pr.c | 11 ++++++++++-
3 files changed, 45 insertions(+), 1 deletion(-)
@@ -47,6 +48,8 @@#define OP_31_XOP_EIOIO 854#define OP_31_XOP_SLBMFEE 915+#define OP_31_XOP_TBEGIN 654+/* DCBZ is actually 1014, but we patch it to 1010 so we get a trap */#define OP_31_XOP_DCBZ 1010
From: Simon Guo <redacted>
Currently kvmppc_handle_fac() will not update NV GPRs and thus it can
return with GUEST_RESUME.
However PR KVM guest always disables MSR_TM bit at privilege state. If PR
privilege guest are trying to read TM SPRs, it will trigger TM facility
unavailable exception and fall into kvmppc_handle_fac(). Then the emulation
will be done by kvmppc_core_emulate_mfspr_pr(). The mfspr instruction can
include a RT with NV reg. So it is necessary to restore NV GPRs at this
case, to reflect the update to NV RT.
This patch make kvmppc_handle_fac() return GUEST_RESUME_NV at TM fac
exception and with guest privilege state.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_pr.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
@@ -1030,6 +1030,18 @@ static int kvmppc_handle_fac(struct kvm_vcpu *vcpu, ulong fac)break;}+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/* Since we disabled MSR_TM at privilege state, the mfspr instruction+*forTMsprcantriggerTMfacunavailable.Inthiscase,the+*emulationishandledbykvmppc_emulate_fac(),whichinvokes+*kvmppc_emulate_mfspr()finally.Butnotethemfsprcaninclude+*RTforNVregisters.SoitneedtorestorethoseNVregtoreflect+*theupdate.+*/+if((fac==FSCR_TM_LG)&&!(kvmppc_get_msr(vcpu)&MSR_PR))+returnRESUME_GUEST_NV;+#endif+returnRESUME_GUEST;}
From: Simon Guo <redacted>
This patch adds support for "treclaim." emulation when PR KVM guest
executes treclaim. and traps to host.
We will firstly doing treclaim. and save TM checkpoint and doing
treclaim. Then it is necessary to update vcpu current reg content
with checkpointed vals. When rfid into guest again, those vcpu
current reg content(now the checkpoint vals) will be loaded into
regs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/reg.h | 4 +++
arch/powerpc/kvm/book3s_emulate.c | 66 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 69 insertions(+), 1 deletion(-)
@@ -50,6 +51,8 @@#define OP_31_XOP_TBEGIN 654+#define OP_31_XOP_TRECLAIM 942+/* DCBZ is actually 1014, but we patch it to 1010 so we get a trap */#define OP_31_XOP_DCBZ 1010
@@ -393,6 +432,31 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,emulated=EMULATE_FAIL;break;}+caseOP_31_XOP_TRECLAIM:+{+ulongguest_msr=kvmppc_get_msr(vcpu);+unsignedlongra_val=0;++/* generate interrupt based on priorities */+if(guest_msr&MSR_PR){+/* Privileged Instruction type Program Interrupt */+kvmppc_core_queue_program(vcpu,SRR1_PROGPRIV);+emulated=EMULATE_AGAIN;+break;+}++if(!MSR_TM_SUSPENDED(guest_msr)){+/* TM bad thing interrupt */+kvmppc_core_queue_program(vcpu,SRR1_PROGTM);+emulated=EMULATE_AGAIN;+break;+}++if(ra)+ra_val=kvmppc_get_gpr(vcpu,ra);+kvmppc_emulate_treclaim(vcpu,ra_val);+break;+}#endifdefault:emulated=EMULATE_FAIL;
From: Simon Guo <redacted>
This patch adds host emulation when guest PR KVM executes "trechkpt.",
which is a privileged instruction and will trap into host.
We firstly copy vcpu ongoing content into vcpu tm checkpoint
content, then perform kvmppc_restore_tm_pr() to do trechkpt.
with updated vcpu tm checkpoint vals.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 57 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 56 insertions(+), 1 deletion(-)
@@ -52,6 +52,7 @@#define OP_31_XOP_TBEGIN 654#define OP_31_XOP_TRECLAIM 942+#define OP_31_XOP_TRCHKPT 1006/* DCBZ is actually 1014, but we patch it to 1010 so we get a trap */#define OP_31_XOP_DCBZ 1010
@@ -457,6 +484,34 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,kvmppc_emulate_treclaim(vcpu,ra_val);break;}+caseOP_31_XOP_TRCHKPT:+{+ulongguest_msr=kvmppc_get_msr(vcpu);+unsignedlongtexasr;++/* generate interrupt based on priorities */+if(guest_msr&MSR_PR){+/* Privileged Instruction type Program Intr */+kvmppc_core_queue_program(vcpu,SRR1_PROGPRIV);+emulated=EMULATE_AGAIN;+break;+}++tm_enable();+texasr=mfspr(SPRN_TEXASR);+tm_disable();++if(MSR_TM_ACTIVE(guest_msr)||+!(texasr&(TEXASR_FS))){+/* TM bad thing interrupt */+kvmppc_core_queue_program(vcpu,SRR1_PROGTM);+emulated=EMULATE_AGAIN;+break;+}++kvmppc_emulate_trchkpt(vcpu);+break;+}#endifdefault:emulated=EMULATE_FAIL;
From: Simon Guo <redacted>
Currently privilege guest will be run with TM disabled.
Although the privilege guest cannot initiate a new transaction,
it can use tabort to terminate its problem state's transaction.
So it is still necessary to emulate tabort. for privilege guest.
This patch adds emulation for tabort. of privilege guest.
Tested with:
https://github.com/justdoitqd/publicFiles/blob/master/test_tabort.c
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 31 +++++++++++++++++++++++++++++++
arch/powerpc/kvm/book3s_pr.c | 2 +-
3 files changed, 33 insertions(+), 1 deletion(-)
@@ -193,6 +194,19 @@ static void kvmppc_emulate_trchkpt(struct kvm_vcpu *vcpu)guest_msr|=MSR_TS_S;kvmppc_set_msr(vcpu,guest_msr);}++/* emulate tabort. at guest privilege state */+staticvoidkvmppc_emulate_tabort(structkvm_vcpu*vcpu,intra_val)+{+/* currently we only emulate tabort. but no emulation of other+*tabortvariantssincethereisnokernelusageofthemat+*present.+*/+tm_enable();+tm_abort(ra_val);+tm_disable();+}+#endifintkvmppc_core_emulate_op_pr(structkvm_run*run,structkvm_vcpu*vcpu,
@@ -459,6 +473,23 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,emulated=EMULATE_FAIL;break;}+caseOP_31_XOP_TABORT:+{+ulongguest_msr=kvmppc_get_msr(vcpu);+unsignedlongra_val=0;++/* only emulate for privilege guest, since problem state+*guestcanrunwithTMenabledandwedon'texpectto+*trapathereforthatcase.+*/+WARN_ON(guest_msr&MSR_PR);++if(ra)+ra_val=kvmppc_get_gpr(vcpu,ra);++kvmppc_emulate_tabort(vcpu,ra_val);+break;+}caseOP_31_XOP_TRECLAIM:{ulongguest_msr=kvmppc_get_msr(vcpu);
From: Simon Guo <redacted>
Currently PR KVM doesn't support transaction memory at guest privilege
state.
This patch adds a check at setting guest msr, so that we can never return
to guest with PR=0 and TS=0b10. A tabort will be emulated to indicate
this and fail transaction immediately.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/uapi/asm/tm.h | 2 +-
arch/powerpc/kvm/book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 2 +-
arch/powerpc/kvm/book3s_pr.c | 13 ++++++++++++-
4 files changed, 15 insertions(+), 3 deletions(-)
@@ -196,7 +196,7 @@ static void kvmppc_emulate_trchkpt(struct kvm_vcpu *vcpu)}/* emulate tabort. at guest privilege state */-staticvoidkvmppc_emulate_tabort(structkvm_vcpu*vcpu,intra_val)+voidkvmppc_emulate_tabort(structkvm_vcpu*vcpu,intra_val){/* currently we only emulate tabort. but no emulation of other*tabortvariantssincethereisnokernelusageofthemat
@@ -461,12 +461,23 @@ static void kvmppc_recalc_shadow_msr(struct kvm_vcpu *vcpu)staticvoidkvmppc_set_msr_pr(structkvm_vcpu*vcpu,u64msr){-ulongold_msr=kvmppc_get_msr(vcpu);+ulongold_msr;#ifdef EXIT_DEBUGprintk(KERN_INFO"KVM: Set MSR to 0x%llx\n",msr);#endif+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/* We should never target guest MSR to TS=10 && PR=0,+*sincewealwaysfailtransactionforguestprivilege+*state.+*/+if(!(msr&MSR_PR)&&MSR_TM_TRANSACTIONAL(msr))+kvmppc_emulate_tabort(vcpu,+TM_CAUSE_PRIV_T|TM_CAUSE_PERSISTENT);+#endif++old_msr=kvmppc_get_msr(vcpu);msr&=to_book3s(vcpu)->msr_mask;kvmppc_set_msr_fast(vcpu,msr);kvmppc_recalc_shadow_msr(vcpu);
From: Simon Guo <redacted>
Currently guest kernel doesn't handle TAR fac unavailable and it always
runs with TAR bit on. PR KVM will lazily enable TAR. TAR is not a
frequent-use reg and it is not included in SVCPU struct.
To make it work for transaction memory at PR KVM:
1). Flush/giveup TAR at kvmppc_save_tm_pr().
2) If we are receiving a TAR fac unavail exception inside a transaction,
the checkpointed TAR might be a TAR value from another process. So we need
treclaim the transaction, then load the desired TAR value into reg, and
perform trecheckpoint.
3) Load TAR facility at kvmppc_restore_tm_pr() when TM active.
The reason we always loads TAR when restoring TM is that:
If we don't do this way, when there is a TAR fac unavailable exception
during TM active:
case 1: it is the 1st TAR fac unavail exception after tbegin.
vcpu->arch.tar should be reloaded as checkpoint tar val.
case 2: it is the 2nd or later TAR fac unavail exception after tbegin.
vcpu->arch.tar_tm should be reloaded as checkpoint tar val.
There will be unnecessary difficulty to handle the above 2 cases.
at the end of emulating treclaim., the correct TAR val need to be loaded
into reg if FSCR_TAR bit is on.
at the beginning of emulating trechkpt., TAR needs to be flushed so that
the right tar val can be copy into tar_tm.
Tested with:
tools/testing/selftests/powerpc/tm/tm-tar
tools/testing/selftests/powerpc/ptrace/ptrace-tm-tar (remove DSCR/PPR
related testing).
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 4 ++++
arch/powerpc/kvm/book3s_pr.c | 31 +++++++++++++++++++++++++++++--
arch/powerpc/kvm/tm.S | 16 ++++++++++++++--
4 files changed, 48 insertions(+), 4 deletions(-)
@@ -320,8 +320,20 @@ void kvmppc_restore_tm_pr(struct kvm_vcpu *vcpu)return;}+preempt_disable();_kvmppc_restore_tm_pr(vcpu,vcpu->arch.save_msr_tm);++if(!(vcpu->arch.shadow_fscr&FSCR_TAR)){+/* always restore TAR in TM active state, since we don't+*wanttobeconfusedatfacunavailablewhileTMactive:+*loadvcpu->arch.tarorvcpu->arch.tar_tmaschkptvalue?+*/+current->thread.tar=mfspr(SPRN_TAR);+mtspr(SPRN_TAR,vcpu->arch.tar);+vcpu->arch.shadow_fscr|=FSCR_TAR;+}+preempt_enable();if(vcpu->arch.save_msr_tm&MSR_VSX)
@@ -828,7 +841,7 @@ void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr)}/* Give up facility (TAR / EBB / DSCR) */-staticvoidkvmppc_giveup_fac(structkvm_vcpu*vcpu,ulongfac)+voidkvmppc_giveup_fac(structkvm_vcpu*vcpu,ulongfac){#ifdef CONFIG_PPC_BOOK3S_64if(!(vcpu->arch.shadow_fscr&(1ULL<<fac))){
@@ -1031,6 +1044,20 @@ static int kvmppc_handle_fac(struct kvm_vcpu *vcpu, ulong fac)switch(fac){caseFSCR_TAR_LG:+if(MSR_TM_ACTIVE(mfmsr())){+/* When tbegin. was executed, the TAR in checkpoint+*statemightbeinvalid.Weneedtreclaim.,then+*loadcorrectTARvalue,andperformtrechkpt.,+*sothatvalidTARvalcanbecheckpointed.+*/+preempt_disable();+kvmppc_save_tm_pr(vcpu);++vcpu->arch.tar_tm=vcpu->arch.tar;++kvmppc_restore_tm_pr(vcpu);+preempt_enable();+}/* TAR switching isn't lazy in Linux yet */current->thread.tar=mfspr(SPRN_TAR);mtspr(SPRN_TAR,vcpu->arch.tar);
From: Simon Guo <redacted>
With current patch set, PR KVM now supports HTM. So this patch turns it
on for PR KVM.
Tested with:
https://github.com/justdoitqd/publicFiles/blob/master/test_kvm_htm_cap.c
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/powerpc.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
@@ -643,8 +643,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)break;#endifcaseKVM_CAP_PPC_HTM:-r=hv_enabled&&-(cur_cpu_spec->cpu_user_features2&PPC_FEATURE2_HTM_COMP);+r=(cur_cpu_spec->cpu_user_features2&PPC_FEATURE2_HTM_COMP);break;default:r=0;
Hi Simon,
On 01/11/2018 08:11 AM, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Is this correct to assume that this emulation mode will just kick in on P9
with kernel TM workarounds and HV KVM will continue to be used on POWER8
since HV KVM is supported on POWER8 hosts?
Regards,
Gustavo
Test cases performed:
linux/tools/testing/selftests/powerpc/tm/tm-syscall
linux/tools/testing/selftests/powerpc/tm/tm-fork
linux/tools/testing/selftests/powerpc/tm/tm-vmx-unavail
linux/tools/testing/selftests/powerpc/tm/tm-tmspr
linux/tools/testing/selftests/powerpc/tm/tm-signal-msr-resv
linux/tools/testing/selftests/powerpc/math/vsx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_signal
linux/tools/testing/selftests/powerpc/math/vmx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_syscall
linux/tools/testing/selftests/powerpc/math/vmx_syscall
linux/tools/testing/selftests/powerpc/math/fpu_preempt
linux/tools/testing/selftests/powerpc/math/vmx_signal
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-vsx
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-vsx
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.chttps://github.com/justdoitqd/publicFiles/blob/master/test_tabort.chttps://github.com/justdoitqd/publicFiles/blob/master/test_kvm_htm_cap.c
Simon Guo (25):
KVM: PPC: Book3S PR: Move kvmppc_save_tm/kvmppc_restore_tm to separate
file
KVM: PPC: Book3S PR: add new parameter (guest MSR) for
kvmppc_save_tm()/kvmppc_restore_tm()
KVM: PPC: Book3S PR: turn on FP/VSX/VMX MSR bits in kvmppc_save_tm()
KVM: PPC: Book3S PR: add C function wrapper for
_kvmppc_save/restore_tm()
KVM: PPC: Book3S PR: In PR KVM suspends Transactional state when
inject an interrupt.
KVM: PPC: Book3S PR: PR KVM pass through MSR TM/TS bits to shadow_msr.
KVM: PPC: Book3S PR: add TEXASR related macros
KVM: PPC: Book3S PR: Sync TM bits to shadow msr for problem state
guest
KVM: PPC: Book3S PR: implement RFID TM behavior to suppress change
from S0 to N0
KVM: PPC: Book3S PR: set MSR HV bit accordingly for PPC970 and others.
KVM: PPC: Book3S PR: prevent TS bits change in kvmppc_interrupt_pr()
powerpc: export symbol msr_check_and_set().
KVM: PPC: Book3S PR: adds new
kvmppc_copyto_vcpu_tm/kvmppc_copyfrom_vcpu_tm API for PR KVM.
KVM: PPC: Book3S PR: export tm_enable()/tm_disable/tm_abort() APIs
KVM: PPC: Book3S PR: add kvmppc_save/restore_tm_sprs() APIs
KVM: PPC: Book3S PR: add transaction memory save/restore skeleton for
PR KVM
KVM: PPC: Book3S PR: add math support for PR KVM HTM
KVM: PPC: Book3S PR: make mtspr/mfspr emulation behavior based on
active TM SPRs
KVM: PPC: Book3S PR: always fail transaction in guest privilege state
KVM: PPC: Book3S PR: enable NV reg restore for reading TM SPR at guest
privilege state
KVM: PPC: Book3S PR: adds emulation for treclaim.
KVM: PPC: Book3S PR: add emulation for trechkpt in PR KVM.
KVM: PPC: Book3S PR: add emulation for tabort. for privilege guest
KVM: PPC: Book3S PR: add guard code to prevent returning to guest with
PR=0 and Transactional state
KVM: PPC: Book3S PR: enable HTM for PR KVM for KVM_CHECK_EXTENSION
ioctl
arch/powerpc/include/asm/asm-prototypes.h | 10 +
arch/powerpc/include/asm/kvm_book3s.h | 8 +
arch/powerpc/include/asm/kvm_host.h | 3 +
arch/powerpc/include/asm/reg.h | 25 +-
arch/powerpc/include/asm/tm.h | 2 -
arch/powerpc/include/uapi/asm/tm.h | 2 +-
arch/powerpc/kernel/process.c | 1 +
arch/powerpc/kernel/tm.S | 12 +
arch/powerpc/kvm/Makefile | 3 +
arch/powerpc/kvm/book3s.h | 1 +
arch/powerpc/kvm/book3s_64_mmu.c | 11 +-
arch/powerpc/kvm/book3s_emulate.c | 279 +++++++++++++++++++-
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 259 ++-----------------
arch/powerpc/kvm/book3s_pr.c | 256 +++++++++++++++++--
arch/powerpc/kvm/book3s_segment.S | 13 +
arch/powerpc/kvm/powerpc.c | 3 +-
arch/powerpc/kvm/tm.S | 379 ++++++++++++++++++++++++++++
arch/powerpc/mm/hash_utils_64.c | 1 +
arch/powerpc/platforms/powernv/copy-paste.h | 3 +-
19 files changed, 982 insertions(+), 289 deletions(-)
create mode 100644 arch/powerpc/kvm/tm.S
From: Benjamin Herrenschmidt <hidden> Date: 2018-01-11 22:04:15
On Thu, 2018-01-11 at 11:56 -0200, Gustavo Romero wrote:
Hi Simon,
On 01/11/2018 08:11 AM, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Is this correct to assume that this emulation mode will just kick in on P9
with kernel TM workarounds and HV KVM will continue to be used on POWER8
since HV KVM is supported on POWER8 hosts?
HV KVM is supported on POWER9. In fact it's PR KVM that isn't (at least
not yet and never will be in Radix mode at least).
Cheers,
Ben.
Regards,
Gustavo
quoted
Test cases performed:
linux/tools/testing/selftests/powerpc/tm/tm-syscall
linux/tools/testing/selftests/powerpc/tm/tm-fork
linux/tools/testing/selftests/powerpc/tm/tm-vmx-unavail
linux/tools/testing/selftests/powerpc/tm/tm-tmspr
linux/tools/testing/selftests/powerpc/tm/tm-signal-msr-resv
linux/tools/testing/selftests/powerpc/math/vsx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_signal
linux/tools/testing/selftests/powerpc/math/vmx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_syscall
linux/tools/testing/selftests/powerpc/math/vmx_syscall
linux/tools/testing/selftests/powerpc/math/fpu_preempt
linux/tools/testing/selftests/powerpc/math/vmx_signal
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-vsx
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-vsx
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.chttps://github.com/justdoitqd/publicFiles/blob/master/test_tabort.chttps://github.com/justdoitqd/publicFiles/blob/master/test_kvm_htm_cap.c
Simon Guo (25):
KVM: PPC: Book3S PR: Move kvmppc_save_tm/kvmppc_restore_tm to separate
file
KVM: PPC: Book3S PR: add new parameter (guest MSR) for
kvmppc_save_tm()/kvmppc_restore_tm()
KVM: PPC: Book3S PR: turn on FP/VSX/VMX MSR bits in kvmppc_save_tm()
KVM: PPC: Book3S PR: add C function wrapper for
_kvmppc_save/restore_tm()
KVM: PPC: Book3S PR: In PR KVM suspends Transactional state when
inject an interrupt.
KVM: PPC: Book3S PR: PR KVM pass through MSR TM/TS bits to shadow_msr.
KVM: PPC: Book3S PR: add TEXASR related macros
KVM: PPC: Book3S PR: Sync TM bits to shadow msr for problem state
guest
KVM: PPC: Book3S PR: implement RFID TM behavior to suppress change
from S0 to N0
KVM: PPC: Book3S PR: set MSR HV bit accordingly for PPC970 and others.
KVM: PPC: Book3S PR: prevent TS bits change in kvmppc_interrupt_pr()
powerpc: export symbol msr_check_and_set().
KVM: PPC: Book3S PR: adds new
kvmppc_copyto_vcpu_tm/kvmppc_copyfrom_vcpu_tm API for PR KVM.
KVM: PPC: Book3S PR: export tm_enable()/tm_disable/tm_abort() APIs
KVM: PPC: Book3S PR: add kvmppc_save/restore_tm_sprs() APIs
KVM: PPC: Book3S PR: add transaction memory save/restore skeleton for
PR KVM
KVM: PPC: Book3S PR: add math support for PR KVM HTM
KVM: PPC: Book3S PR: make mtspr/mfspr emulation behavior based on
active TM SPRs
KVM: PPC: Book3S PR: always fail transaction in guest privilege state
KVM: PPC: Book3S PR: enable NV reg restore for reading TM SPR at guest
privilege state
KVM: PPC: Book3S PR: adds emulation for treclaim.
KVM: PPC: Book3S PR: add emulation for trechkpt in PR KVM.
KVM: PPC: Book3S PR: add emulation for tabort. for privilege guest
KVM: PPC: Book3S PR: add guard code to prevent returning to guest with
PR=0 and Transactional state
KVM: PPC: Book3S PR: enable HTM for PR KVM for KVM_CHECK_EXTENSION
ioctl
arch/powerpc/include/asm/asm-prototypes.h | 10 +
arch/powerpc/include/asm/kvm_book3s.h | 8 +
arch/powerpc/include/asm/kvm_host.h | 3 +
arch/powerpc/include/asm/reg.h | 25 +-
arch/powerpc/include/asm/tm.h | 2 -
arch/powerpc/include/uapi/asm/tm.h | 2 +-
arch/powerpc/kernel/process.c | 1 +
arch/powerpc/kernel/tm.S | 12 +
arch/powerpc/kvm/Makefile | 3 +
arch/powerpc/kvm/book3s.h | 1 +
arch/powerpc/kvm/book3s_64_mmu.c | 11 +-
arch/powerpc/kvm/book3s_emulate.c | 279 +++++++++++++++++++-
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 259 ++-----------------
arch/powerpc/kvm/book3s_pr.c | 256 +++++++++++++++++--
arch/powerpc/kvm/book3s_segment.S | 13 +
arch/powerpc/kvm/powerpc.c | 3 +-
arch/powerpc/kvm/tm.S | 379 ++++++++++++++++++++++++++++
arch/powerpc/mm/hash_utils_64.c | 1 +
arch/powerpc/platforms/powernv/copy-paste.h | 3 +-
19 files changed, 982 insertions(+), 289 deletions(-)
create mode 100644 arch/powerpc/kvm/tm.S
From: Simon Guo <hidden> Date: 2018-01-12 02:41:30
Hi Gustavo,
On Thu, Jan 11, 2018 at 11:56:59AM -0200, Gustavo Romero wrote:
Hi Simon,
On 01/11/2018 08:11 AM, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Is this correct to assume that this emulation mode will just kick in on P9
with kernel TM workarounds and HV KVM will continue to be used on POWER8
since HV KVM is supported on POWER8 hosts?
As Ben mentioned, this patch set aims to enhancement PR KVM on Power8
to support transaction memory.
Thanks,
- Simon
Regards,
Gustavo
quoted
Test cases performed:
linux/tools/testing/selftests/powerpc/tm/tm-syscall
linux/tools/testing/selftests/powerpc/tm/tm-fork
linux/tools/testing/selftests/powerpc/tm/tm-vmx-unavail
linux/tools/testing/selftests/powerpc/tm/tm-tmspr
linux/tools/testing/selftests/powerpc/tm/tm-signal-msr-resv
linux/tools/testing/selftests/powerpc/math/vsx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_signal
linux/tools/testing/selftests/powerpc/math/vmx_preempt
linux/tools/testing/selftests/powerpc/math/fpu_syscall
linux/tools/testing/selftests/powerpc/math/vmx_syscall
linux/tools/testing/selftests/powerpc/math/fpu_preempt
linux/tools/testing/selftests/powerpc/math/vmx_signal
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-gpr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spd-vsx
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-spr
linux/tools/testing/selftests/powerpc/ptrace/ptrace-tm-vsx
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.chttps://github.com/justdoitqd/publicFiles/blob/master/test_tabort.chttps://github.com/justdoitqd/publicFiles/blob/master/test_kvm_htm_cap.c
Simon Guo (25):
KVM: PPC: Book3S PR: Move kvmppc_save_tm/kvmppc_restore_tm to separate
file
KVM: PPC: Book3S PR: add new parameter (guest MSR) for
kvmppc_save_tm()/kvmppc_restore_tm()
KVM: PPC: Book3S PR: turn on FP/VSX/VMX MSR bits in kvmppc_save_tm()
KVM: PPC: Book3S PR: add C function wrapper for
_kvmppc_save/restore_tm()
KVM: PPC: Book3S PR: In PR KVM suspends Transactional state when
inject an interrupt.
KVM: PPC: Book3S PR: PR KVM pass through MSR TM/TS bits to shadow_msr.
KVM: PPC: Book3S PR: add TEXASR related macros
KVM: PPC: Book3S PR: Sync TM bits to shadow msr for problem state
guest
KVM: PPC: Book3S PR: implement RFID TM behavior to suppress change
from S0 to N0
KVM: PPC: Book3S PR: set MSR HV bit accordingly for PPC970 and others.
KVM: PPC: Book3S PR: prevent TS bits change in kvmppc_interrupt_pr()
powerpc: export symbol msr_check_and_set().
KVM: PPC: Book3S PR: adds new
kvmppc_copyto_vcpu_tm/kvmppc_copyfrom_vcpu_tm API for PR KVM.
KVM: PPC: Book3S PR: export tm_enable()/tm_disable/tm_abort() APIs
KVM: PPC: Book3S PR: add kvmppc_save/restore_tm_sprs() APIs
KVM: PPC: Book3S PR: add transaction memory save/restore skeleton for
PR KVM
KVM: PPC: Book3S PR: add math support for PR KVM HTM
KVM: PPC: Book3S PR: make mtspr/mfspr emulation behavior based on
active TM SPRs
KVM: PPC: Book3S PR: always fail transaction in guest privilege state
KVM: PPC: Book3S PR: enable NV reg restore for reading TM SPR at guest
privilege state
KVM: PPC: Book3S PR: adds emulation for treclaim.
KVM: PPC: Book3S PR: add emulation for trechkpt in PR KVM.
KVM: PPC: Book3S PR: add emulation for tabort. for privilege guest
KVM: PPC: Book3S PR: add guard code to prevent returning to guest with
PR=0 and Transactional state
KVM: PPC: Book3S PR: enable HTM for PR KVM for KVM_CHECK_EXTENSION
ioctl
arch/powerpc/include/asm/asm-prototypes.h | 10 +
arch/powerpc/include/asm/kvm_book3s.h | 8 +
arch/powerpc/include/asm/kvm_host.h | 3 +
arch/powerpc/include/asm/reg.h | 25 +-
arch/powerpc/include/asm/tm.h | 2 -
arch/powerpc/include/uapi/asm/tm.h | 2 +-
arch/powerpc/kernel/process.c | 1 +
arch/powerpc/kernel/tm.S | 12 +
arch/powerpc/kvm/Makefile | 3 +
arch/powerpc/kvm/book3s.h | 1 +
arch/powerpc/kvm/book3s_64_mmu.c | 11 +-
arch/powerpc/kvm/book3s_emulate.c | 279 +++++++++++++++++++-
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 259 ++-----------------
arch/powerpc/kvm/book3s_pr.c | 256 +++++++++++++++++--
arch/powerpc/kvm/book3s_segment.S | 13 +
arch/powerpc/kvm/powerpc.c | 3 +-
arch/powerpc/kvm/tm.S | 379 ++++++++++++++++++++++++++++
arch/powerpc/mm/hash_utils_64.c | 1 +
arch/powerpc/platforms/powernv/copy-paste.h | 3 +-
19 files changed, 982 insertions(+), 289 deletions(-)
create mode 100644 arch/powerpc/kvm/tm.S
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:36
On Thu, Jan 11, 2018 at 06:11:26PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
This patch adds 2 new APIs: kvmppc_copyto_vcpu_tm() and
kvmppc_copyfrom_vcpu_tm(). These 2 APIs will be used to copy from/to TM
data between VCPU_TM/VCPU area.
PR KVM will use these APIs for treclaim. or trchkpt. emulation.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
Actually, I take that back. You have missed XER. :)
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:36
On Thu, Jan 11, 2018 at 06:11:23PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
Apple G5 machines(PPC970/FX/GX/MP) have supervisor mode disabled and
MSR HV bit is forced into 1. We should follow this in PR KVM guest.
This patch set MSR HV=1 for G5 machines and HV=0 for others on PR
KVM guest.
Signed-off-by: Simon Guo <redacted>
Suggested-by: Paul Mackerras <redacted>
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:36
On Thu, Jan 11, 2018 at 06:11:15PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
HV KVM and PR KVM need different MSR source to indicate whether
treclaim. or trecheckpoint. is necessary.
This patch add new parameter (guest MSR) for these kvmppc_save_tm/
kvmppc_restore_tm() APIs:
- For HV KVM, it is VCPU_MSR
- For PR KVM, it is current host MSR or VCPU_SHADOW_SRR1
This enhancement enables these 2 APIs to be reused by PR KVM later.
And the patch keeps HV KVM logic unchanged.
This patch also reworks kvmppc_save_tm()/kvmppc_restore_tm() to
have a clean ABI: r3 for vcpu and r4 for guest_msr.
Signed-off-by: Simon Guo <redacted>
Question: why do you switch from using HSTATE_HOST_R1 to HSTATE_SCRATCH2
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:36
On Thu, Jan 11, 2018 at 06:11:29PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
The transaction memory checkpoint area save/restore behavior is
triggered when VCPU qemu process is switching out/into CPU. ie.
at kvmppc_core_vcpu_put_pr() and kvmppc_core_vcpu_load_pr().
MSR TM active state is determined by TS bits:
active: 10(transactional) or 01 (suspended)
inactive: 00 (non-transactional)
We don't "fake" TM functionality for guest. We "sync" guest virtual
MSR TM active state(10 or 01) with shadow MSR. That is to say,
we don't emulate a transactional guest with a TM inactive MSR.
TM SPR support(TFIAR/TFAR/TEXASR) has already been supported by
commit 9916d57e64a4 ("KVM: PPC: Book3S PR: Expose TM registers").
Math register support (FPR/VMX/VSX) will be done at subsequent
patch.
- TM save:
When kvmppc_save_tm_pr() is invoked, whether TM context need to
be saved can be determined by current host MSR state:
* TM active - save TM context
* TM inactive - no need to do so and only save TM SPRs.
- TM restore:
However when kvmppc_restore_tm_pr() is invoked, there is an
issue to determine whether TM restore should be performed.
The TM active host MSR val saved in kernel stack is not loaded yet.
I don't follow this exactly. What is the value saved on the kernel
stack?
I get that we may not have done the sync from the shadow MSR back to
the guest MSR, since that is done in kvmppc_handle_exit_pr() with
interrupts enabled and we might be unloading because we got
preempted. In that case we would have svcpu->in_use = 1, and we
should in fact do the sync of the TS bits from shadow_msr to the vcpu
MSR value in kvmppc_copy_from_svcpu(). If you did that then both the
load and put functions could just rely on the vcpu's MSR value.
quoted hunk
We don't know whether there is a transaction to be restored from
current host MSR TM status at kvmppc_restore_tm_pr(). To solve this
issue, we save current MSR into vcpu->arch.save_msr_tm at
kvmppc_save_tm_pr(), and kvmppc_restore_tm_pr() check TS bits of
vcpu->arch.save_msr_tm to decide whether to do TM restore.
Signed-off-by: Simon Guo <redacted>
Suggested-by: Paul Mackerras <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 6 +++++
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/kvm/book3s_pr.c | 41 +++++++++++++++++++++++++++++++++++
3 files changed, 48 insertions(+)
It would be cleaner at the point where you use these if you added a
#else clause to define a null version for the case when transactional
memory support is not configured, like this:
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+void kvmppc_save_tm_pr(struct kvm_vcpu *vcpu);
+void kvmppc_restore_tm_pr(struct kvm_vcpu *vcpu);
+#else
+static inline void kvmppc_save_tm_pr(struct kvm_vcpu *vcpu) {}
+static inline void kvmppc_restore_tm_pr(struct kvm_vcpu *vcpu) {}
+#endif
That way you don't need the #ifdef at the call site.
I think you should do these giveup_ext/giveup_fac calls before calling
kvmppc_save_tm_pr, because the treclaim in kvmppc_save_tm_pr will
modify all the FP/VEC/VSX registers and the TAR.
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:36
On Thu, Jan 11, 2018 at 06:11:17PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
Currently _kvmppc_save/restore_tm() APIs can only be invoked from
assembly function. This patch adds C function wrappers for them so
that they can be safely called from C function.
Signed-off-by: Simon Guo <redacted>
@@ -126,4 +126,11 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void_mcount(void);unsignedlongprepare_ftrace_return(unsignedlongparent,unsignedlongip);+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/* Transaction memory related */+structkvm_vcpu;+void_kvmppc_restore_tm_pr(structkvm_vcpu*vcpu,u64guest_msr);+void_kvmppc_save_tm_pr(structkvm_vcpu*vcpu,u64guest_msr);+#endif
It's not generally necessary to have ifdefs around function
declarations. If the function is never defined because the feature
is not configured in, that is fine.
quoted hunk
@@ -149,6 +149,58 @@ _GLOBAL(kvmppc_save_tm) blr /*+ * _kvmppc_save_tm() is a wrapper around __kvmppc_save_tm(), so that it can+ * be invoked from C function by PR KVM only.+ */+_GLOBAL(_kvmppc_save_tm_pr)+ mflr r5+ std r5, PPC_LR_STKOFF(r1)+ stdu r1, -SWITCH_FRAME_SIZE(r1)+ SAVE_NVGPRS(r1)++ /* save MSR since TM/math bits might be impacted+ * by __kvmppc_save_tm().+ */+ mfmsr r5+ SAVE_GPR(5, r1)++ /* also save DSCR/CR so that it can be recovered later */+ mfspr r6, SPRN_DSCR+ SAVE_GPR(6, r1)++ mfcr r7+ stw r7, _CCR(r1)++ /* allocate stack frame for __kvmppc_save_tm since+ * it will save LR into its stackframe and we don't+ * want to corrupt _kvmppc_save_tm_pr's.+ */+ stdu r1, -PPC_MIN_STKFRM(r1)
You don't need to do this. In the PowerPC ELF ABI, functions always
save their LR (i.e. their return address) in their *caller's* stack
frame, not their own. You have established a stack frame for
_kvmppc_save_tm_pr above, and that is sufficient. Same comment
applies for _kvmppc_restore_tm_pr.
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:37
On Thu, Jan 11, 2018 at 06:11:20PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
This patches add some macros for CR0/TEXASR bits so that PR KVM TM
logic(tbegin./treclaim./tabort.) can make use of them later.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
This and some of your other patches will need to go via Michael
Ellerman's tree.
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:09:37
On Thu, Jan 11, 2018 at 06:11:13PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Thanks for the patch set. It mostly looks good, though I have some
comments on the individual patches.
I don't see where you are implementing support for userspace accessing
the TM checkpointed register values using the GET_ONE_REG/SET_ONE_REG
API. This would mean that you couldn't migrate a guest that was in
the middle of a transaction. We will need to have the one_reg API
access to the TM checkpoint implemented, though there will be a
difficulty in that kvmppc_get_one_reg() and kvmppc_set_one_reg() are
called with the vcpu context loaded. With your scheme of having the
TM checkpoint stored in the CPU while the vcpu context is loaded, the
values you want to access in kvmppc_get/set_one_reg are inaccessible
since they're stored in the CPU. You would have to arrange for
kvmppc_get/set_one_reg to be called without the vcpu context loaded
(recent patches in the kvm next branch probably make that easier) or
else explicitly unload and reload the vcpu context in those functions.
(This is easier in HV KVM since the checkpoint is not in the CPU at
the point of doing kvmppc_get/set_one_reg.)
There is also complexity added because it's possible for the guest to
have TM, FP, VEC and VSX all enabled from its point of view but to
have FP/VEC/VSX not actually enabled in the hardware when the guest is
running. As you note in your patch descriptions, this means that the
guest can do tbegin and create a checkpoint with bogus values for the
FP/VEC/VSX registers. Rather than trying to detect and fix up this
situation after the fact, I would suggest that if the guest has TM
enabled then we make sure that the real FP/VEC/VSX bits in the MSR
match what the guest thinks it has. That way we would avoid the bogus
checkpoint problem. (There is still the possibility of getting bogus
checkpointed FP/VEC/VSX registers if the guest does tbegin with the
FP/VEC/VSX bits clear in the MSR, but that is the guest's problem to
deal with.)
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:16:52
On Tue, Jan 23, 2018 at 04:38:32PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:13PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Thanks for the patch set. It mostly looks good, though I have some
comments on the individual patches.
I don't see where you are implementing support for userspace accessing
the TM checkpointed register values using the GET_ONE_REG/SET_ONE_REG
API. This would mean that you couldn't migrate a guest that was in
the middle of a transaction. We will need to have the one_reg API
access to the TM checkpoint implemented, though there will be a
difficulty in that kvmppc_get_one_reg() and kvmppc_set_one_reg() are
called with the vcpu context loaded. With your scheme of having the
TM checkpoint stored in the CPU while the vcpu context is loaded, the
values you want to access in kvmppc_get/set_one_reg are inaccessible
since they're stored in the CPU. You would have to arrange for
kvmppc_get/set_one_reg to be called without the vcpu context loaded
(recent patches in the kvm next branch probably make that easier) or
else explicitly unload and reload the vcpu context in those functions.
(This is easier in HV KVM since the checkpoint is not in the CPU at
the point of doing kvmppc_get/set_one_reg.)
Another complexity that hasn't been dealt with as far as I can see is
that if userspace does a KVM_SET_REGS that changes the TS field in the
guest MSR, we don't do anything to make the state of the CPU match.
As with GET/SET_ONE_REG, KVM_SET_REGS is called with the vcpu loaded,
so it needs to make the physical CPU state match what it would have
been had the new state been present at load time, perhaps by unloading
the CPU before changing the state, then reloading it. But if you do
that and you are using the vcpu->arch.save_msr_tm field that you add,
then you need to modify that when you do kvmppc_set_msr(). (I would
rather that your save/restore TM functions work off kvmppc_get_msr()
rather than having the save_msr_tm field.)
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 07:29:41
On Thu, Jan 11, 2018 at 06:11:30PM +0800, wei.guo.simon@gmail.com wrote:
ines: 219
From: Simon Guo <redacted>
The math registers will be saved into vcpu->arch.fp/vr and corresponding
vcpu->arch.fp_tm/vr_tm area.
We flush or giveup the math regs into vcpu->arch.fp/vr before saving
transaction. After transaction is restored, the math regs will be loaded
back into regs.
It looks to me that you are loading up the math regs on every vcpu
load, not just those with an active transaction. That seems like
overkill.
If there is a FP/VEC/VSX unavailable exception during transaction active
state, the math checkpoint content might be incorrect and we need to do
treclaim./load the correct checkpoint val/trechkpt. sequence to retry the
transaction.
I would prefer a simpler approach where just before entering the
guest, we check if the guest MSR TM bit is set, and if so we make sure
that whichever math regs are enabled in the guest MSR are actually
loaded on the CPU, that is, that guest_owned_ext has the same bits set
as the guest MSR. Then we never have to handle a FP/VEC/VSX
unavailable interrupt with a transaction active (other than by simply
passing it on to the guest).
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 08:17:53
On Thu, Jan 11, 2018 at 06:11:31PM +0800, wei.guo.simon@gmail.com wrote:
quoted hunk
From: Simon Guo <redacted>
The mfspr/mtspr on TM SPRs(TEXASR/TFIAR/TFHAR) are non-privileged
instructions and can be executed at PR KVM guest without trapping
into host in problem state. We only emulate mtspr/mfspr
texasr/tfiar/tfhar at guest PR=0 state.
When we are emulating mtspr tm sprs at guest PR=0 state, the emulation
result need to be visible to guest PR=1 state. That is, the actual TM
SPR val should be loaded into actual registers.
We already flush TM SPRs into vcpu when switching out of CPU, and load
TM SPRs when switching back.
This patch corrects mfspr()/mtspr() emulation for TM SPRs to make the
actual source/dest based on actual TM SPRs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 35 +++++++++++++++++++++++++++--------
1 file changed, 27 insertions(+), 8 deletions(-)
@@ -521,13 +521,26 @@ int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)break;#ifdef CONFIG_PPC_TRANSACTIONAL_MEMcaseSPRN_TFHAR:-vcpu->arch.tfhar=spr_val;-break;caseSPRN_TEXASR:-vcpu->arch.texasr=spr_val;-break;caseSPRN_TFIAR:-vcpu->arch.tfiar=spr_val;+if(MSR_TM_ACTIVE(kvmppc_get_msr(vcpu))){+/* it is illegal to mtspr() TM regs in+*otherthannon-transactionalstate.+*/+kvmppc_core_queue_program(vcpu,SRR1_PROGTM);+emulated=EMULATE_AGAIN;+break;+}
We also need to check that the guest has TM enabled in the guest MSR,
and give them a facility unavailable interrupt if not.
I haven't seen any checks that we are on a CPU that has TM. What
happens if a guest does a mtmsrd with TM=1 and then a mtspr to TEXASR
when running on a POWER7 (assuming the host kernel was compiled with
CONFIG_PPC_TRANSACTIONAL_MEM=y)?
Ideally, if the host CPU does not have TM functionality, these mtsprs
would be treated as no-ops and attempts to set the TM or TS fields in
the guest MSR would be ignored.
quoted hunk
+
break;
#endif
#endif
@@ -674,13 +687,19 @@ int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val break; #ifdef CONFIG_PPC_TRANSACTIONAL_MEM case SPRN_TFHAR:- *spr_val = vcpu->arch.tfhar;+ tm_enable();+ *spr_val = mfspr(SPRN_TFHAR);+ tm_disable(); break; case SPRN_TEXASR:- *spr_val = vcpu->arch.texasr;+ tm_enable();+ *spr_val = mfspr(SPRN_TEXASR);+ tm_disable(); break; case SPRN_TFIAR:- *spr_val = vcpu->arch.tfiar;+ tm_enable();+ *spr_val = mfspr(SPRN_TFIAR);+ tm_disable(); break;
These need to check MSR_TM in the guest MSR, and become no-ops on
machines without TM capability.
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 08:30:37
On Thu, Jan 11, 2018 at 06:11:32PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
Currently kernel doesn't use transaction memory.
And there is an issue for privilege guest that:
tbegin/tsuspend/tresume/tabort TM instructions can impact MSR TM bits
without trap into PR host. So following code will lead to a false mfmsr
result:
tbegin <- MSR bits update to Transaction active.
beq <- failover handler branch
mfmsr <- still read MSR bits from magic page with
transaction inactive.
It is not an issue for non-privilege guest since its mfmsr is not patched
with magic page and will always trap into PR host.
This patch will always fail tbegin attempt for privilege guest, so that
the above issue is prevented. It is benign since currently (guest) kernel
doesn't initiate a transaction.
Test case:
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.c
Signed-off-by: Simon Guo <redacted>
You need to handle the case where MSR_TM is not set in the guest MSR,
and give the guest a facility unavailable interrupt.
[snip]
From: Paul Mackerras <hidden> Date: 2018-01-23 09:08:31
On Thu, Jan 11, 2018 at 06:11:33PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
Currently kvmppc_handle_fac() will not update NV GPRs and thus it can
return with GUEST_RESUME.
However PR KVM guest always disables MSR_TM bit at privilege state. If PR
privilege guest are trying to read TM SPRs, it will trigger TM facility
unavailable exception and fall into kvmppc_handle_fac(). Then the emulation
will be done by kvmppc_core_emulate_mfspr_pr(). The mfspr instruction can
include a RT with NV reg. So it is necessary to restore NV GPRs at this
case, to reflect the update to NV RT.
This patch make kvmppc_handle_fac() return GUEST_RESUME_NV at TM fac
exception and with guest privilege state.
Signed-off-by: Simon Guo <redacted>
From: Paul Mackerras <hidden> Date: 2018-01-23 09:36:49
On Thu, Jan 11, 2018 at 06:11:34PM +0800, wei.guo.simon@gmail.com wrote:
quoted hunk
From: Simon Guo <redacted>
This patch adds support for "treclaim." emulation when PR KVM guest
executes treclaim. and traps to host.
We will firstly doing treclaim. and save TM checkpoint and doing
treclaim. Then it is necessary to update vcpu current reg content
with checkpointed vals. When rfid into guest again, those vcpu
current reg content(now the checkpoint vals) will be loaded into
regs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/reg.h | 4 +++
arch/powerpc/kvm/book3s_emulate.c | 66 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 69 insertions(+), 1 deletion(-)
It would be good to collect up all the modifications you need to make
to reg.h into a single patch at the beginning of the patch series --
that will make it easier to merge it all.
You're doing failure recording here unconditionally, but the
architecture says that treclaim. only does failure recording if
TEXASR_FS is not already set.
From: Paul Mackerras <hidden> Date: 2018-01-23 09:36:49
On Thu, Jan 11, 2018 at 06:11:35PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
This patch adds host emulation when guest PR KVM executes "trechkpt.",
which is a privileged instruction and will trap into host.
We firstly copy vcpu ongoing content into vcpu tm checkpoint
content, then perform kvmppc_restore_tm_pr() to do trechkpt.
with updated vcpu tm checkpoint vals.
Signed-off-by: Simon Guo <redacted>
This looks odd, since you are clearing bits when you have just set
save_msr_tm to a constant value that doesn't have these bits set.
This could be taken as a sign that the previous line has a bug and you
meant "|=" or something similar instead of "=". I think you probably
did mean "=", in which case you should remove the line clearing
FP/VEC/VSX.
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-23 09:44:28
On Thu, Jan 11, 2018 at 06:11:36PM +0800, wei.guo.simon@gmail.com wrote:
quoted hunk
From: Simon Guo <redacted>
Currently privilege guest will be run with TM disabled.
Although the privilege guest cannot initiate a new transaction,
it can use tabort to terminate its problem state's transaction.
So it is still necessary to emulate tabort. for privilege guest.
This patch adds emulation for tabort. of privilege guest.
Tested with:
https://github.com/justdoitqd/publicFiles/blob/master/test_tabort.c
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 31 +++++++++++++++++++++++++++++++
arch/powerpc/kvm/book3s_pr.c | 2 +-
3 files changed, 33 insertions(+), 1 deletion(-)
Why do you add this declaration, and change it from "static inline" to
"inline" below, when this patch doesn't use it? Also, making it
"inline" is pointless if it has a caller outside the source file where
it's defined (if gcc wants to inline uses of it inside the same source
file, it will do so anyway even without the "inline" keyword.)
Paul.
From: Paul Mackerras <hidden> Date: 2018-01-24 09:46:19
On Thu, Jan 11, 2018 at 06:11:38PM +0800, wei.guo.simon@gmail.com wrote:
From: Simon Guo <redacted>
Currently guest kernel doesn't handle TAR fac unavailable and it always
runs with TAR bit on. PR KVM will lazily enable TAR. TAR is not a
frequent-use reg and it is not included in SVCPU struct.
To make it work for transaction memory at PR KVM:
1). Flush/giveup TAR at kvmppc_save_tm_pr().
2) If we are receiving a TAR fac unavail exception inside a transaction,
the checkpointed TAR might be a TAR value from another process. So we need
treclaim the transaction, then load the desired TAR value into reg, and
perform trecheckpoint.
3) Load TAR facility at kvmppc_restore_tm_pr() when TM active.
The reason we always loads TAR when restoring TM is that:
If we don't do this way, when there is a TAR fac unavailable exception
during TM active:
case 1: it is the 1st TAR fac unavail exception after tbegin.
vcpu->arch.tar should be reloaded as checkpoint tar val.
case 2: it is the 2nd or later TAR fac unavail exception after tbegin.
vcpu->arch.tar_tm should be reloaded as checkpoint tar val.
There will be unnecessary difficulty to handle the above 2 cases.
at the end of emulating treclaim., the correct TAR val need to be loaded
into reg if FSCR_TAR bit is on.
at the beginning of emulating trechkpt., TAR needs to be flushed so that
the right tar val can be copy into tar_tm.
Would it be simpler always to load up TAR when guest_MSR[TM] is 1?
Paul.
From: Simon Guo <hidden> Date: 2018-01-27 13:10:18
Hi Paul,
On Tue, Jan 23, 2018 at 04:38:32PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:13PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
In current days, many OS distributions have utilized transaction
memory functionality. In PowerPC, HV KVM supports TM. But PR KVM
does not.
The drive for the transaction memory support of PR KVM is the
openstack Continuous Integration testing - They runs a HV(hypervisor)
KVM(as level 1) and then run PR KVM(as level 2) on top of that.
This patch set add transaction memory support on PR KVM.
Thanks for the patch set. It mostly looks good, though I have some
comments on the individual patches.
I don't see where you are implementing support for userspace accessing
the TM checkpointed register values using the GET_ONE_REG/SET_ONE_REG
API. This would mean that you couldn't migrate a guest that was in
the middle of a transaction. We will need to have the one_reg API
access to the TM checkpoint implemented, though there will be a
difficulty in that kvmppc_get_one_reg() and kvmppc_set_one_reg() are
called with the vcpu context loaded. With your scheme of having the
TM checkpoint stored in the CPU while the vcpu context is loaded, the
values you want to access in kvmppc_get/set_one_reg are inaccessible
since they're stored in the CPU. You would have to arrange for
kvmppc_get/set_one_reg to be called without the vcpu context loaded
(recent patches in the kvm next branch probably make that easier) or
else explicitly unload and reload the vcpu context in those functions.
(This is easier in HV KVM since the checkpoint is not in the CPU at
the point of doing kvmppc_get/set_one_reg.)
Thanks for point it out. I didn't think about it before and will
investigate.
I plan to work out this PR KVM HTM kvmppc_get/set_one_reg()
(and the KVM_SET_REGS you mentioned in another mail) with seperate
patch/patch set, so that the reworked V2 of current patches can be
sent out in parallel. In case it is not appropriate for you, please
let me know.
There is also complexity added because it's possible for the guest to
have TM, FP, VEC and VSX all enabled from its point of view but to
have FP/VEC/VSX not actually enabled in the hardware when the guest is
running. As you note in your patch descriptions, this means that the
guest can do tbegin and create a checkpoint with bogus values for the
FP/VEC/VSX registers. Rather than trying to detect and fix up this
situation after the fact, I would suggest that if the guest has TM
enabled then we make sure that the real FP/VEC/VSX bits in the MSR
match what the guest thinks it has. That way we would avoid the bogus
checkpoint problem. (There is still the possibility of getting bogus
checkpointed FP/VEC/VSX registers if the guest does tbegin with the
FP/VEC/VSX bits clear in the MSR, but that is the guest's problem to
deal with.)
Good idea. I will look into kvmppc_set_msr_pr() / kvmppc_giveup_ext()
to simplify the solution.
Thanks for your review and time.
BR,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 02:15:13
Hi Paul,
On Tue, Jan 23, 2018 at 04:52:19PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:26PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
This patch adds 2 new APIs: kvmppc_copyto_vcpu_tm() and
kvmppc_copyfrom_vcpu_tm(). These 2 APIs will be used to copy from/to TM
data between VCPU_TM/VCPU area.
PR KVM will use these APIs for treclaim. or trchkpt. emulation.
Signed-off-by: Simon Guo <redacted>
Reviewed-by: Paul Mackerras <redacted>
Actually, I take that back. You have missed XER. :)
From: Simon Guo <hidden> Date: 2018-01-30 02:33:59
Hi Paul,
On Tue, Jan 23, 2018 at 04:42:09PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:15PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
HV KVM and PR KVM need different MSR source to indicate whether
treclaim. or trecheckpoint. is necessary.
This patch add new parameter (guest MSR) for these kvmppc_save_tm/
kvmppc_restore_tm() APIs:
- For HV KVM, it is VCPU_MSR
- For PR KVM, it is current host MSR or VCPU_SHADOW_SRR1
This enhancement enables these 2 APIs to be reused by PR KVM later.
And the patch keeps HV KVM logic unchanged.
This patch also reworks kvmppc_save_tm()/kvmppc_restore_tm() to
have a clean ABI: r3 for vcpu and r4 for guest_msr.
Signed-off-by: Simon Guo <redacted>
Question: why do you switch from using HSTATE_HOST_R1 to HSTATE_SCRATCH2
@@ -166,17 +173,17 @@ _GLOBAL(kvmppc_restore_tm) * The user may change these outside of a transaction, so they must * always be context switched. */- ld r5, VCPU_TFHAR(r4)- ld r6, VCPU_TFIAR(r4)- ld r7, VCPU_TEXASR(r4)+ ld r5, VCPU_TFHAR(r3)+ ld r6, VCPU_TFIAR(r3)+ ld r7, VCPU_TEXASR(r3) mtspr SPRN_TFHAR, r5 mtspr SPRN_TFIAR, r6 mtspr SPRN_TEXASR, r7- ld r5, VCPU_MSR(r4)+ mr r5, r4 rldicl. r5, r5, 64 - MSR_TS_S_LG, 62 beqlr /* TM not active in guest */- std r1, HSTATE_HOST_R1(r13)+ std r1, HSTATE_SCRATCH2(r13)
and here?
Please add a paragraph to the patch description explaining why you are
making that change.
In subsequent patches, kvmppc_save_tm/kvmppc_restore_tm() will be
invoked by wrapper function who setup addtional stack frame and
update R1(then update HSTATE_HOST_R1 with addtional offset). Although
HSTATE_HOST_R1 is now used safely(always PPC_STL before entering
guest and PPC_LL in kvmppc_interrupt_pr()), I worried a future usage
will take an assumption on HSTATE_HOST_R1 value and bring trouble.
As a result, in kvmppc_save_tm/kvmppc_restore_tm() case, I choose
HSTATE_SCRATCH2 to restore the r1. I will update the commit message.
Thanks,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 02:38:40
Hi Paul,
On Tue, Jan 23, 2018 at 04:49:16PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:17PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
Currently _kvmppc_save/restore_tm() APIs can only be invoked from
assembly function. This patch adds C function wrappers for them so
that they can be safely called from C function.
Signed-off-by: Simon Guo <redacted>
@@ -126,4 +126,11 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,void_mcount(void);unsignedlongprepare_ftrace_return(unsignedlongparent,unsignedlongip);+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM+/* Transaction memory related */+structkvm_vcpu;+void_kvmppc_restore_tm_pr(structkvm_vcpu*vcpu,u64guest_msr);+void_kvmppc_save_tm_pr(structkvm_vcpu*vcpu,u64guest_msr);+#endif
It's not generally necessary to have ifdefs around function
declarations. If the function is never defined because the feature
is not configured in, that is fine.
Got it. Thanks.
quoted
@@ -149,6 +149,58 @@ _GLOBAL(kvmppc_save_tm) blr /*+ * _kvmppc_save_tm() is a wrapper around __kvmppc_save_tm(), so that it can+ * be invoked from C function by PR KVM only.+ */+_GLOBAL(_kvmppc_save_tm_pr)+ mflr r5+ std r5, PPC_LR_STKOFF(r1)+ stdu r1, -SWITCH_FRAME_SIZE(r1)+ SAVE_NVGPRS(r1)++ /* save MSR since TM/math bits might be impacted+ * by __kvmppc_save_tm().+ */+ mfmsr r5+ SAVE_GPR(5, r1)++ /* also save DSCR/CR so that it can be recovered later */+ mfspr r6, SPRN_DSCR+ SAVE_GPR(6, r1)++ mfcr r7+ stw r7, _CCR(r1)++ /* allocate stack frame for __kvmppc_save_tm since+ * it will save LR into its stackframe and we don't+ * want to corrupt _kvmppc_save_tm_pr's.+ */+ stdu r1, -PPC_MIN_STKFRM(r1)
You don't need to do this. In the PowerPC ELF ABI, functions always
save their LR (i.e. their return address) in their *caller's* stack
frame, not their own. You have established a stack frame for
_kvmppc_save_tm_pr above, and that is sufficient. Same comment
applies for _kvmppc_restore_tm_pr.
From: Simon Guo <hidden> Date: 2018-01-30 02:58:02
Hi Paul,
On Tue, Jan 23, 2018 at 05:04:09PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:29PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
The transaction memory checkpoint area save/restore behavior is
triggered when VCPU qemu process is switching out/into CPU. ie.
at kvmppc_core_vcpu_put_pr() and kvmppc_core_vcpu_load_pr().
MSR TM active state is determined by TS bits:
active: 10(transactional) or 01 (suspended)
inactive: 00 (non-transactional)
We don't "fake" TM functionality for guest. We "sync" guest virtual
MSR TM active state(10 or 01) with shadow MSR. That is to say,
we don't emulate a transactional guest with a TM inactive MSR.
TM SPR support(TFIAR/TFAR/TEXASR) has already been supported by
commit 9916d57e64a4 ("KVM: PPC: Book3S PR: Expose TM registers").
Math register support (FPR/VMX/VSX) will be done at subsequent
patch.
- TM save:
When kvmppc_save_tm_pr() is invoked, whether TM context need to
be saved can be determined by current host MSR state:
* TM active - save TM context
* TM inactive - no need to do so and only save TM SPRs.
- TM restore:
However when kvmppc_restore_tm_pr() is invoked, there is an
issue to determine whether TM restore should be performed.
The TM active host MSR val saved in kernel stack is not loaded yet.
I don't follow this exactly. What is the value saved on the kernel
stack?
I get that we may not have done the sync from the shadow MSR back to
the guest MSR, since that is done in kvmppc_handle_exit_pr() with
interrupts enabled and we might be unloading because we got
preempted. In that case we would have svcpu->in_use = 1, and we
should in fact do the sync of the TS bits from shadow_msr to the vcpu
MSR value in kvmppc_copy_from_svcpu(). If you did that then both the
load and put functions could just rely on the vcpu's MSR value.
Yes. that looks more clean and simpler!
quoted
We don't know whether there is a transaction to be restored from
current host MSR TM status at kvmppc_restore_tm_pr(). To solve this
issue, we save current MSR into vcpu->arch.save_msr_tm at
kvmppc_save_tm_pr(), and kvmppc_restore_tm_pr() check TS bits of
vcpu->arch.save_msr_tm to decide whether to do TM restore.
Signed-off-by: Simon Guo <redacted>
Suggested-by: Paul Mackerras <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 6 +++++
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/kvm/book3s_pr.c | 41 +++++++++++++++++++++++++++++++++++
3 files changed, 48 insertions(+)
It would be cleaner at the point where you use these if you added a
#else clause to define a null version for the case when transactional
memory support is not configured, like this:
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+void kvmppc_save_tm_pr(struct kvm_vcpu *vcpu);
+void kvmppc_restore_tm_pr(struct kvm_vcpu *vcpu);
+#else
+static inline void kvmppc_save_tm_pr(struct kvm_vcpu *vcpu) {}
+static inline void kvmppc_restore_tm_pr(struct kvm_vcpu *vcpu) {}
+#endif
That way you don't need the #ifdef at the call site.
I think you should do these giveup_ext/giveup_fac calls before calling
kvmppc_save_tm_pr, because the treclaim in kvmppc_save_tm_pr will
modify all the FP/VEC/VSX registers and the TAR.
I handled giveup_ext/giveup_fac() within kvmppc_save_tm_pr(), so that
other place (like kvmppc_emulate_treclaim() can invoke
kvmppc_save_tm_pr() easily). But I think moving the calls sequence as
you suggested above will be more readable.
Thanks,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:00:09
Hi Paul,
On Tue, Jan 23, 2018 at 06:29:27PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:30PM +0800, wei.guo.simon@gmail.com wrote:
quoted
ines: 219
From: Simon Guo <redacted>
The math registers will be saved into vcpu->arch.fp/vr and corresponding
vcpu->arch.fp_tm/vr_tm area.
We flush or giveup the math regs into vcpu->arch.fp/vr before saving
transaction. After transaction is restored, the math regs will be loaded
back into regs.
It looks to me that you are loading up the math regs on every vcpu
load, not just those with an active transaction. That seems like
overkill.
quoted
If there is a FP/VEC/VSX unavailable exception during transaction active
state, the math checkpoint content might be incorrect and we need to do
treclaim./load the correct checkpoint val/trechkpt. sequence to retry the
transaction.
I would prefer a simpler approach where just before entering the
guest, we check if the guest MSR TM bit is set, and if so we make sure
that whichever math regs are enabled in the guest MSR are actually
loaded on the CPU, that is, that guest_owned_ext has the same bits set
as the guest MSR. Then we never have to handle a FP/VEC/VSX
unavailable interrupt with a transaction active (other than by simply
passing it on to the guest).
Good idea. I will rework as this way.
Thanks,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:03:00
Hi Paul,
On Tue, Jan 23, 2018 at 07:17:45PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:31PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
The mfspr/mtspr on TM SPRs(TEXASR/TFIAR/TFHAR) are non-privileged
instructions and can be executed at PR KVM guest without trapping
into host in problem state. We only emulate mtspr/mfspr
texasr/tfiar/tfhar at guest PR=0 state.
When we are emulating mtspr tm sprs at guest PR=0 state, the emulation
result need to be visible to guest PR=1 state. That is, the actual TM
SPR val should be loaded into actual registers.
We already flush TM SPRs into vcpu when switching out of CPU, and load
TM SPRs when switching back.
This patch corrects mfspr()/mtspr() emulation for TM SPRs to make the
actual source/dest based on actual TM SPRs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/kvm/book3s_emulate.c | 35 +++++++++++++++++++++++++++--------
1 file changed, 27 insertions(+), 8 deletions(-)
@@ -521,13 +521,26 @@ int kvmppc_core_emulate_mtspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)break;#ifdef CONFIG_PPC_TRANSACTIONAL_MEMcaseSPRN_TFHAR:-vcpu->arch.tfhar=spr_val;-break;caseSPRN_TEXASR:-vcpu->arch.texasr=spr_val;-break;caseSPRN_TFIAR:-vcpu->arch.tfiar=spr_val;+if(MSR_TM_ACTIVE(kvmppc_get_msr(vcpu))){+/* it is illegal to mtspr() TM regs in+*otherthannon-transactionalstate.+*/+kvmppc_core_queue_program(vcpu,SRR1_PROGTM);+emulated=EMULATE_AGAIN;+break;+}
We also need to check that the guest has TM enabled in the guest MSR,
and give them a facility unavailable interrupt if not.
I haven't seen any checks that we are on a CPU that has TM. What
happens if a guest does a mtmsrd with TM=1 and then a mtspr to TEXASR
when running on a POWER7 (assuming the host kernel was compiled with
CONFIG_PPC_TRANSACTIONAL_MEM=y)?
Ideally, if the host CPU does not have TM functionality, these mtsprs
would be treated as no-ops and attempts to set the TM or TS fields in
the guest MSR would be ignored.
quoted
+
break;
#endif
#endif
@@ -674,13 +687,19 @@ int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val break; #ifdef CONFIG_PPC_TRANSACTIONAL_MEM case SPRN_TFHAR:- *spr_val = vcpu->arch.tfhar;+ tm_enable();+ *spr_val = mfspr(SPRN_TFHAR);+ tm_disable(); break; case SPRN_TEXASR:- *spr_val = vcpu->arch.texasr;+ tm_enable();+ *spr_val = mfspr(SPRN_TEXASR);+ tm_disable(); break; case SPRN_TFIAR:- *spr_val = vcpu->arch.tfiar;+ tm_enable();+ *spr_val = mfspr(SPRN_TFIAR);+ tm_disable(); break;
These need to check MSR_TM in the guest MSR, and become no-ops on
machines without TM capability.
Thanks for the above catches. I will rework later.
BR,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:11:30
Hi Paul,
On Tue, Jan 23, 2018 at 07:30:33PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:32PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
Currently kernel doesn't use transaction memory.
And there is an issue for privilege guest that:
tbegin/tsuspend/tresume/tabort TM instructions can impact MSR TM bits
without trap into PR host. So following code will lead to a false mfmsr
result:
tbegin <- MSR bits update to Transaction active.
beq <- failover handler branch
mfmsr <- still read MSR bits from magic page with
transaction inactive.
It is not an issue for non-privilege guest since its mfmsr is not patched
with magic page and will always trap into PR host.
This patch will always fail tbegin attempt for privilege guest, so that
the above issue is prevented. It is benign since currently (guest) kernel
doesn't initiate a transaction.
Test case:
https://github.com/justdoitqd/publicFiles/blob/master/test_tbegin_pr.c
Signed-off-by: Simon Guo <redacted>
You need to handle the case where MSR_TM is not set in the guest MSR,
and give the guest a facility unavailable interrupt.
From: Simon Guo <hidden> Date: 2018-01-30 03:13:55
Hi Paul,
On Tue, Jan 23, 2018 at 08:36:44PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:35PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
This patch adds host emulation when guest PR KVM executes "trechkpt.",
which is a privileged instruction and will trap into host.
We firstly copy vcpu ongoing content into vcpu tm checkpoint
content, then perform kvmppc_restore_tm_pr() to do trechkpt.
with updated vcpu tm checkpoint vals.
Signed-off-by: Simon Guo <redacted>
This looks odd, since you are clearing bits when you have just set
save_msr_tm to a constant value that doesn't have these bits set.
This could be taken as a sign that the previous line has a bug and you
meant "|=" or something similar instead of "=". I think you probably
did mean "=", in which case you should remove the line clearing
FP/VEC/VSX.
I will rework and remove "save_msr_tm" from the code.
Thanks,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:18:09
Hi Paul,
On Tue, Jan 23, 2018 at 08:23:23PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:34PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
This patch adds support for "treclaim." emulation when PR KVM guest
executes treclaim. and traps to host.
We will firstly doing treclaim. and save TM checkpoint and doing
treclaim. Then it is necessary to update vcpu current reg content
with checkpointed vals. When rfid into guest again, those vcpu
current reg content(now the checkpoint vals) will be loaded into
regs.
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/reg.h | 4 +++
arch/powerpc/kvm/book3s_emulate.c | 66 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 69 insertions(+), 1 deletion(-)
It would be good to collect up all the modifications you need to make
to reg.h into a single patch at the beginning of the patch series --
that will make it easier to merge it all.
You're doing failure recording here unconditionally, but the
architecture says that treclaim. only does failure recording if
TEXASR_FS is not already set.
I need add that. And the CR0 setting is also missed.
Thanks for the catch.
[snip]
BR,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:24:41
Hi Paul,
On Tue, Jan 23, 2018 at 08:44:16PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:36PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
Currently privilege guest will be run with TM disabled.
Although the privilege guest cannot initiate a new transaction,
it can use tabort to terminate its problem state's transaction.
So it is still necessary to emulate tabort. for privilege guest.
This patch adds emulation for tabort. of privilege guest.
Tested with:
https://github.com/justdoitqd/publicFiles/blob/master/test_tabort.c
Signed-off-by: Simon Guo <redacted>
---
arch/powerpc/include/asm/kvm_book3s.h | 1 +
arch/powerpc/kvm/book3s_emulate.c | 31 +++++++++++++++++++++++++++++++
arch/powerpc/kvm/book3s_pr.c | 2 +-
3 files changed, 33 insertions(+), 1 deletion(-)
Why do you add this declaration, and change it from "static inline" to
"inline" below, when this patch doesn't use it? Also, making it
"inline" is pointless if it has a caller outside the source file where
it's defined (if gcc wants to inline uses of it inside the same source
file, it will do so anyway even without the "inline" keyword.)
Paul.
It is a leave over of my previous rework. Sorry and I will remove
them.
Thanks,
- Simon
From: Simon Guo <hidden> Date: 2018-01-30 03:26:38
Hi Paul,
On Wed, Jan 24, 2018 at 03:02:58PM +1100, Paul Mackerras wrote:
On Thu, Jan 11, 2018 at 06:11:38PM +0800, wei.guo.simon@gmail.com wrote:
quoted
From: Simon Guo <redacted>
Currently guest kernel doesn't handle TAR fac unavailable and it always
runs with TAR bit on. PR KVM will lazily enable TAR. TAR is not a
frequent-use reg and it is not included in SVCPU struct.
To make it work for transaction memory at PR KVM:
1). Flush/giveup TAR at kvmppc_save_tm_pr().
2) If we are receiving a TAR fac unavail exception inside a transaction,
the checkpointed TAR might be a TAR value from another process. So we need
treclaim the transaction, then load the desired TAR value into reg, and
perform trecheckpoint.
3) Load TAR facility at kvmppc_restore_tm_pr() when TM active.
The reason we always loads TAR when restoring TM is that:
If we don't do this way, when there is a TAR fac unavailable exception
during TM active:
case 1: it is the 1st TAR fac unavail exception after tbegin.
vcpu->arch.tar should be reloaded as checkpoint tar val.
case 2: it is the 2nd or later TAR fac unavail exception after tbegin.
vcpu->arch.tar_tm should be reloaded as checkpoint tar val.
There will be unnecessary difficulty to handle the above 2 cases.
at the end of emulating treclaim., the correct TAR val need to be loaded
into reg if FSCR_TAR bit is on.
at the beginning of emulating trechkpt., TAR needs to be flushed so that
the right tar val can be copy into tar_tm.
Would it be simpler always to load up TAR when guest_MSR[TM] is 1?
Paul.
Sure. it will have a similar solution with math regs.
Thanks for the suggestion,
BR
- Simon