POWER platforms that supports the Protected Execution Facility (PEF)
introduce features that combine hardware facilities and firmware to
enable secure virtual machines. That includes a new processor mode
(ultravisor mode) and the ultravisor firmware.
In PEF enabled systems, the ultravisor firmware runs at a privilege
level above the hypervisor and also takes control over some system
resources. The hypervisor, though, can make system calls to access these
resources. Such system calls, a.k.a. ucalls, are handled by the
ultravisor firmware.
The processor allows part of the system memory to be configured as
secure memory, and introduces a a new mode, called secure mode, where
any software entity in that mode can access secure memory. The
hypervisor doesn't (and can't) run in secure mode, but a secure guest
and the ultravisor firmware do.
This patch set adds support for ultravisor calls and do some preparation
for running secure guests.
---
Changelog:
---
v2->v3:
- Squashed patches:
"[PATCH v2 08/10] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM"
"[PATCH v2 09/10] KVM: PPC: Book3S HV: Fixed for running secure guests"
- Renamed patch from/to:
"[PATCH v2 08/10] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM"
"[PATCH v3 08/09] KVM: PPC: Ultravisor: Enter a secure guest
- Rebased
- Addressed comments from Paul Mackerras
- Dropped ultravisor checks made in power8 code
- Updated the commit message for:
"[PATCH v3 08/09] KVM: PPC: Ultravisor: Enter a secure guest"
- Addressed comments from Maddy
- Dropped imc-pmu.c changes
- Changed opal-imc.c to fail the probe when the ultravisor is enabled
- Fixed "ucall defined but not used" issue when CONFIG_PPC_UV not set
v1->v2:
- Addressed comments from Paul Mackerras:
- Write the pate in HV's table before doing that in UV's
- Renamed and better documented the ultravisor header files. Also added
all possible return codes for each ucall
- Updated the commit message that introduces the MSR_S bit
- Moved ultravisor.c and ucall.S to arch/powerpc/kernel
- Changed ucall.S to not save CR
- Rebased
- Changed the patches order
- Updated several commit messages
- Added FW_FEATURE_ULTRAVISOR to enable use of firmware_has_feature()
- Renamed CONFIG_PPC_KVM_UV to CONFIG_PPC_UV and used it to ifdef the ucall
handler and the code that populates the powerpc_firmware_features for
ultravisor
- Exported the ucall symbol. KVM may be built as module.
- Restricted LDBAR access if the ultravisor firmware is available
- Dropped patches:
"[PATCH 06/13] KVM: PPC: Ultravisor: UV_RESTRICTED_SPR_WRITE ucall"
"[PATCH 07/13] KVM: PPC: Ultravisor: UV_RESTRICTED_SPR_READ ucall"
"[PATCH 08/13] KVM: PPC: Ultravisor: fix mtspr and mfspr"
- Squashed patches:
"[PATCH 09/13] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM"
"[PATCH 13/13] KVM: PPC: UV: Have fast_guest_return check secure_guest"
Anshuman Khandual (1):
KVM: PPC: Ultravisor: Add PPC_UV config option
Claudio Carvalho (2):
powerpc: Introduce FW_FEATURE_ULTRAVISOR
KVM: PPC: Ultravisor: Restrict LDBAR access
Michael Anderson (2):
KVM: PPC: Ultravisor: Use UV_WRITE_PATE ucall to register a PATE
KVM: PPC: Ultravisor: Check for MSR_S during hv_reset_msr
Ram Pai (2):
KVM: PPC: Ultravisor: Add generic ultravisor call handler
KVM: PPC: Ultravisor: Restrict flush of the partition tlb cache
Sukadev Bhattiprolu (2):
KVM: PPC: Ultravisor: Introduce the MSR_S bit
KVM: PPC: Ultravisor: Enter a secure guest
arch/powerpc/Kconfig | 20 +++++++
arch/powerpc/include/asm/firmware.h | 5 +-
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/include/asm/reg.h | 3 ++
arch/powerpc/include/asm/ultravisor-api.h | 24 +++++++++
arch/powerpc/include/asm/ultravisor.h | 49 +++++++++++++++++
arch/powerpc/kernel/Makefile | 1 +
arch/powerpc/kernel/asm-offsets.c | 1 +
arch/powerpc/kernel/prom.c | 6 +++
arch/powerpc/kernel/ucall.S | 31 +++++++++++
arch/powerpc/kernel/ultravisor.c | 28 ++++++++++
arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 +
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 39 +++++++++++---
arch/powerpc/mm/book3s64/hash_utils.c | 3 +-
arch/powerpc/mm/book3s64/pgtable.c | 65 +++++++++++++++++------
arch/powerpc/mm/book3s64/radix_pgtable.c | 9 ++--
arch/powerpc/platforms/powernv/idle.c | 6 ++-
arch/powerpc/platforms/powernv/opal-imc.c | 7 +++
18 files changed, 269 insertions(+), 30 deletions(-)
create mode 100644 arch/powerpc/include/asm/ultravisor-api.h
create mode 100644 arch/powerpc/include/asm/ultravisor.h
create mode 100644 arch/powerpc/kernel/ucall.S
create mode 100644 arch/powerpc/kernel/ultravisor.c
--
2.20.1
From: Sukadev Bhattiprolu <redacted>
The ultravisor processor mode is introduced in POWER platforms that
supports the Protected Execution Facility (PEF). Ultravisor is higher
privileged than hypervisor mode.
In PEF enabled platforms, the MSR_S bit is used to indicate if the
thread is in secure state. With the MSR_S bit, the privilege state of
the thread is now determined by MSR_S, MSR_HV and MSR_PR, as follows:
S HV PR
-----------------------
0 x 1 problem
1 0 1 problem
x x 0 privileged
x 1 0 hypervisor
1 1 0 ultravisor
1 1 1 reserved
The hypervisor doesn't (and can't) run with the MSR_S bit set, but a
secure guest and the ultravisor firmware do.
Signed-off-by: Sukadev Bhattiprolu <redacted>
Signed-off-by: Ram Pai <redacted>
[Update the commit message]
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/include/asm/reg.h | 3 +++
1 file changed, 3 insertions(+)
@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM---help---Supportuser-modeTransactionalMemoryonPOWERPC.+configPPC_UV+bool"Ultravisor support"+depends onKVM_BOOK3S_HV_POSSIBLE+selectHMM_MIRROR+selectHMM+selectZONE_DEVICE+selectMIGRATE_VMA_HELPER+selectDEV_PAGEMAP_OPS+selectDEVICE_PRIVATE+selectMEMORY_HOTPLUG+selectMEMORY_HOTREMOVE+defaultn+help+ThisoptionparavirtualizesthekerneltoruninPOWERplatformsthat+supportstheProtectedExecutionFacility(PEF).Insuchplatforms,+theultravisorfirmwarerunsataprivilegelevelabovethe+hypervisor.++Ifunsure,say"N".+configLD_HEAD_STUB_CATCHbool"Reserve 256 bytes to cope with linker stubs in HEAD text"ifEXPERTdepends onPPC64
@@ -713,6 +714,11 @@ void __init early_init_devtree(void *params)of_scan_flat_dt(early_init_dt_scan_fw_dump,NULL);#endif+#if defined(CONFIG_PPC_UV)+/* Scan tree for ultravisor feature */+of_scan_flat_dt(early_init_dt_scan_ultravisor,NULL);+#endif+/* Retrieve various informations from the /chosen node of the*device-tree,includingtheplatformtype,initrdlocationand*size,TCEreserve,andmore...
From: Ram Pai <redacted>
Add the ucall() function, which can be used to make ultravisor calls
with varied number of in and out arguments. Ultravisor calls can be made
from the host or guests.
This copies the implementation of plpar_hcall().
Signed-off-by: Ram Pai <redacted>
[Change ucall.S to not save CR, rename and move the headers, build
ucall.S if CONFIG_PPC_UV set, and add some comments in the code]
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/include/asm/ultravisor-api.h | 20 +++++++++++++++
arch/powerpc/include/asm/ultravisor.h | 20 +++++++++++++++
arch/powerpc/kernel/Makefile | 2 +-
arch/powerpc/kernel/ucall.S | 31 +++++++++++++++++++++++
arch/powerpc/kernel/ultravisor.c | 4 +++
5 files changed, 76 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/include/asm/ultravisor-api.h
create mode 100644 arch/powerpc/kernel/ucall.S
From: Michael Anderson <redacted>
When running under an ultravisor, the ultravisor controls the real
partition table and has it in secure memory where the hypervisor can't
access it, and therefore we (the HV) have to do a ucall whenever we want
to update an entry.
The HV still keeps a copy of its view of the partition table in normal
memory so that the nest MMU can access it.
Both partition tables will have PATE entries for HV and normal virtual
machines.
Suggested-by: Ryan Grimm <redacted>
Signed-off-by: Michael Anderson <redacted>
Signed-off-by: Madhavan Srinivasan <redacted>
Signed-off-by: Ram Pai <redacted>
[Write the pate in HV's table before doing that in UV's]
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/include/asm/ultravisor-api.h | 5 +++-
arch/powerpc/include/asm/ultravisor.h | 14 ++++++++++
arch/powerpc/mm/book3s64/hash_utils.c | 3 +-
arch/powerpc/mm/book3s64/pgtable.c | 34 +++++++++++++++++++++--
arch/powerpc/mm/book3s64/radix_pgtable.c | 9 ++++--
5 files changed, 57 insertions(+), 8 deletions(-)
@@ -238,6 +253,19 @@ void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,/* do we need fixup here ?*/asmvolatile("eieio; tlbsync; ptesync":::"memory");}++voidmmu_partition_table_set_entry(unsignedintlpid,unsignedlongdw0,+unsignedlongdw1)+{+__mmu_partition_table_set_entry(lpid,dw0,dw1);++if(firmware_has_feature(FW_FEATURE_ULTRAVISOR)){+uv_register_pate(lpid,dw0,dw1);+pr_info("PATE registered by ultravisor: dw0 = 0x%lx, dw1 = 0x%lx\n",+dw0,dw1);+}+}+EXPORT_SYMBOL_GPL(mmu_partition_table_set_entry);staticpmd_t*get_pmd_from_cache(structmm_struct*mm)
From: Ram Pai <redacted>
Ultravisor is responsible for flushing the tlb cache, since it manages
the PATE entries. Hence skip tlb flush, if the ultravisor firmware is
available.
Signed-off-by: Ram Pai <redacted>
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++-------------
1 file changed, 19 insertions(+), 14 deletions(-)
@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr(ptcr);}+staticvoidflush_partition(unsignedintlpid,unsignedlongdw0)+{+if(dw0&PATB_HR){+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+asmvolatile(PPC_TLBIE_5(%0,%1,2,1,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,1);+}else{+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,0)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,0);+}+/* do we need fixup here ?*/+asmvolatile("eieio; tlbsync; ptesync":::"memory");+}+staticvoid__mmu_partition_table_set_entry(unsignedintlpid,unsignedlongdw0,unsignedlongdw1)
@@ -238,20 +255,8 @@ static void __mmu_partition_table_set_entry(unsigned int lpid,*Thetypeofflush(hashorradix)dependsonwhattheprevious*useofthispartitionIDwas,notthenewuse.*/-asmvolatile("ptesync":::"memory");-if(old&PATB_HR){-asmvolatile(PPC_TLBIE_5(%0,%1,2,0,1)::-"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));-asmvolatile(PPC_TLBIE_5(%0,%1,2,1,1)::-"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));-trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,1);-}else{-asmvolatile(PPC_TLBIE_5(%0,%1,2,0,0)::-"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));-trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,0);-}-/* do we need fixup here ?*/-asmvolatile("eieio; tlbsync; ptesync":::"memory");+if(!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+flush_partition(lpid,old);}voidmmu_partition_table_set_entry(unsignedintlpid,unsignedlongdw0,
From: Sukadev Bhattiprolu <redacted>
To enter a secure guest, we have to go through the ultravisor, therefore
we do a ucall when we are entering a secure guest.
This change is needed for any sort of entry to the secure guest from the
hypervisor, whether it is a return from an hcall, a return from a
hypervisor interrupt, or the first time that a secure guest vCPU is run.
If we are returning from an hcall, the results are already in the
appropriate registers (R3:12), except for R6,7, which need to be
restored before doing the ucall (UV_RETURN).
Have fast_guest_return check the kvm_arch.secure_guest field so that a
new CPU enters UV when started (in response to a RTAS start-cpu call).
Thanks to input from Paul Mackerras, Ram Pai and Mike Anderson.
Signed-off-by: Sukadev Bhattiprolu <redacted>
[Pass SRR1 in r11 for UV_RETURN, fix kvmppc_msr_interrupt to preserve
the MSR_S bit]
Signed-off-by: Paul Mackerras <redacted>
[Fix UV_RETURN token number and arch.secure_guest check]
Signed-off-by: Ram Pai <redacted>
[Update commit message and ret_to_ultra comment]
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/include/asm/ultravisor-api.h | 1 +
arch/powerpc/kernel/asm-offsets.c | 1 +
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 37 +++++++++++++++++++----
4 files changed, 34 insertions(+), 6 deletions(-)
When the ultravisor firmware is available, it takes control over the
LDBAR register. In this case, thread-imc updates and save/restore
operations on the LDBAR register are handled by ultravisor.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Ram Pai <redacted>
---
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++
arch/powerpc/platforms/powernv/idle.c | 6 ++++--
arch/powerpc/platforms/powernv/opal-imc.c | 7 +++++++
3 files changed, 13 insertions(+), 2 deletions(-)
@@ -679,7 +679,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)sprs.ptcr=mfspr(SPRN_PTCR);sprs.rpr=mfspr(SPRN_RPR);sprs.tscr=mfspr(SPRN_TSCR);-sprs.ldbar=mfspr(SPRN_LDBAR);+if(!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+sprs.ldbar=mfspr(SPRN_LDBAR);sprs_saved=true;
@@ -762,7 +763,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)mtspr(SPRN_PTCR,sprs.ptcr);mtspr(SPRN_RPR,sprs.rpr);mtspr(SPRN_TSCR,sprs.tscr);-mtspr(SPRN_LDBAR,sprs.ldbar);+if(!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+mtspr(SPRN_LDBAR,sprs.ldbar);if(pls>=pnv_first_tb_loss_level){/* TB loss */
From: Michael Anderson <redacted>
- Check for MSR_S so that kvmppc_set_msr will include. Prior to this
change return to guest would not have the S bit set.
- Patch based on comment from Paul Mackerras [off-list ref]
Signed-off-by: Michael Anderson <redacted>
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 +
1 file changed, 1 insertion(+)
From: Ram Pai <redacted>
Ultravisor is responsible for flushing the tlb cache, since it manages
the PATE entries. Hence skip tlb flush, if the ultravisor firmware is
available.
Signed-off-by: Ram Pai <redacted>
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++-------------
1 file changed, 19 insertions(+), 14 deletions(-)
@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr(ptcr);}+staticvoidflush_partition(unsignedintlpid,unsignedlongdw0)+{+if(dw0&PATB_HR){+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+asmvolatile(PPC_TLBIE_5(%0,%1,2,1,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,1);+}else{+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,0)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,0);+}+/* do we need fixup here ?*/+asmvolatile("eieio; tlbsync; ptesync":::"memory");+}+
checkpatch.pl seems to complain:
ERROR: need consistent spacing around '%' (ctx:WxV)
#125: FILE: arch/powerpc/mm/book3s64/pgtable.c:230:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV)
#127: FILE: arch/powerpc/mm/book3s64/pgtable.c:232:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV)
#131: FILE: arch/powerpc/mm/book3s64/pgtable.c:236:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : :
^
quoted hunk
static void __mmu_partition_table_set_entry(unsigned int lpid,
unsigned long dw0,
unsigned long dw1)
@@ -238,20 +255,8 @@ static void __mmu_partition_table_set_entry(unsigned int lpid, * The type of flush (hash or radix) depends on what the previous * use of this partition ID was, not the new use. */- asm volatile("ptesync" : : : "memory");- if (old & PATB_HR) {- asm volatile(PPC_TLBIE_5(%0,%1,2,0,1) : :- "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));- asm volatile(PPC_TLBIE_5(%0,%1,2,1,1) : :- "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));- trace_tlbie(lpid, 0, TLBIEL_INVAL_SET_LPID, lpid, 2, 0, 1);- } else {- asm volatile(PPC_TLBIE_5(%0,%1,2,0,0) : :- "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));- trace_tlbie(lpid, 0, TLBIEL_INVAL_SET_LPID, lpid, 2, 0, 0);- }- /* do we need fixup here ?*/- asm volatile("eieio; tlbsync; ptesync" : : : "memory");+ if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+ flush_partition(lpid, old); } void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,--
From: Paul Mackerras <hidden> Date: 2019-06-06 21:56:51
On Thu, Jun 06, 2019 at 04:39:04PM -0300, Murilo Opsfelder Araújo wrote:
Claudio Carvalho [off-list ref] writes:
quoted
From: Ram Pai <redacted>
Ultravisor is responsible for flushing the tlb cache, since it manages
the PATE entries. Hence skip tlb flush, if the ultravisor firmware is
available.
Signed-off-by: Ram Pai <redacted>
Signed-off-by: Claudio Carvalho <redacted>
---
arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++-------------
1 file changed, 19 insertions(+), 14 deletions(-)
@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr(ptcr);}+staticvoidflush_partition(unsignedintlpid,unsignedlongdw0)+{+if(dw0&PATB_HR){+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+asmvolatile(PPC_TLBIE_5(%0,%1,2,1,1)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,1);+}else{+asmvolatile(PPC_TLBIE_5(%0,%1,2,0,0)::+"r"(TLBIEL_INVAL_SET_LPID),"r"(lpid));+trace_tlbie(lpid,0,TLBIEL_INVAL_SET_LPID,lpid,2,0,0);+}+/* do we need fixup here ?*/+asmvolatile("eieio; tlbsync; ptesync":::"memory");+}+
checkpatch.pl seems to complain:
ERROR: need consistent spacing around '%' (ctx:WxV)
#125: FILE: arch/powerpc/mm/book3s64/pgtable.c:230:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV)
#127: FILE: arch/powerpc/mm/book3s64/pgtable.c:232:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV)
#131: FILE: arch/powerpc/mm/book3s64/pgtable.c:236:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : :
^
When the ultravisor firmware is available, it takes control over the
LDBAR register. In this case, thread-imc updates and save/restore
operations on the LDBAR register are handled by ultravisor.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Ram Pai <redacted>
---
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++
arch/powerpc/platforms/powernv/idle.c | 6 ++++--
arch/powerpc/platforms/powernv/opal-imc.c | 7 +++++++
3 files changed, 13 insertions(+), 2 deletions(-)
@@ -679,7 +679,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)sprs.ptcr=mfspr(SPRN_PTCR);sprs.rpr=mfspr(SPRN_RPR);sprs.tscr=mfspr(SPRN_TSCR);-sprs.ldbar=mfspr(SPRN_LDBAR);+if(!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+sprs.ldbar=mfspr(SPRN_LDBAR);sprs_saved=true;
@@ -762,7 +763,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)mtspr(SPRN_PTCR,sprs.ptcr);mtspr(SPRN_RPR,sprs.rpr);mtspr(SPRN_TSCR,sprs.tscr);-mtspr(SPRN_LDBAR,sprs.ldbar);+if(!firmware_has_feature(FW_FEATURE_ULTRAVISOR))+mtspr(SPRN_LDBAR,sprs.ldbar);if(pls>=pnv_first_tb_loss_level){/* TB loss */
@@ -254,6 +254,13 @@ static int opal_imc_counters_probe(struct platform_device *pdev)boolcore_imc_reg=false,thread_imc_reg=false;u32type;+/*+*WhentheUltravisorisenabled,itisresponsibleforthread-imc+*updates+*/
Would prefer the comment to be "Disable IMC devices, when Ultravisor is
enabled"
Rest looks good.
Acked-by: Madhavan Srinivasan <redacted>
+ if (firmware_has_feature(FW_FEATURE_ULTRAVISOR))
+ return -EACCES;
+
/*
* Check whether this is kdump kernel. If yes, force the engines to
* stop and return.
When the ultravisor firmware is available, it takes control over the
LDBAR register. In this case, thread-imc updates and save/restore
operations on the LDBAR register are handled by ultravisor.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Ram Pai <redacted>
---
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++
arch/powerpc/platforms/powernv/idle.c | 6 ++++--
arch/powerpc/platforms/powernv/opal-imc.c | 7 +++++++
3 files changed, 13 insertions(+), 2 deletions(-)
@@ -254,6 +254,13 @@ static int opal_imc_counters_probe(struct
platform_device *pdev)
bool core_imc_reg = false, thread_imc_reg = false;
u32 type;
+ /*
+ * When the Ultravisor is enabled, it is responsible for thread-imc
+ * updates
+ */
Would prefer the comment to be "Disable IMC devices, when Ultravisor is
enabled"
Rest looks good.
Acked-by: Madhavan Srinivasan <redacted>
Thanks Maddy. I applied that change to the next version.
Claudio
quoted
+ if (firmware_has_feature(FW_FEATURE_ULTRAVISOR))
+ return -EACCES;
+
/*
* Check whether this is kdump kernel. If yes, force the engines to
* stop and return.
@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM---help---Supportuser-modeTransactionalMemoryonPOWERPC.+configPPC_UV+bool"Ultravisor support"+depends onKVM_BOOK3S_HV_POSSIBLE+selectHMM_MIRROR+selectHMM+selectZONE_DEVICE+selectMIGRATE_VMA_HELPER+selectDEV_PAGEMAP_OPS+selectDEVICE_PRIVATE+selectMEMORY_HOTPLUG+selectMEMORY_HOTREMOVE+defaultn+help+ThisoptionparavirtualizesthekerneltoruninPOWERplatformsthat+supportstheProtectedExecutionFacility(PEF).Insuchplatforms,+theultravisorfirmwarerunsataprivilegelevelabovethe+hypervisor.++Ifunsure,say"N".+configLD_HEAD_STUB_CATCHbool"Reserve 256 bytes to cope with linker stubs in HEAD text"ifEXPERTdepends onPPC64
Maybe this patch should be the last of the series, as it may cause some
bisect trouble to have this option enabled while missing some of the
patches.
@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM---help---Supportuser-modeTransactionalMemoryonPOWERPC.+configPPC_UV+bool"Ultravisor support"+depends onKVM_BOOK3S_HV_POSSIBLE+selectHMM_MIRROR+selectHMM+selectZONE_DEVICE+selectMIGRATE_VMA_HELPER+selectDEV_PAGEMAP_OPS+selectDEVICE_PRIVATE+selectMEMORY_HOTPLUG+selectMEMORY_HOTREMOVE+defaultn+help+ThisoptionparavirtualizesthekerneltoruninPOWERplatformsthat+supportstheProtectedExecutionFacility(PEF).Insuchplatforms,+theultravisorfirmwarerunsataprivilegelevelabovethe+hypervisor.++Ifunsure,say"N".+configLD_HEAD_STUB_CATCHbool"Reserve 256 bytes to cope with linker stubs in HEAD text"ifEXPERTdepends onPPC64
Maybe this patch should be the last of the series, as it may cause some
bisect trouble to have this option enabled while missing some of the
patches.
Thanks Leonardo. I changed that for the next version.
Claudio
From: Paul Mackerras <hidden> Date: 2019-06-15 07:39:23
On Thu, Jun 06, 2019 at 02:36:08PM -0300, Claudio Carvalho wrote:
This feature tells if the ultravisor firmware is available to handle
ucalls.
Everything in this patch that depends on CONFIG_PPC_UV should just
depend on CONFIG_PPC_POWERNV instead. The reason is that every host
kernel needs to be able to do the ultracall to set partition table
entry 0, in case it ends up being run on a machine with an ultravisor.
Otherwise we will have the situation where a host kernel may crash
early in boot just because the machine it's booted on happens to have
an ultravisor running. The crash will be a particularly nasty one
because it will happen before we have probed the machine type and
initialized the console; therefore it will just look like the machine
hangs for no discernable reason.
We also need to think about how to provide a way for petitboot to know
whether the kernel it is booting knows how to do a ucall to set its
partition table entry. One suggestion would be to modify
vmlinux.lds.S to add a new PT_NOTE entry in the program header of the
binary with (say) a 64-bit doubleword which is a bitmap indicating
capabilities of the binary. We would define the first bit as
indicating that the kernel knows how to run under an ultravisor.
When running under an ultravisor, petitboot could then look for the
PT_NOTE and the ultravisor-capable bit in it, and if the PT_NOTE is
not there or the bit is zero, put up a dialog warning the user that
the kernel will probably crash early in boot, and asking for explicit
confirmation that the user wants to proceed.
Paul.
From: Paul Mackerras <hidden> Date: 2019-06-15 07:41:18
On Thu, Jun 06, 2019 at 02:36:09PM -0300, Claudio Carvalho wrote:
From: Ram Pai <redacted>
Add the ucall() function, which can be used to make ultravisor calls
with varied number of in and out arguments. Ultravisor calls can be made
from the host or guests.
This copies the implementation of plpar_hcall().
Again, we will want all of this on every powernv-capable kernel, since
they will all need to do UV_WRITE_PATE, even if they have no other
support for the ultravisor.
Paul.
From: Paul Mackerras <hidden> Date: 2019-06-15 07:43:17
On Thu, Jun 06, 2019 at 02:36:10PM -0300, Claudio Carvalho wrote:
From: Michael Anderson <redacted>
When running under an ultravisor, the ultravisor controls the real
partition table and has it in secure memory where the hypervisor can't
access it, and therefore we (the HV) have to do a ucall whenever we want
to update an entry.
The HV still keeps a copy of its view of the partition table in normal
memory so that the nest MMU can access it.
Both partition tables will have PATE entries for HV and normal virtual
machines.
As discussed before, all of this should depend only on
CONFIG_PPC_POWERNV.
Paul.
From: Paul Mackerras <hidden> Date: 2019-06-15 07:49:41
On Thu, Jun 06, 2019 at 02:36:12PM -0300, Claudio Carvalho wrote:
When the ultravisor firmware is available, it takes control over the
LDBAR register. In this case, thread-imc updates and save/restore
operations on the LDBAR register are handled by ultravisor.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Ram Pai <redacted>
Acked-by: Paul Mackerras <redacted>
Just a note on the signed-off-by: it's a bit weird to have Ram's
signoff when he is neither the author nor the sender of the patch.
The author is assumed to be Claudio; if that is not correct then the
patch should have a From: line at the start telling us who the author
is, and ideally that person should have a signoff line before
Claudio's signoff as the sender of the patch.
Paul.
From: Paul Mackerras <hidden> Date: 2019-06-15 07:51:25
On Thu, Jun 06, 2019 at 02:36:14PM -0300, Claudio Carvalho wrote:
From: Michael Anderson <redacted>
- Check for MSR_S so that kvmppc_set_msr will include. Prior to this
Will include what? "it" maybe?
change return to guest would not have the S bit set.
- Patch based on comment from Paul Mackerras [off-list ref]
Signed-off-by: Michael Anderson <redacted>
Signed-off-by: Claudio Carvalho <redacted>
Acked-by: Paul Mackerras <redacted>
but you should reword the commit message fix that first sentence.
Paul.
From: Paul Mackerras <hidden> Date: 2019-06-15 07:53:13
On Thu, Jun 06, 2019 at 02:36:13PM -0300, Claudio Carvalho wrote:
From: Sukadev Bhattiprolu <redacted>
To enter a secure guest, we have to go through the ultravisor, therefore
we do a ucall when we are entering a secure guest.
This change is needed for any sort of entry to the secure guest from the
hypervisor, whether it is a return from an hcall, a return from a
hypervisor interrupt, or the first time that a secure guest vCPU is run.
If we are returning from an hcall, the results are already in the
appropriate registers (R3:12), except for R6,7, which need to be
restored before doing the ucall (UV_RETURN).
Have fast_guest_return check the kvm_arch.secure_guest field so that a
new CPU enters UV when started (in response to a RTAS start-cpu call).
Thanks to input from Paul Mackerras, Ram Pai and Mike Anderson.
Signed-off-by: Sukadev Bhattiprolu <redacted>
[Pass SRR1 in r11 for UV_RETURN, fix kvmppc_msr_interrupt to preserve
the MSR_S bit]
Signed-off-by: Paul Mackerras <redacted>
[Fix UV_RETURN token number and arch.secure_guest check]
Signed-off-by: Ram Pai <redacted>
[Update commit message and ret_to_ultra comment]
Signed-off-by: Claudio Carvalho <redacted>
On Sat, Jun 15, 2019 at 05:43:34PM +1000, Paul Mackerras wrote:
On Thu, Jun 06, 2019 at 02:36:12PM -0300, Claudio Carvalho wrote:
quoted
When the ultravisor firmware is available, it takes control over the
LDBAR register. In this case, thread-imc updates and save/restore
operations on the LDBAR register are handled by ultravisor.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Ram Pai <redacted>
Acked-by: Paul Mackerras <redacted>
Just a note on the signed-off-by: it's a bit weird to have Ram's
signoff when he is neither the author nor the sender of the patch.
The author is assumed to be Claudio; if that is not correct then the
patch should have a From: line at the start telling us who the author
is, and ideally that person should have a signoff line before
Claudio's signoff as the sender of the patch.
Ryan originally wrote this patch, which I than modified,
before Claudio further modified it to its current form.
So I think the author should be Ryan.
RP
From: Paul Mackerras <hidden> Date: 2019-06-17 02:08:58
On Thu, Jun 06, 2019 at 02:36:09PM -0300, Claudio Carvalho wrote:
From: Ram Pai <redacted>
Add the ucall() function, which can be used to make ultravisor calls
with varied number of in and out arguments. Ultravisor calls can be made
from the host or guests.
This copies the implementation of plpar_hcall().
One point which I missed when I looked at this patch previously is
that the ABI that we're defining here is different from the hcall ABI
in that we are putting the ucall number in r0, whereas hcalls have the
hcall number in r3. That makes ucalls more like syscalls, which have
the syscall number in r0. So that last sentence quoted above is
somewhat misleading.
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall
instruction becomes a hcall (that is, sc 2 is executed as if it was
sc 1). In that case, the first argument to the ucall will be
interpreted as the hcall number. Mostly that will happen not to be a
valid hcall number, but sometimes it might unavoidably be a valid but
unintended hcall number.
I think that will make it difficult to get ucalls to fail gracefully
in the case where SMF/PEF is disabled. It seems like the assignment
of ucall numbers was made so that they wouldn't overlap with valid
hcall numbers; presumably that was so that we could tell when an hcall
was actually intended to be a ucall. However, using a different GPR
to pass the ucall number defeats that.
I realize that there is ultravisor code in development that takes the
ucall number in r0, and also that having the ucall number in r3 would
possibly make life more difficult for the place where we call
UV_RETURN in assembler code. Nevertheless, perhaps we should consider
changing the ABI to be like the hcall ABI before everything gets set
in concrete.
Paul.
On Mon, Jun 17, 2019 at 12:06:32PM +1000, Paul Mackerras wrote:
On Thu, Jun 06, 2019 at 02:36:09PM -0300, Claudio Carvalho wrote:
quoted
From: Ram Pai <redacted>
Add the ucall() function, which can be used to make ultravisor calls
with varied number of in and out arguments. Ultravisor calls can be made
from the host or guests.
This copies the implementation of plpar_hcall().
One point which I missed when I looked at this patch previously is
that the ABI that we're defining here is different from the hcall ABI
in that we are putting the ucall number in r0, whereas hcalls have the
hcall number in r3. That makes ucalls more like syscalls, which have
the syscall number in r0. So that last sentence quoted above is
somewhat misleading.
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall
instruction becomes a hcall (that is, sc 2 is executed as if it was
sc 1). In that case, the first argument to the ucall will be
interpreted as the hcall number. Mostly that will happen not to be a
valid hcall number, but sometimes it might unavoidably be a valid but
unintended hcall number.
I think that will make it difficult to get ucalls to fail gracefully
in the case where SMF/PEF is disabled. It seems like the assignment
of ucall numbers was made so that they wouldn't overlap with valid
hcall numbers; presumably that was so that we could tell when an hcall
was actually intended to be a ucall. However, using a different GPR
to pass the ucall number defeats that.
Right this is a valid point. Glad that you caught it, otherwise it would
have become a difficult to fix it in the future.
I realize that there is ultravisor code in development that takes the
ucall number in r0, and also that having the ucall number in r3 would
possibly make life more difficult for the place where we call
UV_RETURN in assembler code.
Its called from one place in the hypervisor, and the changes look
simple.
- LOAD_REG_IMMEDIATE(r0, UV_RETURN)
+ LOAD_REG_IMMEDIATE(r3, UV_RETURN)
ld r7, VCPU_GPR(R7)(r4)
ld r6, VCPU_GPR(R6)(r4)
ld r4, VCPU_GPR(R4)(r4)
What am i missing?
Nevertheless, perhaps we should consider
changing the ABI to be like the hcall ABI before everything gets set
in concrete.
Hi Paul,
On Mon, Jun 17, 2019 at 12:06:32PM +1000, Paul Mackerras wrote:
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall
instruction becomes a hcall (that is, sc 2 is executed as if it was
sc 1). In that case, the first argument to the ucall will be
interpreted as the hcall number. Mostly that will happen not to be a
valid hcall number, but sometimes it might unavoidably be a valid but
unintended hcall number.
Shouldn't a caller of the ultravisor *know* that it is talking to the
ultravisor in the first place? And not to the hypervisor.
Segher
On Tue, Jun 18, 2019 at 06:47:01AM -0500, Segher Boessenkool wrote:
Hi Paul,
On Mon, Jun 17, 2019 at 12:06:32PM +1000, Paul Mackerras wrote:
quoted
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall
instruction becomes a hcall (that is, sc 2 is executed as if it was
sc 1). In that case, the first argument to the ucall will be
interpreted as the hcall number. Mostly that will happen not to be a
valid hcall number, but sometimes it might unavoidably be a valid but
unintended hcall number.
Shouldn't a caller of the ultravisor *know* that it is talking to the
ultravisor in the first place? And not to the hypervisor.
It may or may not. But if it knows and still decides to make the ucall,
the hypervisor must gracefully handle it.
We can't control who makes a ucall. A normal process within the VM could
make a ucall too. Or a normal process running on top of the hypervisor
could make a ucall.
RP
On Thu, Jun 06, 2019 at 02:36:08PM -0300, Claudio Carvalho wrote:
quoted
This feature tells if the ultravisor firmware is available to handle
ucalls.
Everything in this patch that depends on CONFIG_PPC_UV should just
depend on CONFIG_PPC_POWERNV instead. The reason is that every host
kernel needs to be able to do the ultracall to set partition table
entry 0, in case it ends up being run on a machine with an ultravisor.
Otherwise we will have the situation where a host kernel may crash
early in boot just because the machine it's booted on happens to have
an ultravisor running. The crash will be a particularly nasty one
because it will happen before we have probed the machine type and
initialized the console; therefore it will just look like the machine
hangs for no discernable reason.
We also need to think about how to provide a way for petitboot to know
whether the kernel it is booting knows how to do a ucall to set its
partition table entry. One suggestion would be to modify
vmlinux.lds.S to add a new PT_NOTE entry in the program header of the
binary with (say) a 64-bit doubleword which is a bitmap indicating
capabilities of the binary. We would define the first bit as
indicating that the kernel knows how to run under an ultravisor.
When running under an ultravisor, petitboot could then look for the
PT_NOTE and the ultravisor-capable bit in it, and if the PT_NOTE is
not there or the bit is zero, put up a dialog warning the user that
the kernel will probably crash early in boot, and asking for explicit
confirmation that the user wants to proceed.
I just posted a separated RFC patch for the ELF note.
Thanks, Claudio.