Hello,
There are three new patches from Claudio's "kvmppc: Paravirtualize KVM to
support ultravisor" series. Two of them are borrowed (meaning that he will
submit them with his series as well) and included here so that this series
can stand on its own. They are the one introducing ucall_norets() (patch 1)
and the one adding documentation on the Ultravisor (patch 15).
The third new patch is migrating from the kvmppc series to this one because
it won't be needed there anymore but is needed here. This is the patch
introducing the MSR_S bit (patch 6).
There are also changes to a couple of patches stemming from review comments
by Alexey Kardashevskiy and Segher Boessenkool (thanks!). The changelog
below has the details.
This series applies on top of v4 of the <asm/mem_encrypt.h> cleanup series:
https://lore.kernel.org/linuxppc-dev/20190806044919.10622-1-bauerman@linux.ibm.com/
Everything is available in branch ultravisor-secure-vm at this repo:
https://github.com/bauermann/linux.git
Original cover letter below, and changelog at the bottom:
This series enables Secure Virtual Machines (SVMs) on powerpc. SVMs use the
Protected Execution Facility (PEF) and request to be migrated to secure
memory during prom_init() so by default all of their memory is inaccessible
to the hypervisor. There is an Ultravisor call that the VM can use to
request certain pages to be made accessible to (or shared with) the
hypervisor.
The objective of these patches is to have the guest perform this request
for buffers that need to be accessed by the hypervisor such as the LPPACAs,
the SWIOTLB memory and the Debug Trace Log.
Patch 3 ("powerpc: Add support for adding an ESM blob to the zImage
wrapper") is posted as RFC because we are still finalizing the details on
how the ESM blob will be passed along with the kernel. All other patches are
(hopefully) in upstreamable shape and don't depend on this patch.
Unfortunately this series still doesn't enable the use of virtio devices in
the secure guest. This support depends on a discussion that is currently
ongoing with the virtio community:
https://lore.kernel.org/linuxppc-dev/87womn8inf.fsf@morokweng.localdomain/
I was able to test it using Claudio's patches in the host kernel, booting
normally using an initramfs for the root filesystem.
This is the command used to start up the guest with QEMU 4.0:
qemu-system-ppc64 \
-nodefaults \
-cpu host \
-machine pseries,accel=kvm,kvm-type=HV,cap-htm=off,cap-cfpc=broken,cap-sbbc=broken,cap-ibs=broken \
-display none \
-serial mon:stdio \
-smp 1 \
-m 4G \
-kernel /root/bauermann/vmlinux \
-initrd /root/bauermann/fs_small.cpio \
-append 'debug'
Changelog
Since v2:
- Patch "powerpc/kernel: Add ucall_norets() ultravisor call handler"
- Borrowed unchanged from Claudio's "kvmppc: Paravirtualize KVM to support
ultravisor" series.
- Patch "powerpc/prom_init: Add the ESM call to prom_init"
- Briefly mention in the commit message why we pass the kernel base address
and FDT to the Enter Secure Mode ultracall. Suggested by Alexey
Kardashevskiy.
- Use enter_secure_mode() version provided by Segher Boessenkool.
- Patch "powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE"
- Use ucall_norets() which doesn't need to be passed a return buffer.
Suggested by Alexey Kardashevskiy.
- Patch "powerpc: Introduce the MSR_S bit"
- Moved from Claudio's "kvmppc: Paravirtualize KVM to support ultravisor"
series to this series.
- Patch "Documentation/powerpc: Ultravisor API"
- New patch from Sukadev Bhattiprolu. Will also appear on Claudio's
kvmppc series.
Since v1:
- Patch "powerpc/pseries: Introduce option to build secure virtual machines"
- Dropped redundant "default n" from CONFIG_PPC_SVM. Suggested by Christoph
Hellwig.
- Patch "powerpc: Add support for adding an ESM blob to the zImage wrapper"
- Renamed prom_rtas_os_term_hcall() to prom_rtas_hcall(). Suggested by Alexey
Kardashevskiy.
- In prom_rtas_hcall(), changed prom_printf() calls to prom_debug(), and
use H_RTAS constant instead of raw value.
- Changed enter_secure_mode() to new ABI passing ucall number in r3.
Also changed it to accept kbase argument instead of ESM blob address.
- Changed setup_secure_guest() to only make the ESM ultracall if svm=1 was
passed on the kernel command line.
- Patch "powerpc/pseries/svm: Unshare all pages before kexecing a new kernel"
- New patch from Ram Pai.
- Patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"
- No need to define sme_me_mask, sme_active() and sev_active() anymore.
- Add definitions for mem_encrypt_active() and force_dma_unencrypted().
- Select ARCH_HAS_FORCE_DMA_UNENCRYPTED in CONFIG_PPC_SVM.
Anshuman Khandual (3):
powerpc/pseries/svm: Use shared memory for LPPACA structures
powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL)
powerpc/pseries/svm: Force SWIOTLB for secure guests
Benjamin Herrenschmidt (1):
powerpc: Add support for adding an ESM blob to the zImage wrapper
Claudio Carvalho (1):
powerpc/kernel: Add ucall_norets() ultravisor call handler
Ram Pai (3):
powerpc/prom_init: Add the ESM call to prom_init
powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE
powerpc/pseries/svm: Unshare all pages before kexecing a new kernel
Ryan Grimm (2):
powerpc/pseries/svm: Export guest SVM status to user space via sysfs
powerpc/configs: Enable secure guest support in pseries and ppc64
defconfigs
Sukadev Bhattiprolu (3):
powerpc: Introduce the MSR_S bit
powerpc/pseries/svm: Disable doorbells in SVM guests
Documentation/powerpc: Ultravisor API
Thiago Jung Bauermann (3):
powerpc/pseries: Introduce option to build secure virtual machines
powerpc/pseries: Add and use LPPACA_SIZE constant
powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests
.../admin-guide/kernel-parameters.txt | 5 +
Documentation/powerpc/ultravisor.rst | 1048 +++++++++++++++++
arch/powerpc/boot/main.c | 41 +
arch/powerpc/boot/ops.h | 2 +
arch/powerpc/boot/wrapper | 24 +-
arch/powerpc/boot/zImage.lds.S | 8 +
arch/powerpc/configs/ppc64_defconfig | 1 +
arch/powerpc/configs/pseries_defconfig | 1 +
arch/powerpc/include/asm/asm-prototypes.h | 11 +
arch/powerpc/include/asm/mem_encrypt.h | 26 +
arch/powerpc/include/asm/reg.h | 3 +
arch/powerpc/include/asm/svm.h | 31 +
arch/powerpc/include/asm/ultravisor-api.h | 29 +
arch/powerpc/include/asm/ultravisor.h | 29 +
arch/powerpc/kernel/Makefile | 3 +
arch/powerpc/kernel/machine_kexec_64.c | 9 +
arch/powerpc/kernel/paca.c | 52 +-
arch/powerpc/kernel/prom_init.c | 96 ++
arch/powerpc/kernel/sysfs.c | 29 +
arch/powerpc/kernel/ucall.S | 20 +
arch/powerpc/platforms/pseries/Kconfig | 14 +
arch/powerpc/platforms/pseries/Makefile | 1 +
arch/powerpc/platforms/pseries/iommu.c | 6 +-
arch/powerpc/platforms/pseries/setup.c | 5 +-
arch/powerpc/platforms/pseries/smp.c | 3 +-
arch/powerpc/platforms/pseries/svm.c | 85 ++
26 files changed, 1571 insertions(+), 11 deletions(-)
create mode 100644 Documentation/powerpc/ultravisor.rst
create mode 100644 arch/powerpc/include/asm/mem_encrypt.h
create mode 100644 arch/powerpc/include/asm/svm.h
create mode 100644 arch/powerpc/include/asm/ultravisor-api.h
create mode 100644 arch/powerpc/include/asm/ultravisor.h
create mode 100644 arch/powerpc/kernel/ucall.S
create mode 100644 arch/powerpc/platforms/pseries/svm.c
From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
For secure VMs, the signing tool will create a ticket called the "ESM blob"
for the Enter Secure Mode ultravisor call with the signatures of the kernel
and initrd among other things.
This adds support to the wrapper script for adding that blob via the "-e"
option to the zImage.pseries.
It also adds code to the zImage wrapper itself to retrieve and if necessary
relocate the blob, and pass its address to Linux via the device-tree, to be
later consumed by prom_init.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
[ bauerman: Minor adjustments to some comments. ]
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/boot/main.c | 41 ++++++++++++++++++++++++++++++++++
arch/powerpc/boot/ops.h | 2 ++
arch/powerpc/boot/wrapper | 24 +++++++++++++++++---
arch/powerpc/boot/zImage.lds.S | 8 +++++++
4 files changed, 72 insertions(+), 3 deletions(-)
@@ -146,6 +146,46 @@ static struct addr_range prep_initrd(struct addr_range vmlinux, void *chosen,return(structaddr_range){(void*)initrd_addr,initrd_size};}+#ifdef __powerpc64__+staticvoidprep_esm_blob(structaddr_rangevmlinux,void*chosen)+{+unsignedlongesm_blob_addr,esm_blob_size;++/* Do we have an ESM (Enter Secure Mode) blob? */+if(_esm_blob_end<=_esm_blob_start)+return;++printf("Attached ESM blob at 0x%p-0x%p\n\r",+_esm_blob_start,_esm_blob_end);+esm_blob_addr=(unsignedlong)_esm_blob_start;+esm_blob_size=_esm_blob_end-_esm_blob_start;++/*+*IftheESMblobistoolowitwillbeclobberedwhenthe+*kernelrelocatestoitsfinallocation.Inthiscase,+*allocateasaferplaceandmoveit.+*/+if(esm_blob_addr<vmlinux.size){+void*old_addr=(void*)esm_blob_addr;++printf("Allocating 0x%lx bytes for esm_blob ...\n\r",+esm_blob_size);+esm_blob_addr=(unsignedlong)malloc(esm_blob_size);+if(!esm_blob_addr)+fatal("Can't allocate memory for ESM blob !\n\r");+printf("Relocating ESM blob 0x%lx <- 0x%p (0x%lx bytes)\n\r",+esm_blob_addr,old_addr,esm_blob_size);+memmove((void*)esm_blob_addr,old_addr,esm_blob_size);+}++/* Tell the kernel ESM blob address via device tree. */+setprop_val(chosen,"linux,esm-blob-start",(u32)(esm_blob_addr));+setprop_val(chosen,"linux,esm-blob-end",(u32)(esm_blob_addr+esm_blob_size));+}+#else+staticinlinevoidprep_esm_blob(structaddr_rangevmlinux,void*chosen){}+#endif+/* A buffer that may be edited by tools operating on a zImage binary so as to*editthecommandlinepassedtovmlinux(bysetting/chosen/bootargs).*Thebufferisputinit'sownsectionsothattoolsmaylocateiteasier.
From: Sukadev Bhattiprolu <redacted>
The ultravisor processor mode is introduced in POWER platforms that
supports the Protected Execution Facility (PEF). Ultravisor is higher
privileged than hypervisor mode.
In PEF enabled platforms, the MSR_S bit is used to indicate if the
thread is in secure state. With the MSR_S bit, the privilege state of
the thread is now determined by MSR_S, MSR_HV and MSR_PR, as follows:
HV PR S=0 S=1
---------------------------------------------
0 0 privileged privileged (secure guest kernel)
0 1 problem problem (secure guest userspace)
1 0 hypervisor ultravisor
1 1 problem reserved
Signed-off-by: Sukadev Bhattiprolu <redacted>
Signed-off-by: Ram Pai <redacted>
[ cclaudio: Update the commit message ]
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/reg.h | 3 +++
1 file changed, 3 insertions(+)
Introduce CONFIG_PPC_SVM to control support for secure guests and include
Ultravisor-related helpers when it is selected
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/asm-prototypes.h | 2 +-
arch/powerpc/kernel/Makefile | 4 +++-
arch/powerpc/platforms/pseries/Kconfig | 11 +++++++++++
3 files changed, 15 insertions(+), 2 deletions(-)
@@ -145,3 +145,14 @@ config PAPR_SCMtristate"Support for the PAPR Storage Class Memory interface"helpEnableaccesstohypervisorprovidedstorageclassmemory.++configPPC_SVM+bool"Secure virtual machine (SVM) support for POWER"+depends onPPC_PSERIES+help+TherearecertainPOWERplatformswhichsupportsecureguestsusing+theProtectedExecutionFacility,withthehelpofanUltravisor+executingbelowthehypervisorlayer.Thisenablessupportfor+thoseguests.++Ifunsure,say"N".
From: Claudio Carvalho <redacted>
The ultracalls (ucalls for short) allow the Secure Virtual Machines
(SVM)s and hypervisor to request services from the ultravisor such as
accessing a register or memory region that can only be accessed when
running in ultravisor-privileged mode.
This patch adds ucall_norets() ultravisor call handler. Like
plpar_hcall_norets(), it also saves and restores the Condition
Register (CR).
The specific service needed from an ucall is specified in register
R3 (the first parameter to the ucall). Other parameters to the
ucall, if any, are specified in registers R4 through R12.
Return value of all ucalls is in register R3. Other output values
from the ucall, if any, are returned in registers R4 through R12.
Each ucall returns specific error codes, applicable in the context
of the ucall. However, like with the PowerPC Architecture Platform
Reference (PAPR), if no specific error code is defined for a particular
situation, then the ucall will fallback to an erroneous
parameter-position based code. i.e U_PARAMETER, U_P2, U_P3 etc depending
on the ucall parameter that may have caused the error.
Every host kernel (powernv) needs to be able to do ucalls in case it
ends up being run in a machine with ultravisor enabled. Otherwise, the
kernel may crash early in boot trying to access ultravisor resources,
for instance, trying to set the partition table entry 0.
Since secure guests also need to be able to do ucalls and its kernel may
not have CONFIG_PPC_POWERNV=y, the ucall.S file is placed under
arch/powerpc/kernel.
If ultravisor is not enabled, the ucalls will be redirected to the
hypervisor which must handle/fail the call.
Thanks to inputs from Ram Pai and Michael Anderson.
Signed-off-by: Claudio Carvalho <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/asm-prototypes.h | 11 +++++++++++
arch/powerpc/include/asm/ultravisor-api.h | 23 +++++++++++++++++++++++
arch/powerpc/kernel/Makefile | 1 +
arch/powerpc/kernel/ucall.S | 20 ++++++++++++++++++++
4 files changed, 55 insertions(+)
Helps document what the hard-coded number means.
Also take the opportunity to fix an #endif comment.
Suggested-by: Alexey Kardashevskiy <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/kernel/paca.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
Secure guest memory is inacessible to devices so regular DMA isn't
possible.
In that case set devices' dma_map_ops to NULL so that the generic
DMA code path will use SWIOTLB and DMA to bounce buffers.
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/platforms/pseries/iommu.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
From: Anshuman Khandual <redacted>
SWIOTLB checks range of incoming CPU addresses to be bounced and sees if
the device can access it through its DMA window without requiring bouncing.
In such cases it just chooses to skip bouncing. But for cases like secure
guests on powerpc platform all addresses need to be bounced into the shared
pool of memory because the host cannot access it otherwise. Hence the need
to do the bouncing is not related to device's DMA window and use of bounce
buffers is forced by setting swiotlb_force.
Also, connect the shared memory conversion functions into the
ARCH_HAS_MEM_ENCRYPT hooks and call swiotlb_update_mem_attributes() to
convert SWIOTLB's memory pool to shared memory.
Signed-off-by: Anshuman Khandual <redacted>
[ bauerman: Use ARCH_HAS_MEM_ENCRYPT hooks to share swiotlb memory pool. ]
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/mem_encrypt.h | 26 +++++++++++++++
arch/powerpc/platforms/pseries/Kconfig | 3 ++
arch/powerpc/platforms/pseries/svm.c | 45 ++++++++++++++++++++++++++
3 files changed, 74 insertions(+)
From: Anshuman Khandual <redacted>
LPPACA structures need to be shared with the host. Hence they need to be in
shared memory. Instead of allocating individual chunks of memory for a
given structure from memblock, a contiguous chunk of memory is allocated
and then converted into shared memory. Subsequent allocation requests will
come from the contiguous chunk which will be always shared memory for all
structures.
While we are able to use a kmem_cache constructor for the Debug Trace Log,
LPPACAs are allocated very early in the boot process (before SLUB is
available) so we need to use a simpler scheme here.
Introduce helper is_svm_platform() which uses the S bit of the MSR to tell
whether we're running as a secure guest.
Signed-off-by: Anshuman Khandual <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
arch/powerpc/kernel/paca.c | 43 +++++++++++++++++++++++++++++++++-
2 files changed, 68 insertions(+), 1 deletion(-)
From: Ram Pai <redacted>
Make the Enter-Secure-Mode (ESM) ultravisor call to switch the VM to secure
mode. Pass kernel base address and FDT address so that the Ultravisor is
able to verify the integrity of the VM using information from the ESM blob.
Add "svm=" command line option to turn on switching to secure mode.
Signed-off-by: Ram Pai <redacted>
[ andmike: Generate an RTAS os-term hcall when the ESM ucall fails. ]
Signed-off-by: Michael Anderson <redacted>
[ bauerman: Cleaned up the code a bit. ]
Signed-off-by: Thiago Jung Bauermann <redacted>
---
.../admin-guide/kernel-parameters.txt | 5 +
arch/powerpc/include/asm/ultravisor-api.h | 3 +
arch/powerpc/kernel/prom_init.c | 96 +++++++++++++++++++
3 files changed, 104 insertions(+)
@@ -4620,6 +4620,11 @@ /sys/power/pm_test). Only available when CONFIG_PM_DEBUG is set. Default value is 5.+ svm= [PPC]+ Format: { on | off | y | n | 1 | 0 }+ This parameter controls use of the Protected+ Execution Facility on pSeries.+ swapaccount=[0|1] [KNL] Enable accounting of swap in memory resource controller if no parameter or 1 is given or disable
@@ -1712,6 +1728,43 @@ static void __init prom_close_stdin(void)}}+#ifdef CONFIG_PPC_SVM+staticintprom_rtas_hcall(uint64_targs)+{+registeruint64_targ1asm("r3")=H_RTAS;+registeruint64_targ2asm("r4")=args;++asmvolatile("sc 1\n":"=r"(arg1):+"r"(arg1),+"r"(arg2):);+returnarg1;+}++staticstructrtas_args__prombssos_term_args;++staticvoid__initprom_rtas_os_term(char*str)+{+phandlertas_node;+__be32val;+u32token;++prom_debug("%s: start...\n",__func__);+rtas_node=call_prom("finddevice",1,1,ADDR("/rtas"));+prom_debug("rtas_node: %x\n",rtas_node);+if(!PHANDLE_VALID(rtas_node))+return;++val=0;+prom_getprop(rtas_node,"ibm,os-term",&val,sizeof(val));+token=be32_to_cpu(val);+prom_debug("ibm,os-term: %x\n",token);+if(token==0)+prom_panic("Could not get token for ibm,os-term\n");+os_term_args.token=cpu_to_be32(token);+prom_rtas_hcall((uint64_t)&os_term_args);+}+#endif /* CONFIG_PPC_SVM */+/**AllocateroomforandinstantiateRTAS*/
@@ -3168,6 +3221,46 @@ static void unreloc_toc(void)#endif#endif+#ifdef CONFIG_PPC_SVM+/*+*PerformtheEnterSecureModeultracall.+*/+staticintenter_secure_mode(unsignedlongkbase,unsignedlongfdt)+{+registerunsignedlongr3asm("r3")=UV_ESM;+registerunsignedlongr4asm("r4")=kbase;+registerunsignedlongr5asm("r5")=fdt;++asmvolatile("sc 2":"+r"(r3):"r"(r4),"r"(r5));++returnr3;+}++/*+*CalltheUltravisortotransferustosecurememoryifwehaveanESMblob.+*/+staticvoidsetup_secure_guest(unsignedlongkbase,unsignedlongfdt)+{+intret;++if(!prom_svm_enable)+return;++/* Switch to secure mode. */+prom_printf("Switching to secure mode.\n");++ret=enter_secure_mode(kbase,fdt);+if(ret!=U_SUCCESS){+prom_printf("Returned %d from switching to secure mode.\n",ret);+prom_rtas_os_term("Switch to secure mode failed.\n");+}+}+#else+staticvoidsetup_secure_guest(unsignedlongkbase,unsignedlongfdt)+{+}+#endif /* CONFIG_PPC_SVM */+/**Weenterhereearlyon,whentheOpenFirmwarepromisstill*handlingexceptionsandtheMMUhashtableforus.
@@ -3366,6 +3459,9 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,unreloc_toc();#endif+/* Move to secure memory if we're supposed to be secure guests. */+setup_secure_guest(kbase,hdr);+__start(hdr,kbase,0,0,0,0,0);return0;
From: Ram Pai <redacted>
These functions are used when the guest wants to grant the hypervisor
access to certain pages.
Signed-off-by: Ram Pai <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/ultravisor-api.h | 2 ++
arch/powerpc/include/asm/ultravisor.h | 24 +++++++++++++++++++++++
2 files changed, 26 insertions(+)
@@ -297,8 +298,10 @@ static inline int alloc_dispatch_logs(void)staticintalloc_dispatch_log_kmem_cache(void){+void(*ctor)(void*)=get_dtl_cache_ctor();+dtl_cache=kmem_cache_create("dtl",DISPATCH_LOG_BYTES,-DISPATCH_LOG_BYTES,0,NULL);+DISPATCH_LOG_BYTES,0,ctor);if(!dtl_cache){pr_warn("Failed to create dispatch trace log buffer cache\n");pr_warn("Stolen time statistics will be unreliable\n");
From: Ram Pai <redacted>
A new kernel deserves a clean slate. Any pages shared with the hypervisor
is unshared before invoking the new kernel. However there are exceptions.
If the new kernel is invoked to dump the current kernel, or if there is a
explicit request to preserve the state of the current kernel, unsharing
of pages is skipped.
NOTE: While testing crashkernel, make sure at least 256M is reserved for
crashkernel. Otherwise SWIOTLB allocation will fail and crash kernel will
fail to boot.
Signed-off-by: Ram Pai <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/ultravisor-api.h | 1 +
arch/powerpc/include/asm/ultravisor.h | 5 +++++
arch/powerpc/kernel/machine_kexec_64.c | 9 +++++++++
3 files changed, 15 insertions(+)
From: Ryan Grimm <redacted>
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the guest's CPU 0.
Signed-off-by: Ryan Grimm <redacted>
Reviewed-by: Ram Pai <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/kernel/sysfs.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
From: Sukadev Bhattiprolu <redacted>
Normally, the HV emulates some instructions like MSGSNDP, MSGCLRP
from a KVM guest. To emulate the instructions, it must first read
the instruction from the guest's memory and decode its parameters.
However for a secure guest (aka SVM), the page containing the
instruction is in secure memory and the HV cannot access directly.
It would need the Ultravisor (UV) to facilitate accessing the
instruction and parameters but the UV currently does not have
the support for such accesses.
Until the UV has such support, disable doorbells in SVMs. This might
incur a performance hit but that is yet to be quantified.
With this patch applied (needed only in SVMs not needed for HV) we
are able to launch SVM guests with multi-core support. Eg:
qemu -smp sockets=2,cores=2,threads=2.
Fix suggested by Benjamin Herrenschmidt. Thanks to input from
Paul Mackerras, Ram Pai and Michael Anderson.
Signed-off-by: Sukadev Bhattiprolu <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/platforms/pseries/smp.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Shoudn't:
if (!is_secure_guest())
set_pci_dma_ops(&dma_iommu_ops);
be enough here, given that NULL is the default?
Also either way I think this conditional needs a comment explaining
why it is there.
Shoudn't:
if (!is_secure_guest())
set_pci_dma_ops(&dma_iommu_ops);
be enough here, given that NULL is the default?
Indeed, it is enough.
Also either way I think this conditional needs a comment explaining
why it is there.
Good point. I added the commit message as a comment in the code.
New version of this patch below.
From 5dc3914efa4765eef2869d554d4ea1c676bb1e75 Mon Sep 17 00:00:00 2001
From: Thiago Jung Bauermann <redacted>
Date: Thu, 24 Jan 2019 22:40:16 -0200
Subject: [PATCH] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure
guests
Secure guest memory is inacessible to devices so regular DMA isn't
possible.
In that case set devices' dma_map_ops to NULL so that the generic
DMA code path will use SWIOTLB to bounce buffers for DMA.
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/platforms/pseries/iommu.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2019-08-12 12:36:15
Thiago Jung Bauermann [off-list ref] writes:
quoted hunk
From: Anshuman Khandual <redacted>
LPPACA structures need to be shared with the host. Hence they need to be in
shared memory. Instead of allocating individual chunks of memory for a
given structure from memblock, a contiguous chunk of memory is allocated
and then converted into shared memory. Subsequent allocation requests will
come from the contiguous chunk which will be always shared memory for all
structures.
While we are able to use a kmem_cache constructor for the Debug Trace Log,
LPPACAs are allocated very early in the boot process (before SLUB is
available) so we need to use a simpler scheme here.
Introduce helper is_svm_platform() which uses the S bit of the MSR to tell
whether we're running as a secure guest.
Signed-off-by: Anshuman Khandual <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
arch/powerpc/kernel/paca.c | 43 +++++++++++++++++++++++++++++++++-
2 files changed, 68 insertions(+), 1 deletion(-)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2019-08-12 13:03:21
Thiago Jung Bauermann [off-list ref] writes:
From: Ryan Grimm <redacted>
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the guest's CPU 0.
Why CPU 0?
If we have different CPUs running with different MSR_S then something
has gone badly wrong, no?
So can't we just read the MSR on whatever CPU the sysfs code happens to
run on.
cheers
From: Anshuman Khandual <redacted>
LPPACA structures need to be shared with the host. Hence they need to be in
shared memory. Instead of allocating individual chunks of memory for a
given structure from memblock, a contiguous chunk of memory is allocated
and then converted into shared memory. Subsequent allocation requests will
come from the contiguous chunk which will be always shared memory for all
structures.
While we are able to use a kmem_cache constructor for the Debug Trace Log,
LPPACAs are allocated very early in the boot process (before SLUB is
available) so we need to use a simpler scheme here.
Introduce helper is_svm_platform() which uses the S bit of the MSR to tell
whether we're running as a secure guest.
Signed-off-by: Anshuman Khandual <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
arch/powerpc/kernel/paca.c | 43 +++++++++++++++++++++++++++++++++-
2 files changed, 68 insertions(+), 1 deletion(-)
I may be confused about which year the copyright refers to. I thought it
was the year when the patch was committed. If it is the first time the
patch was published then this one should be 2018.
--
Thiago Jung Bauermann
IBM Linux Technology Center
From: Ryan Grimm <redacted>
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the guest's CPU 0.
Why CPU 0?
If we have different CPUs running with different MSR_S then something
has gone badly wrong, no?
Yes, that would be very bad.
So can't we just read the MSR on whatever CPU the sysfs code happens to
run on.
Good point. I made the change in the patch below.
--
Thiago Jung Bauermann
IBM Linux Technology Center
From 2d951305e118bf286f8e83cbf396448085186357 Mon Sep 17 00:00:00 2001
From: Ryan Grimm <redacted>
Date: Tue, 15 Jan 2019 11:56:29 -0600
Subject: [PATCH] powerpc/pseries/svm: Export guest SVM status to user space
via sysfs
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the current CPU.
Signed-off-by: Ryan Grimm <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/kernel/sysfs.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2019-08-14 10:54:18
Thiago Jung Bauermann [off-list ref] writes:
Michael Ellerman [off-list ref] writes:
quoted
Thiago Jung Bauermann [off-list ref] writes:
quoted
From: Anshuman Khandual <redacted>
LPPACA structures need to be shared with the host. Hence they need to be in
shared memory. Instead of allocating individual chunks of memory for a
given structure from memblock, a contiguous chunk of memory is allocated
and then converted into shared memory. Subsequent allocation requests will
come from the contiguous chunk which will be always shared memory for all
structures.
While we are able to use a kmem_cache constructor for the Debug Trace Log,
LPPACAs are allocated very early in the boot process (before SLUB is
available) so we need to use a simpler scheme here.
Introduce helper is_svm_platform() which uses the S bit of the MSR to tell
whether we're running as a secure guest.
Signed-off-by: Anshuman Khandual <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
arch/powerpc/kernel/paca.c | 43 +++++++++++++++++++++++++++++++++-
2 files changed, 68 insertions(+), 1 deletion(-)
I may be confused about which year the copyright refers to. I thought it
was the year when the patch was committed. If it is the first time the
patch was published then this one should be 2018.
I'm not a lawyer etc. but AIUI the date above is about the authorship,
ie. when it was originally written, not when it was published or
committed.
In general I don't think it matters too much, but in this case I'm
pretty sure Anshuman can't have possibly written it in 2019 on behalf of
IBM :)
So we can either change the date to 2018, or drop his name and just say
it's copyright 2019 by IBM.
cheers
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2019-08-15 06:30:14
Thiago Jung Bauermann [off-list ref] writes:
Michael Ellerman [off-list ref] writes:
quoted
Thiago Jung Bauermann [off-list ref] writes:
quoted
From: Ryan Grimm <redacted>
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the guest's CPU 0.
Why CPU 0?
If we have different CPUs running with different MSR_S then something
has gone badly wrong, no?
Yes, that would be very bad.
quoted
So can't we just read the MSR on whatever CPU the sysfs code happens to
run on.
Good point. I made the change in the patch below.
The patch looks good. Although, it raises the question of whether it
should be an attribute of the CPU at all.
I guess there's not obviously anywhere better for it.
Still you should document the attribute in Documentation/ABI/testing/sysfs-devices-system-cpu
cheers
quoted hunk
From 2d951305e118bf286f8e83cbf396448085186357 Mon Sep 17 00:00:00 2001
From: Ryan Grimm <redacted>
Date: Tue, 15 Jan 2019 11:56:29 -0600
Subject: [PATCH] powerpc/pseries/svm: Export guest SVM status to user space
via sysfs
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the current CPU.
Signed-off-by: Ryan Grimm <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/kernel/sysfs.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
From: Ryan Grimm <redacted>
User space might want to know it's running in a secure VM. It can't do
a mfmsr because mfmsr is a privileged instruction.
The solution here is to create a cpu attribute:
/sys/devices/system/cpu/svm
which will read 0 or 1 based on the S bit of the guest's CPU 0.
Why CPU 0?
If we have different CPUs running with different MSR_S then something
has gone badly wrong, no?
Yes, that would be very bad.
quoted
So can't we just read the MSR on whatever CPU the sysfs code happens to
run on.
Good point. I made the change in the patch below.
The patch looks good. Although, it raises the question of whether it
should be an attribute of the CPU at all.
I guess there's not obviously anywhere better for it.
Ok. TBH this patch is not as urgent as the others. It was added so that
tests have an easy way to tell if they're in an SVM. I can leave it out
for now to figure out if there's a better place for this information.
Still you should document the attribute in Documentation/ABI/testing/sysfs-devices-system-cpu
Indedd, will do that.
--
Thiago Jung Bauermann
IBM Linux Technology Center
From: Anshuman Khandual <redacted>
LPPACA structures need to be shared with the host. Hence they need to be in
shared memory. Instead of allocating individual chunks of memory for a
given structure from memblock, a contiguous chunk of memory is allocated
and then converted into shared memory. Subsequent allocation requests will
come from the contiguous chunk which will be always shared memory for all
structures.
While we are able to use a kmem_cache constructor for the Debug Trace Log,
LPPACAs are allocated very early in the boot process (before SLUB is
available) so we need to use a simpler scheme here.
Introduce helper is_svm_platform() which uses the S bit of the MSR to tell
whether we're running as a secure guest.
Signed-off-by: Anshuman Khandual <redacted>
Signed-off-by: Thiago Jung Bauermann <redacted>
---
arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
arch/powerpc/kernel/paca.c | 43 +++++++++++++++++++++++++++++++++-
2 files changed, 68 insertions(+), 1 deletion(-)
I may be confused about which year the copyright refers to. I thought it
was the year when the patch was committed. If it is the first time the
patch was published then this one should be 2018.
I'm not a lawyer etc. but AIUI the date above is about the authorship,
ie. when it was originally written, not when it was published or
committed.
In general I don't think it matters too much, but in this case I'm
pretty sure Anshuman can't have possibly written it in 2019 on behalf of
IBM :)
So we can either change the date to 2018, or drop his name and just say
it's copyright 2019 by IBM.
I think it's better to change the date to 2018. The same should be done
for svm.c, svm.h and mem_encrypt.h. I'll send a new patch series with
the correction.
--
Thiago Jung Bauermann
IBM Linux Technology Center