[PATCH 0/3][RFC] kvmppc: paravirtualization interface - guest part v3

Subsystems: kernel virtual machine for powerpc (kvm/powerpc), linux for powerpc (32-bit and 64-bit), linux for powerpc embedded ppc4xx, the rest · Maintainers: Madhavan Srinivasan, Linus Torvalds

Series: 3 patches touching 7 files, +229/-4

STALE6590d

7 messages, 3 authors, 2008-10-13 · open the first message on its own page

[PATCH 0/3][RFC] kvmppc: paravirtualization interface - guest part v3

From: <hidden>
Date: 2008-09-16 06:28:13

From: Christian Ehrhardt <redacted>

Version 3 updates:
- guest hypercall infrastructure is now generic (in epapr_hcalls.h)
  while the kvm specific functions stay in kvm_para.h
- the hypercalls now use beat style ABI
- dropped the guest coop patch changing wrteei to wrtee (now mfmsr is
  rewritten avoiding side effects and a lot of corner cases. Additionally this
  does not need any guest cooperation to be effective)

This patch series implements a paravirtualization interface using:
- the device tree mechanism to pass hypervisor informations to the guest
- hypercalls for guest->host calls
- an example exploiter of that interface (magic page)

The device tree format used here (=base for the discussions on
embedded-hypervisor) is the following.
- A node "hypervisor" to show the general availability of some hypervisor data
- flags for features like the example "feature,pv-magicpage"
  setting 1 = available, everything else = unavailable
- Some features might need to pass more data and can use an entry in the
  device tree like the example of "data,pv-magicpage-size"

The host side of these patches can be found on kvm-ppc@vger.kernel.org

I hope that eventually this guest patch series (that is modifying the ppc boot
process and adding e.g. new ppc fixmaps could go upstream (when discussed
and agreed somewhen) via linuxppc-dev, while the kvm host part will go via
kvm (Avi Kivity).

[patches in series]
[PATCH 1/3] kvmppc: read device tree hypervisor node infrastructure
[PATCH 2/3] kvmppc: add hypercall infrastructure - guest part
[PATCH 3/3] kvmppc: magic page paravirtualization - guest part

---
[diffstat]
 arch/powerpc/kernel/kvm.c            |   53 +++++++++++++++++++++++++++++++
 b/arch/powerpc/kernel/Makefile       |    2 +
 b/arch/powerpc/kernel/kvm.c          |   30 +++++++++++++++++

7 more lines

[PATCH 1/3] kvmppc: read device tree hypervisor node infrastructure

From: <hidden>
Date: 2008-09-16 06:28:10

From: Christian Ehrhardt <redacted>

This patch adds the guest portion of the device tree based host->guest
communication. Using the device tree infrastructure this patch implements
kvm_para_available and kvm_arch_para_features (in this patch just the
infrastructure, no specific feature registered).

Signed-off-by: Christian Ehrhardt <redacted>
---

[diffstat]
 arch/powerpc/kernel/Makefile       |    2 +
 arch/powerpc/kernel/kvm.c          |   30 +++++++++++++++++++++++++
 arch/powerpc/kernel/setup_32.c     |    3 ++
 arch/powerpc/platforms/44x/Kconfig |    7 ++++++
 include/asm-powerpc/kvm_para.h     |   43 ++++++++++++++++++++++++++++++++++---
 5 files changed, 82 insertions(+), 3 deletions(-)
[diff]
diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -80,6 +80,8 @@
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
+obj-$(CONFIG_KVM_GUEST)		+= kvm.o
+
 ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c
new file mode 100644
--- /dev/null
+++ b/arch/powerpc/kernel/kvm.c
@@ -0,0 +1,30 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corp. 2008
+ *
+ * Authors:
+ * 	Hollis Blanchard <hollisb@us.ibm.com>
+ * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
+ */
+
+#include <linux/percpu.h>
+#include <linux/mm.h>
+#include <linux/kvm_para.h>
+
+void __init kvm_guest_init(void)
+{
+	if (!kvm_para_available())
+		return;
+}
diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c
--- a/arch/powerpc/kernel/setup_32.c
+++ b/arch/powerpc/kernel/setup_32.c
@@ -17,6 +17,7 @@
 #include <linux/cpu.h>
 #include <linux/console.h>
 #include <linux/lmb.h>
+#include <linux/kvm_para.h>
 
 #include <asm/io.h>
 #include <asm/prom.h>
@@ -319,5 +320,7 @@
 		ppc_md.setup_arch();
 	if ( ppc_md.progress ) ppc_md.progress("arch: exit", 0x3eab);
 
+	kvm_guest_init();
+
 	paging_init();
 }
diff --git a/arch/powerpc/platforms/44x/Kconfig b/arch/powerpc/platforms/44x/Kconfig
--- a/arch/powerpc/platforms/44x/Kconfig
+++ b/arch/powerpc/platforms/44x/Kconfig
@@ -152,3 +152,10 @@
 # 44x errata/workaround config symbols, selected by the CPU models above
 config IBM440EP_ERR42
 	bool
+
+config KVM_GUEST
+	bool "KVM Guest support"
+	depends on EXPERIMENTAL
+	help
+	This option enables various optimizations for running under the KVM
+	hypervisor.
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -14,7 +14,9 @@
  *
  * Copyright IBM Corp. 2008
  *
- * Authors: Hollis Blanchard <hollisb@us.ibm.com>
+ * Authors:
+ * 	Hollis Blanchard <hollisb@us.ibm.com>
+ * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
  */
 
 #ifndef __POWERPC_KVM_PARA_H__
@@ -22,15 +24,50 @@
 
 #ifdef __KERNEL__
 
+#include <linux/of.h>
+
+static struct kvmppc_para_features {
+	char *dtcell;
+	int feature;
+} para_features[] = {
+};
+
 static inline int kvm_para_available(void)
 {
-	return 0;
+	struct device_node *dn;
+	int ret;
+
+	dn = of_find_node_by_path("/hypervisor");
+	ret = !!dn;
+
+	of_node_put(dn);
+
+	return ret;
 }
 
 static inline unsigned int kvm_arch_para_features(void)
 {
-	return 0;
+	struct device_node *dn;
+	const int *dtval;
+	unsigned int features = 0;
+	int i;
+
+	dn = of_find_node_by_path("/hypervisor");
+	if (!dn)
+		return 0;
+
+	for (i = 0; i < ARRAY_SIZE(para_features); i++) {
+		dtval = of_get_property(dn, para_features[i].dtcell, NULL);
+		if (dtval && *dtval == 1)
+			features |= (1 << para_features[i].feature);
+	}
+
+	of_node_put(dn);
+
+	return features;
 }
+
+void kvm_guest_init(void);
 
 #endif /* __KERNEL__ */
 

[PATCH 2/3] kvmppc: add hypercall infrastructure - guest part v3

From: <hidden>
Date: 2008-09-16 06:29:04

From: Christian Ehrhardt <redacted>

This adds the guest portion of the hypercall infrastructure.

Version 3 now follows the beat ABI, but proposes a new implementation style as
static inline asm functions instead of pure assembler code. That should allow
the compiler to be more flexible and therefore a better optimization.

If people agree on that new implementation style we might merge this code.
The current implementation of beat style hypercalls can be found in
arch/powerpc/platforms/cell/beat_hvCall.S

Signed-off-by: Christian Ehrhardt <redacted>
---

[diffstat]
 epapr_hcalls.h |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

[diff]
diff --git a/include/asm-powerpc/epapr_hcalls.h b/include/asm-powerpc/epapr_hcalls.h
new file mode 100644
--- /dev/null
+++ b/include/asm-powerpc/epapr_hcalls.h
@@ -0,0 +1,59 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright IBM Corp. 2008
+ *
+ * Authors:
+ * 	Christian Ehrhardt <ehrhardt@de.ibm.com>
+ */
+
+#ifndef __POWERPC_EPAPR_HCALLS_H__
+#define __POWERPC_EPAPR_HCALLS_H__
+
+#ifdef __KERNEL__
+
+/* Hypercalls use the beat ABI */
+#define KVM_HYPERCALL_BIN 0x44000022
+
+static inline long epapr_hypercall_1in_1out(unsigned int nr, unsigned long p1)
+{
+	register unsigned long hcall asm ("r11") = nr;
+	register unsigned long arg1_ret asm ("r3") = p1;
+
+	asm volatile(".long %1"
+			: "+r"(arg1_ret)
+			: "i"(KVM_HYPERCALL_BIN), "r"(hcall)
+			: "r4", "r5", "r6", "r7", "r8",
+			  "r9", "r10", "r12", "cc");
+	return arg1_ret;
+}
+
+static inline long epapr_hypercall_2in_1out(unsigned int nr,
+					unsigned long p1, unsigned long p2)
+{
+	register unsigned long hcall asm ("r11") = nr;
+	register unsigned long arg1_ret asm ("r3") = p1;
+	register unsigned long arg2 asm ("r4") = p2;
+
+	asm volatile(".long %1"
+			: "+r"(arg1_ret)
+			: "i"(KVM_HYPERCALL_BIN), "r"(hcall), "r"(arg2)
+			: "r5", "r6", "r7", "r8",
+			  "r9", "r10", "r12", "cc");
+	return arg1_ret;
+}
+
+#endif /* __KERNEL__ */
+
+#endif /* __POWERPC_EPAPR_HCALLS_H__ */

[PATCH 3/3] kvmppc: magic page paravirtualization - guest part

From: <hidden>
Date: 2008-09-16 06:29:09

From: Christian Ehrhardt <redacted>

This patch adds the guest handling for the magic page mechanism. A Hypervisor
can modify the device tree passed to the guest. Using that already existing
interface a guest can simply detect available hypervisor features and agree
on the supported ones using hypercalls.
In this example it is checked for the feature switch "feature,pv-magicpage"
in the hypervisor node and additional data which represents the size the
hypervisor requests in "data,pv-magicpage-size".
When the guest reads that data and wants to support it the memory is allocated
and passed to the hypervisor using the KVM_HCALL_RESERVE_MAGICPAGE hypercall.

Signed-off-by: Christian Ehrhardt <redacted>
---

[diffstat]
 arch/powerpc/kernel/kvm.c      |   53 +++++++++++++++++++++++++++++++++++++++++
 include/asm-powerpc/fixmap.h   |   10 ++++++-
 include/asm-powerpc/kvm_para.h |   26 ++++++++++++++++++++
 3 files changed, 88 insertions(+), 1 deletion(-)
[diff]
diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c
--- a/arch/powerpc/kernel/kvm.c
+++ b/arch/powerpc/kernel/kvm.c
@@ -22,9 +22,62 @@
 #include <linux/percpu.h>
 #include <linux/mm.h>
 #include <linux/kvm_para.h>
+#include <linux/bootmem.h>
+#include <asm/fixmap.h>
+#include <asm/epapr_hcalls.h>
+
+/*
+ * this is guest memory granted to the hypervisor;
+ * the hypervisor can place data in this area and rewrite
+ * privileged instructions to read from this area without
+ * trapping.
+ * Only the Hypervisor needs to be aware of the structure layout
+ * which makes the guest more felxible - the guest only guarantees
+ * the size which is requested by the hypervisor and read from a
+ * device tree entry.
+ */
+static void *kvm_magicpage;
+
+static void __init kvmppc_register_magic_page(void)
+{
+	unsigned long gvaddr;
+	unsigned long gpaddr;
+	int size;
+	long err;
+
+	size = kvmppc_pv_read_data(KVM_PVDATA_MAGICPAGE_SIZE);
+	if (size < 0) {
+		printk(KERN_ERR "%s: couldn't read size for kvmppc style "
+			"paravirtualization support (got %d)\n",
+			__func__, size);
+		return;
+	}
+
+	/* FIXME Guest SMP needs that percpu which */
+	kvm_magicpage = alloc_bootmem(size);
+	if (!kvm_magicpage) {
+		printk(KERN_ERR "%s - failed to allocate %d bytes\n",
+			 __func__, size);
+		return;
+	}
+	gpaddr = (unsigned long)__pa(kvm_magicpage);
+	gvaddr = fix_to_virt(FIX_KVM_PV);
+
+	err = epapr_hypercall_2in_1out(KVM_HCALL_RESERVE_MAGICPAGE,
+					gvaddr, gpaddr);
+	if (err)
+		printk(KERN_ERR "%s: couldn't register pv mem\n", __func__);
+	else
+		printk(KERN_NOTICE "%s: registered %d bytes for pv mem support"
+			" (gvaddr 0x%08lx gpaddr 0x%08lx)\n",
+			 __func__, size, gvaddr, gpaddr);
+}
 
 void __init kvm_guest_init(void)
 {
 	if (!kvm_para_available())
 		return;
+
+	if (kvm_para_has_feature(KVM_FEATURE_PPCPV_MAGICPAGE))
+		kvmppc_register_magic_page();
 }
diff --git a/include/asm-powerpc/fixmap.h b/include/asm-powerpc/fixmap.h
--- a/include/asm-powerpc/fixmap.h
+++ b/include/asm-powerpc/fixmap.h
@@ -36,7 +36,7 @@
  *
  * these 'compile-time allocated' memory buffers are
  * fixed-size 4k pages. (or larger if used with an increment
- * highger than 1) use fixmap_set(idx,phys) to associate
+ * higher than 1) use fixmap_set(idx,phys) to associate
  * physical memory with fixmap indices.
  *
  * TLB entries of such buffers will not be flushed across
@@ -44,6 +44,14 @@
  */
 enum fixed_addresses {
 	FIX_HOLE,
+#ifdef CONFIG_KVM_GUEST
+	/*
+	 * reserved virtual address space for paravirtualization - needs to be
+	 *  <=32k away from base address 0 to be able to reach it with
+	 * immediate addressing using base 0 instead of needing a register.
+	 */
+	FIX_KVM_PV,
+#endif
 #ifdef CONFIG_HIGHMEM
 	FIX_KMAP_BEGIN,	/* reserved pte's for temporary kernel mappings */
 	FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1,
diff --git a/include/asm-powerpc/kvm_para.h b/include/asm-powerpc/kvm_para.h
--- a/include/asm-powerpc/kvm_para.h
+++ b/include/asm-powerpc/kvm_para.h
@@ -26,10 +26,18 @@
 
 #include <linux/of.h>
 
+#define KVM_HCALL_RESERVE_MAGICPAGE	0
+
+#define KVM_PVDATA_MAGICPAGE_SIZE	"data,pv-magicpage-size"
+
+/* List of PV features supported, returned as a bitfield */
+#define KVM_FEATURE_PPCPV_MAGICPAGE	0
+
 static struct kvmppc_para_features {
 	char *dtcell;
 	int feature;
 } para_features[] = {
+	{ "feature,pv-magicpage", KVM_FEATURE_PPCPV_MAGICPAGE }
 };
 
 static inline int kvm_para_available(void)
@@ -67,6 +75,24 @@
 	return features;
 }
 
+/* reads the specified data field out of the hypervisor node */
+static inline int kvmppc_pv_read_data(char *dtcell)
+{
+	struct device_node *dn;
+	const int *dtval;
+
+	dn = of_find_node_by_path("/hypervisor");
+	if (!dn)
+		return -EINVAL;
+
+	dtval = of_get_property(dn, dtcell, NULL);
+	of_node_put(dn);
+	if (dtval)
+		return *dtval;
+	else
+		return -EINVAL;
+}
+
 void kvm_guest_init(void);
 
 #endif /* __KERNEL__ */

Re: [PATCH 2/3] kvmppc: add hypercall infrastructure - guest part v3

From: Kumar Gala <hidden>
Date: 2008-10-13 16:46:20

On Sep 16, 2008, at 1:27 AM, ehrhardt@linux.vnet.ibm.com wrote:
quoted hunk
From: Christian Ehrhardt <redacted>

This adds the guest portion of the hypercall infrastructure.

Version 3 now follows the beat ABI, but proposes a new implementation style as
static inline asm functions instead of pure assembler code. That should allow
the compiler to be more flexible and therefore a better optimization.

If people agree on that new implementation style we might merge this code.
The current implementation of beat style hypercalls can be found in
arch/powerpc/platforms/cell/beat_hvCall.S

Signed-off-by: Christian Ehrhardt <redacted>
---

[diffstat]
epapr_hcalls.h |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)

[diff]
diff --git a/include/asm-powerpc/epapr_hcalls.h b/include/asm-powerpc/epapr_hcalls.h
new file mode 100644
--- /dev/null
+++ b/include/asm-powerpc/epapr_hcalls.h
@@ -0,0 +1,59 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *

13 more quoted lines

Any reason this isn't 'sc' ?

Also, can we make this "sc 1" so its works when we have HW hypervisor support?

- k

Re: [PATCH 2/3] kvmppc: add hypercall infrastructure - guest part v3

From: Hollis Blanchard <hidden>
Date: 2008-10-13 17:29:52

On Mon, 2008-10-13 at 11:42 -0500, Kumar Gala wrote:
quoted
+
+/* Hypercalls use the beat ABI */
+#define KVM_HYPERCALL_BIN 0x44000022
Any reason this isn't 'sc' ?

Also, can we make this "sc 1" so its works when we have HW hypervisor  
support?
Actually, it is "sc 1".

Many versions of as (including 2.16.1) don't recognize that instruction.
This is the standard workaround for situations like that.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [PATCH 2/3] kvmppc: add hypercall infrastructure - guest part v3

From: Kumar Gala <hidden>
Date: 2008-10-13 17:47:07

On Oct 13, 2008, at 12:29 PM, Hollis Blanchard wrote:
On Mon, 2008-10-13 at 11:42 -0500, Kumar Gala wrote:
quoted
quoted
+
+/* Hypercalls use the beat ABI */
+#define KVM_HYPERCALL_BIN 0x44000022
Any reason this isn't 'sc' ?

Also, can we make this "sc 1" so its works when we have HW hypervisor
support?
Actually, it is "sc 1".

Many versions of as (including 2.16.1) don't recognize that instruction.
This is the standard workaround for situations like that.
Ahh, we'll than can we add that to the comment, and rename 'KVM_HYPERCALL_BIN' to something like 'EPAPR_HYPERCALL_INSTR'

- k
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help