On Mon, Jan 14, 2013 at 05:17:31PM -0500, Christoffer Dall wrote:

On Mon, Jan 14, 2013 at 1:49 PM, Gleb Natapov [off-list ref] wrote:

quoted

A couple of general question about ABI. If they were already answered
just refer me to the previous discussion.

On Tue, Jan 08, 2013 at 01:38:55PM -0500, Christoffer Dall wrote:

quoted

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index a4df553..4237c27 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt

@@ -293,7 +293,7 @@ kvm_run' (see below).
 4.11 KVM_GET_REGS

 Capability: basic
-Architectures: all
+Architectures: all except ARM
 Type: vcpu ioctl
 Parameters: struct kvm_regs (out)
 Returns: 0 on success, -1 on error

@@ -314,7 +314,7 @@ struct kvm_regs {
 4.12 KVM_SET_REGS

 Capability: basic
-Architectures: all
+Architectures: all except ARM
 Type: vcpu ioctl
 Parameters: struct kvm_regs (in)
 Returns: 0 on success, -1 on error

@@ -600,7 +600,7 @@ struct kvm_fpu {
 4.24 KVM_CREATE_IRQCHIP

Why KVM_GET_REGS/KVM_SET_REGS are not usable for arm?

We use the ONE_REG API instead and we don't want to support two
separate APIs to user space.

I suppose fetching all registers is not anywhere on a fast path in
userspace :)

quoted

quoted

 Capability: KVM_CAP_IRQCHIP
-Architectures: x86, ia64
+Architectures: x86, ia64, ARM
 Type: vm ioctl
 Parameters: none
 Returns: 0 on success, -1 on error

@@ -608,7 +608,8 @@ Returns: 0 on success, -1 on error
 Creates an interrupt controller model in the kernel.  On x86, creates a virtual
 ioapic, a virtual PIC (two PICs, nested), and sets up future vcpus to have a
 local APIC.  IRQ routing for GSIs 0-15 is set to both PIC and IOAPIC; GSI 16-23
-only go to the IOAPIC.  On ia64, a IOSAPIC is created.
+only go to the IOAPIC.  On ia64, a IOSAPIC is created. On ARM, a GIC is
+created.


 4.25 KVM_IRQ_LINE

@@ -1775,6 +1776,14 @@ registers, find a list below:
   PPC   | KVM_REG_PPC_VPA_DTL   | 128
   PPC   | KVM_REG_PPC_EPCR   | 32

+ARM registers are mapped using the lower 32 bits.  The upper 16 of that
+is the register group type, or coprocessor number:
+
+ARM core registers have the following id bit patterns:
+  0x4002 0000 0010 <index into the kvm_regs struct:16>
+
+
+
 4.69 KVM_GET_ONE_REG

 Capability: KVM_CAP_ONE_REG

@@ -2127,6 +2136,46 @@ written, then `n_invalid' invalid entries, invalidating any previously
 valid entries found.


+4.77 KVM_ARM_VCPU_INIT
+
+Capability: basic
+Architectures: arm
+Type: vcpu ioctl
+Parameters: struct struct kvm_vcpu_init (in)
+Returns: 0 on success; -1 on error
+Errors:
+  EINVAL:    the target is unknown, or the combination of features is invalid.
+  ENOENT:    a features bit specified is unknown.
+
+This tells KVM what type of CPU to present to the guest, and what
+optional features it should have.  This will cause a reset of the cpu
+registers to their initial values.  If this is not called, KVM_RUN will
+return ENOEXEC for that vcpu.
+

Can different vcpus of the same VM be of different type?

In the future yes. For example, if we ever want to virtualize a
Big.Little system.

quoted

quoted

+Note that because some registers reflect machine topology, all vcpus
+should be created before this ioctl is invoked.

How cpu hot plug suppose to work?

Those CPUs would be added from the beginning, but not powered on, and
would be powered on later on, I suppose.  See
https://lists.cs.columbia.edu/pipermail/kvmarm/2013-January/004617.html.

When we suggested similar "hot plug" for x86, people started screaming
how they suppose to know when they create a VM how much vcpus they will
need in the future. In short people who are asking for hot plug (on x86
at least) do not like such solution.

quoted

quoted

+
+
+4.78 KVM_GET_REG_LIST
+
+Capability: basic
+Architectures: arm
+Type: vcpu ioctl
+Parameters: struct kvm_reg_list (in/out)
+Returns: 0 on success; -1 on error
+Errors:
+  E2BIG:     the reg index list is too big to fit in the array specified by
+             the user (the number required will be written into n).
+
+struct kvm_reg_list {
+     __u64 n; /* number of registers in reg[] */
+     __u64 reg[0];
+};
+
+This ioctl returns the guest registers that are supported for the
+KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
+
+

Doesn't userspace know what registers are supported by each CPU type?

It would know about core registers, but there is a huge space of
co-processors, and we don't emulate all of them or support
getting/setting all of them yet. Surely this is something that will
change over time and we want user space to be able to discover the
available registers for backwards compatibility, migration, etc.

-Christoffer

--
			Gleb.