From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:17
Now that the arm64 rewrite is in mainline, I've taken a stab at fixing
the 32bit code the same way. This is fairly straightforward (once
you've been through it once...), with a few patches that adapt the
code to be similar to the 64bit version.
Note that the timer and GIC code should be made common between the two
architectures, as this is litterally the exact same code (I've posted
some proof of concept for that a while ago, see
http://www.spinics.net/lists/kvm/msg126775.html).
This has been tested on a Dual A7, and the code is pushed on a branch:
git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm/wsinc
M.
* From v1:
- Rebased on -rc2
- Moved VTCR setup out of the init sequence (and into C code)
- Now depends on the first patch of the VHE series
Marc Zyngier (28):
ARM: KVM: Move the HYP code to its own section
ARM: KVM: Remove __kvm_hyp_code_start/__kvm_hyp_code_end
ARM: KVM: Move VFP registers to a CPU context structure
ARM: KVM: Move CP15 array into the CPU context structure
ARM: KVM: Move GP registers into the CPU context structure
ARM: KVM: Add a HYP-specific header file
ARM: KVM: Add system register accessor macros
ARM: KVM: Add TLB invalidation code
ARM: KVM: Add CP15 save/restore code
ARM: KVM: Add timer save/restore
ARM: KVM: Add vgic v2 save/restore
ARM: KVM: Add VFP save/restore
ARM: KVM: Add banked registers save/restore
ARM: KVM: Add guest entry code
ARM: KVM: Add VFP lazy save/restore handler
ARM: KVM: Add the new world switch implementation
ARM: KVM: Add populating of fault data structure
ARM: KVM: Add HYP mode entry code
ARM: KVM: Add panic handling code
ARM: KVM: Change kvm_call_hyp return type to unsigned long
ARM: KVM: Remove the old world switch
ARM: KVM: Switch to C-based stage2 init
ARM: KVM: Remove __weak attributes
ARM: KVM: Turn CP15 defines to an enum
ARM: KVM: Cleanup asm-offsets.c
ARM: KVM: Remove unused hyp_pc field
ARM: KVM: Remove handling of ARM_EXCEPTION_DATA/PREF_ABORT
ARM: KVM: Remove __kvm_hyp_exit/__kvm_hyp_exit_end
arch/arm/include/asm/kvm_asm.h | 41 +--
arch/arm/include/asm/kvm_emulate.h | 15 +-
arch/arm/include/asm/kvm_host.h | 61 +++-
arch/arm/include/asm/kvm_mmu.h | 2 +-
arch/arm/include/asm/virt.h | 4 +
arch/arm/kernel/asm-offsets.c | 40 +--
arch/arm/kernel/vmlinux.lds.S | 6 +
arch/arm/kvm/Makefile | 1 +
arch/arm/kvm/arm.c | 2 +-
arch/arm/kvm/coproc.c | 52 +--
arch/arm/kvm/coproc.h | 16 +-
arch/arm/kvm/emulate.c | 34 +-
arch/arm/kvm/guest.c | 5 +-
arch/arm/kvm/handle_exit.c | 7 -
arch/arm/kvm/hyp/Makefile | 14 +
arch/arm/kvm/hyp/banked-sr.c | 77 +++++
arch/arm/kvm/hyp/cp15-sr.c | 84 +++++
arch/arm/kvm/hyp/entry.S | 101 ++++++
arch/arm/kvm/hyp/hyp-entry.S | 169 ++++++++++
arch/arm/kvm/hyp/hyp.h | 130 ++++++++
arch/arm/kvm/hyp/s2-setup.c | 34 ++
arch/arm/kvm/hyp/switch.c | 228 +++++++++++++
arch/arm/kvm/hyp/timer-sr.c | 71 ++++
arch/arm/kvm/hyp/tlb.c | 71 ++++
arch/arm/kvm/hyp/vfp.S | 68 ++++
arch/arm/kvm/hyp/vgic-v2-sr.c | 84 +++++
arch/arm/kvm/init.S | 8 -
arch/arm/kvm/interrupts.S | 480 +--------------------------
arch/arm/kvm/interrupts_head.S | 648 -------------------------------------
arch/arm/kvm/reset.c | 2 +-
arch/arm64/include/asm/kvm_asm.h | 3 -
31 files changed, 1265 insertions(+), 1293 deletions(-)
create mode 100644 arch/arm/kvm/hyp/Makefile
create mode 100644 arch/arm/kvm/hyp/banked-sr.c
create mode 100644 arch/arm/kvm/hyp/cp15-sr.c
create mode 100644 arch/arm/kvm/hyp/entry.S
create mode 100644 arch/arm/kvm/hyp/hyp-entry.S
create mode 100644 arch/arm/kvm/hyp/hyp.h
create mode 100644 arch/arm/kvm/hyp/s2-setup.c
create mode 100644 arch/arm/kvm/hyp/switch.c
create mode 100644 arch/arm/kvm/hyp/timer-sr.c
create mode 100644 arch/arm/kvm/hyp/tlb.c
create mode 100644 arch/arm/kvm/hyp/vfp.S
create mode 100644 arch/arm/kvm/hyp/vgic-v2-sr.c
delete mode 100644 arch/arm/kvm/interrupts_head.S
--
2.1.4
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:18
In order to be able to spread the HYP code into multiple compilation
units, adopt a layout similar to that of arm64:
- the HYP text is emited in its own section (.hyp.text)
- two linker generated symbols are use to identify the boundaries
of that section
No functionnal change.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_asm.h | 6 ++++--
arch/arm/include/asm/virt.h | 4 ++++
arch/arm/kernel/vmlinux.lds.S | 6 ++++++
arch/arm/kvm/interrupts.S | 13 +++++--------
4 files changed, 19 insertions(+), 10 deletions(-)
From: Christoffer Dall <hidden> Date: 2016-02-09 18:39:53
On Thu, Feb 04, 2016 at 11:00:18AM +0000, Marc Zyngier wrote:
In order to be able to spread the HYP code into multiple compilation
units, adopt a layout similar to that of arm64:
- the HYP text is emited in its own section (.hyp.text)
- two linker generated symbols are use to identify the boundaries
of that section
No functionnal change.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:19
Now that we've unified the way we refer to the HYP text between
arm and arm64, drop __kvm_hyp_code_start/end, and just use the
__hyp_text_start/end symbols.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_asm.h | 3 ---
arch/arm/kvm/arm.c | 2 +-
arch/arm64/include/asm/kvm_asm.h | 3 ---
3 files changed, 1 insertion(+), 7 deletions(-)
From: Christoffer Dall <hidden> Date: 2016-02-09 18:39:59
On Thu, Feb 04, 2016 at 11:00:19AM +0000, Marc Zyngier wrote:
Now that we've unified the way we refer to the HYP text between
arm and arm64, drop __kvm_hyp_code_start/end, and just use the
__hyp_text_start/end symbols.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:20
In order to turn the WS code into something that looks a bit
more like the arm64 version, move the VFP registers into a
CPU context container for both the host and the guest.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_host.h | 11 +++++++----
arch/arm/kernel/asm-offsets.c | 5 +++--
arch/arm/kvm/coproc.c | 20 ++++++++++----------
arch/arm/kvm/interrupts.S | 10 ++++++----
4 files changed, 26 insertions(+), 20 deletions(-)
@@ -88,9 +88,15 @@ struct kvm_vcpu_fault_info {u32hyp_pc;/* PC when exception was taken from Hyp mode */};-typedefstructvfp_hard_structkvm_cpu_context_t;+structkvm_cpu_context{+structvfp_hard_structvfp;+};++typedefstructkvm_cpu_contextkvm_cpu_context_t;structkvm_vcpu_arch{+structkvm_cpu_contextctxt;+structkvm_regsregs;inttarget;/* Processor target */
@@ -111,9 +117,6 @@ struct kvm_vcpu_arch {/* Exception Information */structkvm_vcpu_fault_infofault;-/* Floating point registers (VFP and Advanced SIMD/NEON) */-structvfp_hard_structvfp_guest;-/* Host FP context */kvm_cpu_context_t*host_cpu_context;
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:07
On Thu, Feb 04, 2016 at 11:00:20AM +0000, Marc Zyngier wrote:
In order to turn the WS code into something that looks a bit
more like the arm64 version, move the VFP registers into a
CPU context container for both the host and the guest.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:21
Continuing our rework of the CPU context, we now move the CP15
array into the CPU context structure. As this causes quite a bit
of churn, we introduce the vcpu_cp15() macro that abstract the
location of the actual array. This will probably help next time
we have to revisit that code.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_emulate.h | 2 +-
arch/arm/include/asm/kvm_host.h | 6 +++---
arch/arm/include/asm/kvm_mmu.h | 2 +-
arch/arm/kernel/asm-offsets.c | 2 +-
arch/arm/kvm/coproc.c | 32 ++++++++++++++++----------------
arch/arm/kvm/coproc.h | 16 ++++++++--------
arch/arm/kvm/emulate.c | 22 +++++++++++-----------
arch/arm/kvm/interrupts_head.S | 3 ++-
8 files changed, 43 insertions(+), 42 deletions(-)
@@ -102,9 +103,6 @@ struct kvm_vcpu_arch {inttarget;/* Processor target */DECLARE_BITMAP(features,KVM_VCPU_MAX_FEATURES);-/* System control coprocessor (cp15) */-u32cp15[NR_CP15_REGS];-/* The CPU type we expose to the VM */u32midr;
@@ -47,7 +47,7 @@ struct coproc_reg {/* Initialization for vcpu. */void(*reset)(structkvm_vcpu*,conststructcoproc_reg*);-/* Index into vcpu->arch.cp15[], or 0 if we don't need to save it. */+/* Index into vcpu_cp15(vcpu, ...), or 0 if we don't need to save it. */unsignedlongreg;/* Value (usually reset value) */
@@ -357,22 +357,22 @@ static void inject_abt(struct kvm_vcpu *vcpu, bool is_pabt, unsigned long addr)if(is_pabt){/* Set IFAR and IFSR */-vcpu->arch.cp15[c6_IFAR]=addr;-is_lpae=(vcpu->arch.cp15[c2_TTBCR]>>31);+vcpu_cp15(vcpu,c6_IFAR)=addr;+is_lpae=(vcpu_cp15(vcpu,c2_TTBCR)>>31);/* Always give debug fault for now - should give guest a clue */if(is_lpae)-vcpu->arch.cp15[c5_IFSR]=1<<9|0x22;+vcpu_cp15(vcpu,c5_IFSR)=1<<9|0x22;else-vcpu->arch.cp15[c5_IFSR]=2;+vcpu_cp15(vcpu,c5_IFSR)=2;}else{/* !iabt *//* Set DFAR and DFSR */-vcpu->arch.cp15[c6_DFAR]=addr;-is_lpae=(vcpu->arch.cp15[c2_TTBCR]>>31);+vcpu_cp15(vcpu,c6_DFAR)=addr;+is_lpae=(vcpu_cp15(vcpu,c2_TTBCR)>>31);/* Always give debug fault for now - should give guest a clue */if(is_lpae)-vcpu->arch.cp15[c5_DFSR]=1<<9|0x22;+vcpu_cp15(vcpu,c5_DFSR)=1<<9|0x22;else-vcpu->arch.cp15[c5_DFSR]=2;+vcpu_cp15(vcpu,c5_DFSR)=2;}}
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:13
On Thu, Feb 04, 2016 at 11:00:21AM +0000, Marc Zyngier wrote:
Continuing our rework of the CPU context, we now move the CP15
array into the CPU context structure. As this causes quite a bit
of churn, we introduce the vcpu_cp15() macro that abstract the
location of the actual array. This will probably help next time
we have to revisit that code.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:23
In order to expose the various HYP services that are private to
the hypervisor, add a new hyp.h file.
So far, it only contains mundane things such as section annotation
and VA manipulation.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp.h | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
create mode 100644 arch/arm/kvm/hyp/hyp.h
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:23
On Thu, Feb 04, 2016 at 11:00:23AM +0000, Marc Zyngier wrote:
In order to expose the various HYP services that are private to
the hypervisor, add a new hyp.h file.
So far, it only contains mundane things such as section annotation
and VA manipulation.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:24
In order to move system register (CP15, mostly) access to C code,
add a few macros to facilitate this, and minimize the difference
between 32 and 64bit CP15 registers.
This will get heavily used in the following patches.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
From: Christoffer Dall <hidden> Date: 2016-02-10 17:25:39
On Thu, Feb 04, 2016 at 11:00:24AM +0000, Marc Zyngier wrote:
quoted hunk
In order to move system register (CP15, mostly) access to C code,
add a few macros to facilitate this, and minimize the difference
between 32 and 64bit CP15 registers.
This will get heavily used in the following patches.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
I sort of figured that a reviewed-by tag on patches that actually use
these macros would be an implicit review of this code.
not feeling comfortable enough that I read this jibberish perfectly in
isolation, but given that the stuff compiles and works, I'll just ack
it:
Acked-by: Christoffer Dall <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-10 17:32:24
On 10/02/16 17:25, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:24AM +0000, Marc Zyngier wrote:
quoted
In order to move system register (CP15, mostly) access to C code,
add a few macros to facilitate this, and minimize the difference
between 32 and 64bit CP15 registers.
This will get heavily used in the following patches.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
I sort of figured that a reviewed-by tag on patches that actually use
these macros would be an implicit review of this code.
not feeling comfortable enough that I read this jibberish perfectly in
isolation, but given that the stuff compiles and works, I'll just ack
it:
Acked-by: Christoffer Dall <redacted>
Yeah, this is admittedly cryptic. Despite my efforts, I'm still
considering C (and the preprocessor) as an evolved macro-assembler.
I guess next time, I'll rewrite it in OCaml.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:27
This patch shouldn't exist, as we should be able to reuse the
arm64 version for free. I'll get there eventually, but in the
meantime I need a timer ticking.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 8 +++++
arch/arm/kvm/hyp/timer-sr.c | 71 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 80 insertions(+)
create mode 100644 arch/arm/kvm/hyp/timer-sr.c
@@ -0,0 +1,71 @@+/*+*Copyright(C)2012-2015-ARMLtd+*Author:MarcZyngier<marc.zyngier@arm.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#include<clocksource/arm_arch_timer.h>+#include<linux/compiler.h>+#include<linux/kvm_host.h>++#include<asm/kvm_mmu.h>++#include"hyp.h"++/* vcpu is already in the HYP VA space */+void__hyp_text__timer_save_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structarch_timer_cpu*timer=&vcpu->arch.timer_cpu;+u64val;++if(kvm->arch.timer.enabled){+timer->cntv_ctl=read_sysreg(CNTV_CTL);+timer->cntv_cval=read_sysreg(CNTV_CVAL);+}++/* Disable the virtual timer */+write_sysreg(0,CNTV_CTL);++/* Allow physical timer/counter access for the host */+val=read_sysreg(CNTHCTL);+val|=CNTHCTL_EL1PCTEN|CNTHCTL_EL1PCEN;+write_sysreg(val,CNTHCTL);++/* Clear cntvoff for the host */+write_sysreg(0,CNTVOFF);+}++void__hyp_text__timer_restore_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structarch_timer_cpu*timer=&vcpu->arch.timer_cpu;+u64val;++/*+*Disallowphysicaltimeraccessfortheguest+*Physicalcounteraccessisallowed+*/+val=read_sysreg(CNTHCTL);+val&=~CNTHCTL_EL1PCEN;+val|=CNTHCTL_EL1PCTEN;+write_sysreg(val,CNTHCTL);++if(kvm->arch.timer.enabled){+write_sysreg(kvm->arch.timer.cntvoff,CNTVOFF);+write_sysreg(timer->cntv_cval,CNTV_CVAL);+isb();+write_sysreg(timer->cntv_ctl,CNTV_CTL);+}+}
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:42
On Thu, Feb 04, 2016 at 11:00:27AM +0000, Marc Zyngier wrote:
quoted hunk
This patch shouldn't exist, as we should be able to reuse the
arm64 version for free. I'll get there eventually, but in the
meantime I need a timer ticking.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 8 +++++
arch/arm/kvm/hyp/timer-sr.c | 71 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 80 insertions(+)
create mode 100644 arch/arm/kvm/hyp/timer-sr.c
@@ -0,0 +1,71 @@+/*+*Copyright(C)2012-2015-ARMLtd+*Author:MarcZyngier<marc.zyngier@arm.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#include<clocksource/arm_arch_timer.h>+#include<linux/compiler.h>+#include<linux/kvm_host.h>++#include<asm/kvm_mmu.h>++#include"hyp.h"++/* vcpu is already in the HYP VA space */+void__hyp_text__timer_save_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structarch_timer_cpu*timer=&vcpu->arch.timer_cpu;+u64val;++if(kvm->arch.timer.enabled){+timer->cntv_ctl=read_sysreg(CNTV_CTL);+timer->cntv_cval=read_sysreg(CNTV_CVAL);+}++/* Disable the virtual timer */+write_sysreg(0,CNTV_CTL);++/* Allow physical timer/counter access for the host */+val=read_sysreg(CNTHCTL);+val|=CNTHCTL_EL1PCTEN|CNTHCTL_EL1PCEN;+write_sysreg(val,CNTHCTL);++/* Clear cntvoff for the host */+write_sysreg(0,CNTVOFF);
in the asm version we only did this if the timer was enabled, probably
the theory being that only in that case did we mody the offset. But it
should be safe to just clear the cntvoff in any case, right?
From: Marc Zyngier <hidden> Date: 2016-02-10 15:36:40
On 09/02/16 18:42, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:27AM +0000, Marc Zyngier wrote:
quoted
This patch shouldn't exist, as we should be able to reuse the
arm64 version for free. I'll get there eventually, but in the
meantime I need a timer ticking.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 8 +++++
arch/arm/kvm/hyp/timer-sr.c | 71 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 80 insertions(+)
create mode 100644 arch/arm/kvm/hyp/timer-sr.c
@@ -0,0 +1,71 @@+/*+*Copyright(C)2012-2015-ARMLtd+*Author:MarcZyngier<marc.zyngier@arm.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#include<clocksource/arm_arch_timer.h>+#include<linux/compiler.h>+#include<linux/kvm_host.h>++#include<asm/kvm_mmu.h>++#include"hyp.h"++/* vcpu is already in the HYP VA space */+void__hyp_text__timer_save_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structarch_timer_cpu*timer=&vcpu->arch.timer_cpu;+u64val;++if(kvm->arch.timer.enabled){+timer->cntv_ctl=read_sysreg(CNTV_CTL);+timer->cntv_cval=read_sysreg(CNTV_CVAL);+}++/* Disable the virtual timer */+write_sysreg(0,CNTV_CTL);++/* Allow physical timer/counter access for the host */+val=read_sysreg(CNTHCTL);+val|=CNTHCTL_EL1PCTEN|CNTHCTL_EL1PCEN;+write_sysreg(val,CNTHCTL);++/* Clear cntvoff for the host */+write_sysreg(0,CNTVOFF);
in the asm version we only did this if the timer was enabled, probably
the theory being that only in that case did we mody the offset. But it
should be safe to just clear the cntvoff in any case, right?
It is indeed perfectly safe. I've copied the arm64 code into the 32bit
tree, so we get this cntvoff reset (arm64 requires it since it the
virtual counter is used in the vdso), but this doesn't hurt on 32bit either.
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:28
This patch shouldn't exist, as we should be able to reuse the
arm64 version for free. I'll get there eventually, but in the
meantime I need an interrupt controller.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 3 ++
arch/arm/kvm/hyp/vgic-v2-sr.c | 84 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 88 insertions(+)
create mode 100644 arch/arm/kvm/hyp/vgic-v2-sr.c
@@ -0,0 +1,84 @@+/*+*Copyright(C)2012-2015-ARMLtd+*Author:MarcZyngier<marc.zyngier@arm.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#include<linux/compiler.h>+#include<linux/irqchip/arm-gic.h>+#include<linux/kvm_host.h>++#include<asm/kvm_mmu.h>++#include"hyp.h"++/* vcpu is already in the HYP VA space */+void__hyp_text__vgic_v2_save_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structvgic_v2_cpu_if*cpu_if=&vcpu->arch.vgic_cpu.vgic_v2;+structvgic_dist*vgic=&kvm->arch.vgic;+void__iomem*base=kern_hyp_va(vgic->vctrl_base);+u32eisr0,eisr1,elrsr0,elrsr1;+inti,nr_lr;++if(!base)+return;++nr_lr=vcpu->arch.vgic_cpu.nr_lr;+cpu_if->vgic_vmcr=readl_relaxed(base+GICH_VMCR);+cpu_if->vgic_misr=readl_relaxed(base+GICH_MISR);+eisr0=readl_relaxed(base+GICH_EISR0);+elrsr0=readl_relaxed(base+GICH_ELRSR0);+if(unlikely(nr_lr>32)){+eisr1=readl_relaxed(base+GICH_EISR1);+elrsr1=readl_relaxed(base+GICH_ELRSR1);+}else{+eisr1=elrsr1=0;+}+#ifdef CONFIG_CPU_BIG_ENDIAN+cpu_if->vgic_eisr=((u64)eisr0<<32)|eisr1;+cpu_if->vgic_elrsr=((u64)elrsr0<<32)|elrsr1;+#else+cpu_if->vgic_eisr=((u64)eisr1<<32)|eisr0;+cpu_if->vgic_elrsr=((u64)elrsr1<<32)|elrsr0;+#endif+cpu_if->vgic_apr=readl_relaxed(base+GICH_APR);++writel_relaxed(0,base+GICH_HCR);++for(i=0;i<nr_lr;i++)+cpu_if->vgic_lr[i]=readl_relaxed(base+GICH_LR0+(i*4));+}++/* vcpu is already in the HYP VA space */+void__hyp_text__vgic_v2_restore_state(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+structvgic_v2_cpu_if*cpu_if=&vcpu->arch.vgic_cpu.vgic_v2;+structvgic_dist*vgic=&kvm->arch.vgic;+void__iomem*base=kern_hyp_va(vgic->vctrl_base);+inti,nr_lr;++if(!base)+return;++writel_relaxed(cpu_if->vgic_hcr,base+GICH_HCR);+writel_relaxed(cpu_if->vgic_vmcr,base+GICH_VMCR);+writel_relaxed(cpu_if->vgic_apr,base+GICH_APR);++nr_lr=vcpu->arch.vgic_cpu.nr_lr;+for(i=0;i<nr_lr;i++)+writel_relaxed(cpu_if->vgic_lr[i],base+GICH_LR0+(i*4));+}
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:47
On Thu, Feb 04, 2016 at 11:00:28AM +0000, Marc Zyngier wrote:
This patch shouldn't exist, as we should be able to reuse the
arm64 version for free. I'll get there eventually, but in the
meantime I need an interrupt controller.
Signed-off-by: Marc Zyngier <redacted>
---
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:29
This is almost a copy/paste of the existing version, with a couple
of subtle differences:
- Only write to FPEXC once on the save path
- Add an isb when enabling VFP access
The patch also defines a few sysreg accessors and a __vfp_enabled
predicate that test the VFP trapping state.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 13 +++++++++
arch/arm/kvm/hyp/vfp.S | 68 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 82 insertions(+)
create mode 100644 arch/arm/kvm/hyp/vfp.S
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:52
On Thu, Feb 04, 2016 at 11:00:29AM +0000, Marc Zyngier wrote:
This is almost a copy/paste of the existing version, with a couple
of subtle differences:
- Only write to FPEXC once on the save path
- Add an isb when enabling VFP access
The patch also defines a few sysreg accessors and a __vfp_enabled
predicate that test the VFP trapping state.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:30
Banked registers are one of the many perks of the 32bit architecture,
and the world switch needs to cope with it.
This requires some "special" accessors, as these are not accessed
using a standard coprocessor instruction.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/banked-sr.c | 77 ++++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 11 +++++++
3 files changed, 89 insertions(+)
create mode 100644 arch/arm/kvm/hyp/banked-sr.c
From: Christoffer Dall <hidden> Date: 2016-02-09 18:42:57
On Thu, Feb 04, 2016 at 11:00:30AM +0000, Marc Zyngier wrote:
Banked registers are one of the many perks of the 32bit architecture,
and the world switch needs to cope with it.
This requires some "special" accessors, as these are not accessed
using a standard coprocessor instruction.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:31
Add the very minimal piece of code that is now required to jump
into the guest (and return from it). This code is only concerned
with save/restoring the USR registers (r0-r12+lr for the guest,
r4-r12+lr for the host), as everything else is dealt with in C
(VFP is another matter though).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/entry.S | 70 +++++++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 ++
3 files changed, 73 insertions(+)
create mode 100644 arch/arm/kvm/hyp/entry.S
From: Christoffer Dall <hidden> Date: 2016-02-09 18:44:45
On Thu, Feb 04, 2016 at 11:00:31AM +0000, Marc Zyngier wrote:
quoted hunk
Add the very minimal piece of code that is now required to jump
into the guest (and return from it). This code is only concerned
with save/restoring the USR registers (r0-r12+lr for the guest,
r4-r12+lr for the host), as everything else is dealt with in C
(VFP is another matter though).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/entry.S | 70 +++++++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 ++
3 files changed, 73 insertions(+)
create mode 100644 arch/arm/kvm/hyp/entry.S
this really relies on offsetof(struct pt_regs, ARM_r0) == 0, which I
guess will likely never change, but given there's both a kernel and uapi
version of struct pt_regs, are we sure about this?
From: Marc Zyngier <hidden> Date: 2016-02-10 15:48:35
On 09/02/16 18:44, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:31AM +0000, Marc Zyngier wrote:
quoted
Add the very minimal piece of code that is now required to jump
into the guest (and return from it). This code is only concerned
with save/restoring the USR registers (r0-r12+lr for the guest,
r4-r12+lr for the host), as everything else is dealt with in C
(VFP is another matter though).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/entry.S | 70 +++++++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 ++
3 files changed, 73 insertions(+)
create mode 100644 arch/arm/kvm/hyp/entry.S
this really relies on offsetof(struct pt_regs, ARM_r0) == 0, which I
guess will likely never change, but given there's both a kernel and uapi
version of struct pt_regs, are we sure about this?
If they did diverge, a lot of things would just break. arm64 does have
different types between user and kernel, but the userspace version is
guaranteed to be a strict prefix of the kernel one. I believe arm would
have to enforce the same thing if it changed.
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:32
Similar to the arm64 version, add the code that deals with VFP traps,
re-enabling VFP, save/restoring the registers and resuming the guest.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/entry.S | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
From: Christoffer Dall <hidden> Date: 2016-02-09 18:44:49
On Thu, Feb 04, 2016 at 11:00:32AM +0000, Marc Zyngier wrote:
Similar to the arm64 version, add the code that deals with VFP traps,
re-enabling VFP, save/restoring the registers and resuming the guest.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:33
The new world switch implementation is modeled after the arm64 one,
calling the various save/restore functions in turn, and having as
little state as possible.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 7 +++
arch/arm/kvm/hyp/switch.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 144 insertions(+)
create mode 100644 arch/arm/kvm/hyp/switch.c
@@ -0,0 +1,136 @@+/*+*Copyright(C)2015-ARMLtd+*Author:MarcZyngier<marc.zyngier@arm.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#include<asm/kvm_asm.h>+#include"hyp.h"++__asm__(".arch_extension virt");++staticvoid__hyp_text__activate_traps(structkvm_vcpu*vcpu,u32*fpexc)+{+u32val;++/*+*WeareabouttosetHCPTR.TCP10/11totrapallfloatingpoint+*registeraccessestoHYP,however,theARMARMclearlystatesthat+*trapsareonlytakentoHYPiftheoperationwouldnototherwise+*traptoSVC.Therefore,alwaysmakesurethatfor32-bitguests,+*wesetFPEXC.ENtopreventtrapstoSVC,whensettingtheTCPbits.+*/+val=read_sysreg(VFP_FPEXC);+*fpexc=val;+if(!(val&FPEXC_EN)){+write_sysreg(val|FPEXC_EN,VFP_FPEXC);+isb();+}++write_sysreg(vcpu->arch.hcr|vcpu->arch.irq_lines,HCR);+/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */+write_sysreg(HSTR_T(15),HSTR);+write_sysreg(HCPTR_TTA|HCPTR_TCP(10)|HCPTR_TCP(11),HCPTR);+val=read_sysreg(HDCR);+write_sysreg(val|HDCR_TPM|HDCR_TPMCR,HDCR);+}++staticvoid__hyp_text__deactivate_traps(structkvm_vcpu*vcpu)+{+u32val;++write_sysreg(0,HCR);+write_sysreg(0,HSTR);+val=read_sysreg(HDCR);+write_sysreg(val&~(HDCR_TPM|HDCR_TPMCR),HDCR);+write_sysreg(0,HCPTR);+}++staticvoid__hyp_text__activate_vm(structkvm_vcpu*vcpu)+{+structkvm*kvm=kern_hyp_va(vcpu->kvm);+write_sysreg(kvm->arch.vttbr,VTTBR);+write_sysreg(vcpu->arch.midr,VMIDR);+}++staticvoid__hyp_text__deactivate_vm(structkvm_vcpu*vcpu)+{+write_sysreg(0,VTTBR);+write_sysreg(read_sysreg(MIDR),VMIDR);+}++staticvoid__hyp_text__vgic_save_state(structkvm_vcpu*vcpu)+{+__vgic_v2_save_state(vcpu);+}++staticvoid__hyp_text__vgic_restore_state(structkvm_vcpu*vcpu)+{+__vgic_v2_restore_state(vcpu);+}++staticint__hyp_text__guest_run(structkvm_vcpu*vcpu)+{+structkvm_cpu_context*host_ctxt;+structkvm_cpu_context*guest_ctxt;+boolfp_enabled;+u64exit_code;+u32fpexc;++vcpu=kern_hyp_va(vcpu);+write_sysreg(vcpu,HTPIDR);++host_ctxt=kern_hyp_va(vcpu->arch.host_cpu_context);+guest_ctxt=&vcpu->arch.ctxt;++__sysreg_save_state(host_ctxt);+__banked_save_state(host_ctxt);++__activate_traps(vcpu,&fpexc);+__activate_vm(vcpu);++__vgic_restore_state(vcpu);+__timer_restore_state(vcpu);++__sysreg_restore_state(guest_ctxt);+__banked_restore_state(guest_ctxt);++/* Jump in the fire! */+exit_code=__guest_enter(vcpu,host_ctxt);+/* And we're baaack! */++fp_enabled=__vfp_enabled();++__banked_save_state(guest_ctxt);+__sysreg_save_state(guest_ctxt);+__timer_save_state(vcpu);+__vgic_save_state(vcpu);++__deactivate_traps(vcpu);+__deactivate_vm(vcpu);++__banked_restore_state(host_ctxt);+__sysreg_restore_state(host_ctxt);++if(fp_enabled){+__vfp_save_state(&guest_ctxt->vfp);+__vfp_restore_state(&host_ctxt->vfp);+}++write_sysreg(fpexc,VFP_FPEXC);++returnexit_code;+}++__alias(__guest_run)int__weak__kvm_vcpu_run(structkvm_vcpu*vcpu);
From: Christoffer Dall <hidden> Date: 2016-02-09 18:44:55
On Thu, Feb 04, 2016 at 11:00:33AM +0000, Marc Zyngier wrote:
quoted hunk
The new world switch implementation is modeled after the arm64 one,
calling the various save/restore functions in turn, and having as
little state as possible.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 7 +++
arch/arm/kvm/hyp/switch.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 144 insertions(+)
create mode 100644 arch/arm/kvm/hyp/switch.c
From: Marc Zyngier <hidden> Date: 2016-02-10 16:00:10
On 09/02/16 18:44, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:33AM +0000, Marc Zyngier wrote:
quoted
The new world switch implementation is modeled after the arm64 one,
calling the various save/restore functions in turn, and having as
little state as possible.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp.h | 7 +++
arch/arm/kvm/hyp/switch.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 144 insertions(+)
create mode 100644 arch/arm/kvm/hyp/switch.c
Nit: This is called VPIDR in v7 and VMPIDR_EL2 in v8 IIUC. Should we
refer to it by one of those names instead?
Seems to be VPIDR in all cases, actually (I stupidly made it consistent,
silly me!). I'll definitely fix that, thanks for noticing it!
Cheers,
M.
--
Jazz is not dead. It just smells funny...
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:34
On guest exit, we must take care of populating our fault data
structure so that the host code can handle it. This includes
resolving the IPA for permission faults, which can result in
restarting the guest.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp.h | 4 ++++
arch/arm/kvm/hyp/switch.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 58 insertions(+)
@@ -80,6 +80,56 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)__vgic_v2_restore_state(vcpu);}+staticbool__hyp_text__populate_fault_info(structkvm_vcpu*vcpu)+{+u32hsr=read_sysreg(HSR);+u8ec=hsr>>HSR_EC_SHIFT;+u32hpfar,far;++vcpu->arch.fault.hsr=hsr;++if(ec==HSR_EC_IABT)+far=read_sysreg(HIFAR);+elseif(ec==HSR_EC_DABT)+far=read_sysreg(HDFAR);+else+returntrue;++/*+*B3.13.5ReportingexceptionstakentotheNon-securePL2mode:+*+*Abortonthestage2translationforamemoryaccessfroma+*Non-securePL1orPL0mode:+*+*ForanyAccessflagfaultorTranslationfault,andalsoforany+*Permissionfaultonthestage2translationofamemoryaccess+*madeaspartofatranslationtablewalkforastage1translation,+*theHPFARholdstheIPAthatcausedthefault.Otherwise,theHPFAR+*isUNKNOWN.+*/+if(!(hsr&HSR_DABT_S1PTW)&&(hsr&HSR_FSC_TYPE)==FSC_PERM){+u64par,tmp;++par=read_sysreg(PAR);+write_sysreg(far,ATS1CPR);+isb();++tmp=read_sysreg(PAR);+write_sysreg(par,PAR);++if(unlikely(tmp&1))+returnfalse;/* Translation failed, back to guest */++hpfar=((tmp>>12)&((1UL<<28)-1))<<4;+}else{+hpfar=read_sysreg(HPFAR);+}++vcpu->arch.fault.hxfar=far;+vcpu->arch.fault.hpfar=hpfar;+returntrue;+}+staticint__hyp_text__guest_run(structkvm_vcpu*vcpu){structkvm_cpu_context*host_ctxt;
@@ -107,9 +157,13 @@ static int __hyp_text __guest_run(struct kvm_vcpu *vcpu)__banked_restore_state(guest_ctxt);/* Jump in the fire! */+again:exit_code=__guest_enter(vcpu,host_ctxt);/* And we're baaack! */+if(exit_code==ARM_EXCEPTION_HVC&&!__populate_fault_info(vcpu))+gotoagain;+fp_enabled=__vfp_enabled();__banked_save_state(guest_ctxt);
From: Christoffer Dall <hidden> Date: 2016-02-09 18:44:59
On Thu, Feb 04, 2016 at 11:00:34AM +0000, Marc Zyngier wrote:
On guest exit, we must take care of populating our fault data
structure so that the host code can handle it. This includes
resolving the IPA for permission faults, which can result in
restarting the guest.
Signed-off-by: Marc Zyngier <redacted>
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:35
This part is almost entierely borrowed from the existing code, just
slightly simplifying the HYP function call (as we now save SPSR_hyp
in the world switch).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp-entry.S | 157 +++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 +
3 files changed, 160 insertions(+)
create mode 100644 arch/arm/kvm/hyp/hyp-entry.S
@@ -0,0 +1,157 @@+/*+*Copyright (C)2012-VirtualOpenSystemsandColumbiaUniversity+*Author:ChristofferDall<c.dall@virtualopensystems.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicense,version2,as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram;ifnot,writetotheFreeSoftware+*Foundation,51FranklinStreet,FifthFloor,Boston,MA02110-1301,USA.+*/++#include <linux/linkage.h>+#include <asm/kvm_arm.h>+#include <asm/kvm_asm.h>++.arch_extensionvirt++.text+.pushsection.hyp.text,"ax"++.macroload_vcpureg+mrcp15,4,\reg,c13,c0,2@HTPIDR+.endm++/********************************************************************+*Hypervisorexceptionvectorandhandlers+*+*+*TheKVM/ARMHypervisorABIisdefinedasfollows:+*+*EntrytoHypmodefromthehostkernelwillhappen_only_whenanHVC+*instructionisissuedsincealltrapsaredisabledwhenrunningthehost+*kernelaspertheHyp-modeinitializationatboottime.+*+*HVCinstructionscauseatraptothevectorpage+offset0x14(seehyp_hvc+*below)whentheHVCinstructioniscalledfromSVCmode (i.e.aguestorthe+*hostkernel)andtheycauseatraptothevectorpage+offset0x8whenHVC+*instructionsarecalledfromwithinHyp-mode.+*+*Hyp-ABI:CallingHYP-modefunctionsfromhost (inSVCmode):+*SwitchingtoHypmodeisdonethroughasimpleHVC#0 instruction. The+*exceptionvectorcodewillcheckthattheHVCcomesfromVMID==0.+*-r0containsapointertoaHYPfunction+*-r1,r2,andr3containargumentstotheabovefunction.+*-TheHYPfunctionwillbecalledwithitsargumentsinr0,r1andr2.+*OnHYPfunctionreturn,wereturndirectlytoSVC.+*+*NotethattheaboveisusedtoexecutecodeinHyp-modefromahost-kernel+*pointofview,andisadifferentconceptfromperformingaworld-switchand+*executingguestcodeSVCmode (withaVMID!=0).+*/++.align5+__hyp_vector:+.global__hyp_vector+__kvm_hyp_vector:+.weak__kvm_hyp_vector++@Hyp-modeexceptionvector+W(b)hyp_reset+W(b)hyp_undef+W(b)hyp_svc+W(b)hyp_pabt+W(b)hyp_dabt+W(b)hyp_hvc+W(b)hyp_irq+W(b)hyp_fiq++.macroinvalid_vectorlabel,cause+.align+\label:b.+.endm++invalid_vectorhyp_reset+invalid_vectorhyp_undef+invalid_vectorhyp_svc+invalid_vectorhyp_pabt+invalid_vectorhyp_dabt+invalid_vectorhyp_fiq++hyp_hvc:+/*+*Gettinghereiseitherbecauseofatrapfromaguest,+*orfromexecutingHVCfromthehostkernel,whichmeans+*"do something in Hyp mode".+*/+push{r0,r1,r2}++@Checksyndromeregister+mrcp15,4,r1,c5,c2,0@HSR+lsrr0,r1,#HSR_EC_SHIFT+cmpr0,#HSR_EC_HVC+bneguest_trap@NotHVCinstr.++/*+*Let's check if the HVC came from VMID 0 and allow simple+*switchtoHypmode+*/+mrrcp15,6,r0,r2,c2+lsrr2,r2,#16+andr2,r2,#0xff+cmpr2,#0+bneguest_trap@GuestcalledHVC++/*+*GettingheremeanshostcalledHVC,weshiftparametersandbranch+*toHypfunction.+*/+pop{r0,r1,r2}++/*Checkfor__hyp_get_vectors*/+cmpr0,#-1+mrceqp15,4,r0,c12,c0,0@getHVBAR+beq1f++push{lr}++movlr,r0+movr0,r1+movr1,r2+movr2,r3++THUMB(orrlr,#1)+blxlr@CalltheHYPfunction++pop{lr}+1:eret++guest_trap:+load_vcpur0@LoadVCPUpointertor0++@Checkifweneedthefaultinformation+lsrr1,r1,#HSR_EC_SHIFT+#ifdef CONFIG_VFPv3+cmpr1,#HSR_EC_CP_0_13+beq__vfp_guest_restore+#endif++movr1,#ARM_EXCEPTION_HVC+b__guest_exit++hyp_irq:+push{r0,r1,r2}+movr1,#ARM_EXCEPTION_IRQ+load_vcpur0@LoadVCPUpointertor0+b__guest_exit++.ltorg++.popsection
From: Christoffer Dall <hidden> Date: 2016-02-09 17:00:42
On Thu, Feb 04, 2016 at 11:00:35AM +0000, Marc Zyngier wrote:
quoted hunk
This part is almost entierely borrowed from the existing code, just
slightly simplifying the HYP function call (as we now save SPSR_hyp
in the world switch).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp-entry.S | 157 +++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 +
3 files changed, 160 insertions(+)
create mode 100644 arch/arm/kvm/hyp/hyp-entry.S
@@ -0,0 +1,157 @@+/*+*Copyright (C)2012-VirtualOpenSystemsandColumbiaUniversity+*Author:ChristofferDall<c.dall@virtualopensystems.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicense,version2,as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram;ifnot,writetotheFreeSoftware+*Foundation,51FranklinStreet,FifthFloor,Boston,MA02110-1301,USA.+*/++#include <linux/linkage.h>+#include <asm/kvm_arm.h>+#include <asm/kvm_asm.h>++.arch_extensionvirt++.text+.pushsection.hyp.text,"ax"++.macroload_vcpureg+mrcp15,4,\reg,c13,c0,2@HTPIDR+.endm++/********************************************************************+*Hypervisorexceptionvectorandhandlers+*+*+*TheKVM/ARMHypervisorABIisdefinedasfollows:+*+*EntrytoHypmodefromthehostkernelwillhappen_only_whenanHVC+*instructionisissuedsincealltrapsaredisabledwhenrunningthehost+*kernelaspertheHyp-modeinitializationatboottime.+*+*HVCinstructionscauseatraptothevectorpage+offset0x14(seehyp_hvc+*below)whentheHVCinstructioniscalledfromSVCmode (i.e.aguestorthe+*hostkernel)andtheycauseatraptothevectorpage+offset0x8whenHVC+*instructionsarecalledfromwithinHyp-mode.+*+*Hyp-ABI:CallingHYP-modefunctionsfromhost (inSVCmode):+*SwitchingtoHypmodeisdonethroughasimpleHVC#0 instruction. The+*exceptionvectorcodewillcheckthattheHVCcomesfromVMID==0.+*-r0containsapointertoaHYPfunction+*-r1,r2,andr3containargumentstotheabovefunction.+*-TheHYPfunctionwillbecalledwithitsargumentsinr0,r1andr2.+*OnHYPfunctionreturn,wereturndirectlytoSVC.+*+*NotethattheaboveisusedtoexecutecodeinHyp-modefromahost-kernel+*pointofview,andisadifferentconceptfromperformingaworld-switchand+*executingguestcodeSVCmode (withaVMID!=0).+*/++.align5+__hyp_vector:+.global__hyp_vector+__kvm_hyp_vector:+.weak__kvm_hyp_vector++@Hyp-modeexceptionvector+W(b)hyp_reset+W(b)hyp_undef+W(b)hyp_svc+W(b)hyp_pabt+W(b)hyp_dabt+W(b)hyp_hvc+W(b)hyp_irq+W(b)hyp_fiq++.macroinvalid_vectorlabel,cause+.align+\label:b.+.endm++invalid_vectorhyp_reset+invalid_vectorhyp_undef+invalid_vectorhyp_svc+invalid_vectorhyp_pabt+invalid_vectorhyp_dabt+invalid_vectorhyp_fiq++hyp_hvc:+/*+*Gettinghereiseitherbecauseofatrapfromaguest,+*orfromexecutingHVCfromthehostkernel,whichmeans+*"do something in Hyp mode".+*/+push{r0,r1,r2}++@Checksyndromeregister+mrcp15,4,r1,c5,c2,0@HSR+lsrr0,r1,#HSR_EC_SHIFT+cmpr0,#HSR_EC_HVC+bneguest_trap@NotHVCinstr.++/*+*Let's check if the HVC came from VMID 0 and allow simple+*switchtoHypmode+*/+mrrcp15,6,r0,r2,c2+lsrr2,r2,#16+andr2,r2,#0xff+cmpr2,#0+bneguest_trap@GuestcalledHVC++/*+*GettingheremeanshostcalledHVC,weshiftparametersandbranch+*toHypfunction.+*/+pop{r0,r1,r2}++/*Checkfor__hyp_get_vectors*/+cmpr0,#-1+mrceqp15,4,r0,c12,c0,0@getHVBAR+beq1f++push{lr}++movlr,r0+movr0,r1+movr1,r2+movr2,r3++THUMB(orrlr,#1)+blxlr@CalltheHYPfunction++pop{lr}+1:eret++guest_trap:+load_vcpur0@LoadVCPUpointertor0++@Checkifweneedthefaultinformation
nit: this is not about faults at this point, so this comment should
either go or be reworded to "let's check if we trapped on guest VFP
access"
and I think the lsr can be moved into the ifdef as well.
From: Marc Zyngier <hidden> Date: 2016-02-10 16:02:14
On 09/02/16 17:00, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:35AM +0000, Marc Zyngier wrote:
quoted
This part is almost entierely borrowed from the existing code, just
slightly simplifying the HYP function call (as we now save SPSR_hyp
in the world switch).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp-entry.S | 157 +++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 +
3 files changed, 160 insertions(+)
create mode 100644 arch/arm/kvm/hyp/hyp-entry.S
@@ -0,0 +1,157 @@+/*+*Copyright (C)2012-VirtualOpenSystemsandColumbiaUniversity+*Author:ChristofferDall<c.dall@virtualopensystems.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicense,version2,as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram;ifnot,writetotheFreeSoftware+*Foundation,51FranklinStreet,FifthFloor,Boston,MA02110-1301,USA.+*/++#include <linux/linkage.h>+#include <asm/kvm_arm.h>+#include <asm/kvm_asm.h>++.arch_extensionvirt++.text+.pushsection.hyp.text,"ax"++.macroload_vcpureg+mrcp15,4,\reg,c13,c0,2@HTPIDR+.endm++/********************************************************************+*Hypervisorexceptionvectorandhandlers+*+*+*TheKVM/ARMHypervisorABIisdefinedasfollows:+*+*EntrytoHypmodefromthehostkernelwillhappen_only_whenanHVC+*instructionisissuedsincealltrapsaredisabledwhenrunningthehost+*kernelaspertheHyp-modeinitializationatboottime.+*+*HVCinstructionscauseatraptothevectorpage+offset0x14(seehyp_hvc+*below)whentheHVCinstructioniscalledfromSVCmode (i.e.aguestorthe+*hostkernel)andtheycauseatraptothevectorpage+offset0x8whenHVC+*instructionsarecalledfromwithinHyp-mode.+*+*Hyp-ABI:CallingHYP-modefunctionsfromhost (inSVCmode):+*SwitchingtoHypmodeisdonethroughasimpleHVC#0 instruction. The+*exceptionvectorcodewillcheckthattheHVCcomesfromVMID==0.+*-r0containsapointertoaHYPfunction+*-r1,r2,andr3containargumentstotheabovefunction.+*-TheHYPfunctionwillbecalledwithitsargumentsinr0,r1andr2.+*OnHYPfunctionreturn,wereturndirectlytoSVC.+*+*NotethattheaboveisusedtoexecutecodeinHyp-modefromahost-kernel+*pointofview,andisadifferentconceptfromperformingaworld-switchand+*executingguestcodeSVCmode (withaVMID!=0).+*/++.align5+__hyp_vector:+.global__hyp_vector+__kvm_hyp_vector:+.weak__kvm_hyp_vector++@Hyp-modeexceptionvector+W(b)hyp_reset+W(b)hyp_undef+W(b)hyp_svc+W(b)hyp_pabt+W(b)hyp_dabt+W(b)hyp_hvc+W(b)hyp_irq+W(b)hyp_fiq++.macroinvalid_vectorlabel,cause+.align+\label:b.+.endm++invalid_vectorhyp_reset+invalid_vectorhyp_undef+invalid_vectorhyp_svc+invalid_vectorhyp_pabt+invalid_vectorhyp_dabt+invalid_vectorhyp_fiq++hyp_hvc:+/*+*Gettinghereiseitherbecauseofatrapfromaguest,+*orfromexecutingHVCfromthehostkernel,whichmeans+*"do something in Hyp mode".+*/+push{r0,r1,r2}++@Checksyndromeregister+mrcp15,4,r1,c5,c2,0@HSR+lsrr0,r1,#HSR_EC_SHIFT+cmpr0,#HSR_EC_HVC+bneguest_trap@NotHVCinstr.++/*+*Let's check if the HVC came from VMID 0 and allow simple+*switchtoHypmode+*/+mrrcp15,6,r0,r2,c2+lsrr2,r2,#16+andr2,r2,#0xff+cmpr2,#0+bneguest_trap@GuestcalledHVC++/*+*GettingheremeanshostcalledHVC,weshiftparametersandbranch+*toHypfunction.+*/+pop{r0,r1,r2}++/*Checkfor__hyp_get_vectors*/+cmpr0,#-1+mrceqp15,4,r0,c12,c0,0@getHVBAR+beq1f++push{lr}++movlr,r0+movr0,r1+movr1,r2+movr2,r3++THUMB(orrlr,#1)+blxlr@CalltheHYPfunction++pop{lr}+1:eret++guest_trap:+load_vcpur0@LoadVCPUpointertor0++@Checkifweneedthefaultinformation
nit: this is not about faults at this point, so this comment should
either go or be reworded to "let's check if we trapped on guest VFP
access"
and I think the lsr can be moved into the ifdef as well.
Yes, both good points.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
From: Christoffer Dall <hidden> Date: 2016-02-10 17:23:21
On Wed, Feb 10, 2016 at 04:02:14PM +0000, Marc Zyngier wrote:
On 09/02/16 17:00, Christoffer Dall wrote:
quoted
On Thu, Feb 04, 2016 at 11:00:35AM +0000, Marc Zyngier wrote:
quoted
This part is almost entierely borrowed from the existing code, just
slightly simplifying the HYP function call (as we now save SPSR_hyp
in the world switch).
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/Makefile | 1 +
arch/arm/kvm/hyp/hyp-entry.S | 157 +++++++++++++++++++++++++++++++++++++++++++
arch/arm/kvm/hyp/hyp.h | 2 +
3 files changed, 160 insertions(+)
create mode 100644 arch/arm/kvm/hyp/hyp-entry.S
@@ -0,0 +1,157 @@+/*+*Copyright (C)2012-VirtualOpenSystemsandColumbiaUniversity+*Author:ChristofferDall<c.dall@virtualopensystems.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicense,version2,as+*publishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,+*butWITHOUTANYWARRANTY;withouteventheimpliedwarrantyof+*MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.Seethe+*GNUGeneralPublicLicenseformoredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense+*alongwiththisprogram;ifnot,writetotheFreeSoftware+*Foundation,51FranklinStreet,FifthFloor,Boston,MA02110-1301,USA.+*/++#include <linux/linkage.h>+#include <asm/kvm_arm.h>+#include <asm/kvm_asm.h>++.arch_extensionvirt++.text+.pushsection.hyp.text,"ax"++.macroload_vcpureg+mrcp15,4,\reg,c13,c0,2@HTPIDR+.endm++/********************************************************************+*Hypervisorexceptionvectorandhandlers+*+*+*TheKVM/ARMHypervisorABIisdefinedasfollows:+*+*EntrytoHypmodefromthehostkernelwillhappen_only_whenanHVC+*instructionisissuedsincealltrapsaredisabledwhenrunningthehost+*kernelaspertheHyp-modeinitializationatboottime.+*+*HVCinstructionscauseatraptothevectorpage+offset0x14(seehyp_hvc+*below)whentheHVCinstructioniscalledfromSVCmode (i.e.aguestorthe+*hostkernel)andtheycauseatraptothevectorpage+offset0x8whenHVC+*instructionsarecalledfromwithinHyp-mode.+*+*Hyp-ABI:CallingHYP-modefunctionsfromhost (inSVCmode):+*SwitchingtoHypmodeisdonethroughasimpleHVC#0 instruction. The+*exceptionvectorcodewillcheckthattheHVCcomesfromVMID==0.+*-r0containsapointertoaHYPfunction+*-r1,r2,andr3containargumentstotheabovefunction.+*-TheHYPfunctionwillbecalledwithitsargumentsinr0,r1andr2.+*OnHYPfunctionreturn,wereturndirectlytoSVC.+*+*NotethattheaboveisusedtoexecutecodeinHyp-modefromahost-kernel+*pointofview,andisadifferentconceptfromperformingaworld-switchand+*executingguestcodeSVCmode (withaVMID!=0).+*/++.align5+__hyp_vector:+.global__hyp_vector+__kvm_hyp_vector:+.weak__kvm_hyp_vector++@Hyp-modeexceptionvector+W(b)hyp_reset+W(b)hyp_undef+W(b)hyp_svc+W(b)hyp_pabt+W(b)hyp_dabt+W(b)hyp_hvc+W(b)hyp_irq+W(b)hyp_fiq++.macroinvalid_vectorlabel,cause+.align+\label:b.+.endm++invalid_vectorhyp_reset+invalid_vectorhyp_undef+invalid_vectorhyp_svc+invalid_vectorhyp_pabt+invalid_vectorhyp_dabt+invalid_vectorhyp_fiq++hyp_hvc:+/*+*Gettinghereiseitherbecauseofatrapfromaguest,+*orfromexecutingHVCfromthehostkernel,whichmeans+*"do something in Hyp mode".+*/+push{r0,r1,r2}++@Checksyndromeregister+mrcp15,4,r1,c5,c2,0@HSR+lsrr0,r1,#HSR_EC_SHIFT+cmpr0,#HSR_EC_HVC+bneguest_trap@NotHVCinstr.++/*+*Let's check if the HVC came from VMID 0 and allow simple+*switchtoHypmode+*/+mrrcp15,6,r0,r2,c2+lsrr2,r2,#16+andr2,r2,#0xff+cmpr2,#0+bneguest_trap@GuestcalledHVC++/*+*GettingheremeanshostcalledHVC,weshiftparametersandbranch+*toHypfunction.+*/+pop{r0,r1,r2}++/*Checkfor__hyp_get_vectors*/+cmpr0,#-1+mrceqp15,4,r0,c12,c0,0@getHVBAR+beq1f++push{lr}++movlr,r0+movr0,r1+movr1,r2+movr2,r3++THUMB(orrlr,#1)+blxlr@CalltheHYPfunction++pop{lr}+1:eret++guest_trap:+load_vcpur0@LoadVCPUpointertor0++@Checkifweneedthefaultinformation
nit: this is not about faults at this point, so this comment should
either go or be reworded to "let's check if we trapped on guest VFP
access"
and I think the lsr can be moved into the ifdef as well.
Yes, both good points.
fixing that, you can also have my
Reviewed-by: Christoffer Dall <redacted>
if I didn't give it already.
-Christoffer
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:36
Instead of spinning forever, let's "properly" handle any unexpected
exception ("properly" meaning "print a spat on the console and die").
This has proved useful quite a few times...
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp-entry.S | 28 +++++++++++++++++++++-------
arch/arm/kvm/hyp/switch.c | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 59 insertions(+), 7 deletions(-)
From: Christoffer Dall <hidden> Date: 2016-02-09 18:45:03
On Thu, Feb 04, 2016 at 11:00:36AM +0000, Marc Zyngier wrote:
quoted hunk
Instead of spinning forever, let's "properly" handle any unexpected
exception ("properly" meaning "print a spat on the console and die").
This has proved useful quite a few times...
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp-entry.S | 28 +++++++++++++++++++++-------
arch/arm/kvm/hyp/switch.c | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 59 insertions(+), 7 deletions(-)
From: Marc Zyngier <hidden> Date: 2016-02-10 16:03:48
On 09/02/16 18:45, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:36AM +0000, Marc Zyngier wrote:
quoted
Instead of spinning forever, let's "properly" handle any unexpected
exception ("properly" meaning "print a spat on the console and die").
This has proved useful quite a few times...
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/kvm/hyp/hyp-entry.S | 28 +++++++++++++++++++++-------
arch/arm/kvm/hyp/switch.c | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 59 insertions(+), 7 deletions(-)
From: Marc Zyngier <hidden> Date: 2016-02-04 11:00:37
Having u64 as the kvm_call_hyp return type is problematic, as
it forces all kind of tricks for the return values from HYP
to be promoted to 64bit (LE has the LSB in r0, and BE has them
in r1).
Since the only user of the return value is perfectly happy with
a 32bit value, let's make kvm_call_hyp return an unsigned long,
which is 32bit on ARM.
This solves yet another headache.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_host.h | 2 +-
arch/arm/kvm/interrupts.S | 10 ++--------
2 files changed, 3 insertions(+), 9 deletions(-)
From: Christoffer Dall <hidden> Date: 2016-02-09 18:28:28
On Thu, Feb 04, 2016 at 11:00:37AM +0000, Marc Zyngier wrote:
Having u64 as the kvm_call_hyp return type is problematic, as
it forces all kind of tricks for the return values from HYP
to be promoted to 64bit (LE has the LSB in r0, and BE has them
in r1).
Since the only user of the return value is perfectly happy with
a 32bit value, let's make kvm_call_hyp return an unsigned long,
which is 32bit on ARM.
I wonder why I ever did this as a u64...
should the arm64 counterpart be modified to an unsigned long as well?
quoted hunk
This solves yet another headache.
Signed-off-by: Marc Zyngier <redacted>
---
arch/arm/include/asm/kvm_host.h | 2 +-
arch/arm/kvm/interrupts.S | 10 ++--------
2 files changed, 3 insertions(+), 9 deletions(-)
From: Marc Zyngier <hidden> Date: 2016-02-10 16:07:07
On 09/02/16 18:28, Christoffer Dall wrote:
On Thu, Feb 04, 2016 at 11:00:37AM +0000, Marc Zyngier wrote:
quoted
Having u64 as the kvm_call_hyp return type is problematic, as
it forces all kind of tricks for the return values from HYP
to be promoted to 64bit (LE has the LSB in r0, and BE has them
in r1).
Since the only user of the return value is perfectly happy with
a 32bit value, let's make kvm_call_hyp return an unsigned long,
which is 32bit on ARM.
I wonder why I ever did this as a u64...
Probably to cater for the largest possible return value, before we
started looking at BE... ;-)
should the arm64 counterpart be modified to an unsigned long as well?
That'd be a sensible change.
Thanks,
M.
--
Jazz is not dead. It just smells funny...