From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
This series enables Linux guests running on Hyper-V on ARM64
hardware. New ARM64-specific code in arch/arm64/hyperv initializes
Hyper-V, including its interrupts and hypercall mechanism.
Existing architecture independent drivers for Hyper-V's VMbus and
synthetic devices just work when built for ARM64. Hyper-V code is
built and included in the image and modules only if CONFIG_HYPERV
is enabled.
The seven patches are organized as follows:
1) Update include/linux/arm-smccc.h to provide an HVC wrapper
variant that returns results in other than X0 thru X3
2) Add definitions and functions for making Hyper-V hypercalls
and getting/setting virtual processor registers provided by
Hyper-V
3) Add architecture specific definitions needed by the
architecture independent Hyper-V clocksource driver in
drivers/clocksource/hyperv_timer.c. Update the clocksource
driver to be initialized on ARM64.
4) Add functions needed by the arch independent VMbus driver
for reporting a panic to Hyper-V and as stubs for the kexec
and crash handlers.
5) Add Hyper-V initialization code and utility functions that
report Hyper-v status.
6) Export screen_info so it may be used by the Hyper-V frame buffer
driver built as a module. It is already exported for x86,
powerpc, and alpha architectures.
7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64.
Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests
with 4K/16K/64K page size. Linux guests with this ARM64 enablement
code work with all three supported ARM64 page sizes.
The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has
x86/x64-specific code and is not being built for ARM64. Fixing
this driver to enable vPCI devices on ARM64 will be done later.
In a few cases, terminology from the x86/x64 world has been carried
over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the
x86/x64 terminology and has not replaced it with something more
generic, so the code uses the Hyper-V terminology. This will be
fixed when Hyper-V updates the usage in the TLFS.
This patch set is based on the hyperv-next branch of the code tree
https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/
Changes in v9:
* Added Patch 1 to enable making an SMCCC compliant hypercall
that returns results in other than registers X0 thru X3, per
version 1.2 and later of the SMCCC spec.
* Using the ability to return results in registers X6 and X7,
converted hv_get_vpreg_128() to use a "fast" hypercall that
passes inputs and outputs in registers, and in doing so eliminated
a lot of memory allocation complexity.
* Cleaned up some extra blank lines and use of spaces in aligning
local variables. [Sunil Muthuswamy]
* Based on discussion about future directions, reverted the
population of hv_vp_index array to use a cpuhp state instead
of a hypercall, which is like it was in v7 and earlier.
Changes in v8:
* Removed a lot of code based on refactoring the boundary between
arch independent and arch dependent code for Hyper-V, per comments
from Arnd Bergmann. The removed code was either duplicated on
the x86 side, or has been folded into architecture independent
code as not really being architecture dependent.
* Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann]
* Reworked the approach to Hyper-V initialization. The functionality
is the same, but is now structured like the Xen code with an early
init function called in setup_arch() and an early initcall to
finish the initialization. [Arnd Bergmann]
Changes in v7:
* Separately upstreamed split of hyperv-tlfs.h into arch dependent
and independent versions. In this patch set, update the ARM64
hyperv-tlfs.h to include architecture independent definitions.
This approach eliminates a lot of lines of otherwise duplicated
code on the ARM64 side.
* Break ARM64 mshyperv.h into smaller pieces. Have an initial
baseline, and add code along with patches for a particular
functional area. [Marc Zyngier]
* In mshyperv.h, use static inline functions instead of #defines
where possible. [Arnd Bergmann]
* Use VMbus INTID obtained from ACPI DSDT instead of hardcoding.
The STIMER INTID is still hardcoded because it is needed
before Linux has initialized the ACPI subsystem, so it can't
be obtained from the DSDT. Wedging it into the GTDT seems
dubious, so was not done. [Marc Zyngier]
* Update Hyper-V page size allocation functions to use
alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd
Bergmann]
* Various other minor changes based on feedback and to rebase
to latest linux-next [Marc Zyngier and Arnd Bergmann]
Changes in v6:
* Use SMCCC hypercall interface instead of direct invocation
of HVC instruction and the Hyper-V hypercall interface
[Marc Zyngier]
* Reimplemented functions to alloc/free Hyper-V size pages
using kmalloc/kfree since kmalloc now guarantees alignment of
power of 2 size allocations [Marc Zyngier]
* Export screen_info in arm64 architecture so it can be used
by the Hyper-V buffer driver built as a module
* Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c
to better reflect its content
* Fixed the bit position of certain feature flags presented by
Hyper-V to the guest. The bit positions on ARM64 don't match
the position on x86 like originally thought.
* Minor fixups to rebase to 5.6-rc5 linux-next
Changes in v5:
* Minor fixups to rebase to 5.4-rc1 linux-next
Changes in v4:
* Moved clock-related code into an architecture independent
Hyper-V clocksource driver that is already upstream. Clock
related code is removed from this patch set except for the
ARM64 specific interrupt handler. [Marc Zyngier]
* Separately upstreamed the split of mshyperv.h into arch independent
and arch dependent portions. The arch independent portion has been
removed from this patch set.
* Divided patch #2 of the series into multiple smaller patches
[Marc Zyngier]
* Changed a dozen or so smaller things based on feedback
[Marc Zyngier, Will Deacon]
* Added functions to alloc/free Hyper-V size pages for use by
drivers for Hyper-V synthetic devices when updated to not assume
guest page size and Hyper-v page size are the same
Changes in v3:
* Added initialization of hv_vp_index array like was recently
added on x86 branch [KY Srinivasan]
* Changed Hyper-V ARM64 register symbols to be all uppercase
instead of mixed case [KY Srinivasan]
* Separated mshyperv.h into two files, one architecture
independent and one architecture dependent. After this code
is upstream, will make changes to the x86 code to use the
architecture independent file and remove duplication. And
once we have a multi-architecture Hyper-V TLFS, will do a
separate patch to split hyperv-tlfs.h in the same way.
[KY Srinivasan]
* Minor tweaks to rebase to latest linux-next code
Changes in v2:
* Removed patch to implement slow_virt_to_phys() on ARM64.
Use of slow_virt_to_phys() in arch independent Hyper-V
drivers has been eliminated by commit 6ba34171bcbd
("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()")
* Minor tweaks to rebase to latest linux-next code
*** BLURB HERE ***
Michael Kelley (7):
smccc: Add HVC call variant with result registers other than 0 thru 3
arm64: hyperv: Add Hyper-V hypercall and register access utilities
arm64: hyperv: Add Hyper-V clocksource/clockevent support
arm64: hyperv: Add kexec and panic handlers
arm64: hyperv: Initialize hypervisor on boot
arm64: efi: Export screen_info
Drivers: hv: Enable Hyper-V code to be built on ARM64
MAINTAINERS | 3 +
arch/arm64/Kbuild | 1 +
arch/arm64/hyperv/Makefile | 2 +
arch/arm64/hyperv/hv_core.c | 178 +++++++++++++++++++++++++++++++++++
arch/arm64/hyperv/mshyperv.c | 173 ++++++++++++++++++++++++++++++++++
arch/arm64/include/asm/hyperv-tlfs.h | 69 ++++++++++++++
arch/arm64/include/asm/mshyperv.h | 72 ++++++++++++++
arch/arm64/kernel/efi.c | 1 +
arch/arm64/kernel/setup.c | 4 +
drivers/clocksource/hyperv_timer.c | 14 +++
drivers/hv/Kconfig | 3 +-
include/linux/arm-smccc.h | 29 ++++--
12 files changed, 542 insertions(+), 7 deletions(-)
create mode 100644 arch/arm64/hyperv/Makefile
create mode 100644 arch/arm64/hyperv/hv_core.c
create mode 100644 arch/arm64/hyperv/mshyperv.c
create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
create mode 100644 arch/arm64/include/asm/mshyperv.h
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:46
Hypercalls to Hyper-V on ARM64 may return results in registers other
than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
the caller to specify which 3 registers are returned in addition to X0.
Signed-off-by: Michael Kelley <redacted>
---
There are several ways to support returning results from registers
other than X0 thru X3, and Hyper-V usage should be compatible with
whatever the maintainers prefer. What's implemented in this patch
may be the most flexible, but it has the downside of not being a
true function interface in that args 0 thru 2 must be fixed strings,
and not general "C" expressions.
Other alternatives include:
* Create a variant that hard codes to return X5 thru X7, though
in the future there may be Hyper-V hypercalls that need a
different hard-coded variant.
* Return all of X0 thru X7 in a larger result structure. That
approach may execute more memory stores, but performance is unlikely
to be an issue for the Hyper-V hypercalls that would use it.
However, it's possible in the future that Hyper-V results might
be beyond X7, as allowed by the SMCCC v1.3 spec.
* The macro __arm_smccc_1_1() could be cloned in Hyper-V specific
code and modified to meet Hyper-V specific needs, but this seems
undesirable since Hyper-V is operating within the v1.2 spec.
In any of these cases, the call might be renamed from "_1_1_" to
"_1_2_" to reflect conformance to the later spec version.
include/linux/arm-smccc.h | 29 +++++++++++++++++++++++------
1 file changed, 23 insertions(+), 6 deletions(-)
From: Mark Rutland <mark.rutland@arm.com> Date: 2021-03-24 16:56:10
Hi Michael,
On Mon, Mar 08, 2021 at 11:57:13AM -0800, Michael Kelley wrote:
Hypercalls to Hyper-V on ARM64 may return results in registers other
than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
the caller to specify which 3 registers are returned in addition to X0.
Signed-off-by: Michael Kelley <redacted>
---
There are several ways to support returning results from registers
other than X0 thru X3, and Hyper-V usage should be compatible with
whatever the maintainers prefer. What's implemented in this patch
may be the most flexible, but it has the downside of not being a
true function interface in that args 0 thru 2 must be fixed strings,
and not general "C" expressions.
For the benefit of others here, SMCCCv1.2 allows:
* SMC64/HVC64 to use all of x1-x17 for both parameters and return values
* SMC32/HVC32 to use all of r1-r7 for both parameters and return values
The rationale for this was to make it possible to pass a large number of
arguments in one call without the hypervisor/firmware needing to access
the memory of the caller.
My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
functions which read all the permitted argument registers from a struct,
and write all the permitted result registers to a struct, leaving it to
callers to set those up and decompose them.
That way we only have to write one implementation that all callers can
use, which'll be far easier to maintain. I suspect that in general the
cost of temporarily bouncing the values through memory will be dominated
by whatever the hypervisor/firmware is going to do, and if it's not we
can optimize that away in future.
Other alternatives include:
* Create a variant that hard codes to return X5 thru X7, though
in the future there may be Hyper-V hypercalls that need a
different hard-coded variant.
* Return all of X0 thru X7 in a larger result structure. That
approach may execute more memory stores, but performance is unlikely
to be an issue for the Hyper-V hypercalls that would use it.
However, it's possible in the future that Hyper-V results might
be beyond X7, as allowed by the SMCCC v1.3 spec.
As above, something of this sort would be my preferred approach.
Thanks,
Mark.
quoted hunk
* The macro __arm_smccc_1_1() could be cloned in Hyper-V specific
code and modified to meet Hyper-V specific needs, but this seems
undesirable since Hyper-V is operating within the v1.2 spec.
In any of these cases, the call might be renamed from "_1_1_" to
"_1_2_" to reflect conformance to the later spec version.
include/linux/arm-smccc.h | 29 +++++++++++++++++++++++------
1 file changed, 23 insertions(+), 6 deletions(-)
From: Michael Kelley <hidden> Date: 2021-03-25 04:56:36
From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
Hi Michael,
On Mon, Mar 08, 2021 at 11:57:13AM -0800, Michael Kelley wrote:
quoted
Hypercalls to Hyper-V on ARM64 may return results in registers other
than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
the caller to specify which 3 registers are returned in addition to X0.
Signed-off-by: Michael Kelley <redacted>
---
There are several ways to support returning results from registers
other than X0 thru X3, and Hyper-V usage should be compatible with
whatever the maintainers prefer. What's implemented in this patch
may be the most flexible, but it has the downside of not being a
true function interface in that args 0 thru 2 must be fixed strings,
and not general "C" expressions.
For the benefit of others here, SMCCCv1.2 allows:
* SMC64/HVC64 to use all of x1-x17 for both parameters and return values
* SMC32/HVC32 to use all of r1-r7 for both parameters and return values
The rationale for this was to make it possible to pass a large number of
arguments in one call without the hypervisor/firmware needing to access
the memory of the caller.
My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
functions which read all the permitted argument registers from a struct,
and write all the permitted result registers to a struct, leaving it to
callers to set those up and decompose them.
That way we only have to write one implementation that all callers can
use, which'll be far easier to maintain. I suspect that in general the
cost of temporarily bouncing the values through memory will be dominated
by whatever the hypervisor/firmware is going to do, and if it's not we
can optimize that away in future.
Thanks for the feedback, and I'm working on implementing this approach.
But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
which gives us 15 registers as parameters and 15 registers as return
values, instead of the 18 each allowed by SMCCC v1.2. I will continue
with the 15 register limit for now, unless someone knows a way to exceed
that. The alternative would be to go to pure assembly language.
I'll post a standalone RFC patch when I have something that works. My
C pre-processor wizardry is limited, so others will probably know some
tricks that can improve on my first cut.
Michael
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Mark Rutland <mark.rutland@arm.com> Date: 2021-03-25 09:57:42
On Thu, Mar 25, 2021 at 04:55:51AM +0000, Michael Kelley wrote:
From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
quoted
For the benefit of others here, SMCCCv1.2 allows:
* SMC64/HVC64 to use all of x1-x17 for both parameters and return values
* SMC32/HVC32 to use all of r1-r7 for both parameters and return values
The rationale for this was to make it possible to pass a large number of
arguments in one call without the hypervisor/firmware needing to access
the memory of the caller.
My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
functions which read all the permitted argument registers from a struct,
and write all the permitted result registers to a struct, leaving it to
callers to set those up and decompose them.
That way we only have to write one implementation that all callers can
use, which'll be far easier to maintain. I suspect that in general the
cost of temporarily bouncing the values through memory will be dominated
by whatever the hypervisor/firmware is going to do, and if it's not we
can optimize that away in future.
Thanks for the feedback, and I'm working on implementing this approach.
But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
which gives us 15 registers as parameters and 15 registers as return
values, instead of the 18 each allowed by SMCCC v1.2. I will continue
with the 15 register limit for now, unless someone knows a way to exceed
that. The alternative would be to go to pure assembly language.
I realise in retrospect this is not clear, but when I said "assembly
functions" I had meant raw assembly functions rather than inline
assembly.
We already have __arm_smccc_smc and __arm_smccc_hvc assembly functions
in arch/{arm,arm64}/kernel/smccc-call.S, and I'd expected we'd add the
full fat SMCCCv1.2 variants there.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Michael Kelley <hidden> Date: 2021-03-25 17:20:22
From: Mark Rutland <mark.rutland@arm.com> Sent: Thursday, March 25, 2021 2:56 AM
On Thu, Mar 25, 2021 at 04:55:51AM +0000, Michael Kelley wrote:
quoted
From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
quoted
For the benefit of others here, SMCCCv1.2 allows:
* SMC64/HVC64 to use all of x1-x17 for both parameters and return values
* SMC32/HVC32 to use all of r1-r7 for both parameters and return values
The rationale for this was to make it possible to pass a large number of
arguments in one call without the hypervisor/firmware needing to access
the memory of the caller.
My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
functions which read all the permitted argument registers from a struct,
and write all the permitted result registers to a struct, leaving it to
callers to set those up and decompose them.
That way we only have to write one implementation that all callers can
use, which'll be far easier to maintain. I suspect that in general the
cost of temporarily bouncing the values through memory will be dominated
by whatever the hypervisor/firmware is going to do, and if it's not we
can optimize that away in future.
Thanks for the feedback, and I'm working on implementing this approach.
But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
which gives us 15 registers as parameters and 15 registers as return
values, instead of the 18 each allowed by SMCCC v1.2. I will continue
with the 15 register limit for now, unless someone knows a way to exceed
that. The alternative would be to go to pure assembly language.
I realise in retrospect this is not clear, but when I said "assembly
functions" I had meant raw assembly functions rather than inline
assembly.
We already have __arm_smccc_smc and __arm_smccc_hvc assembly functions
in arch/{arm,arm64}/kernel/smccc-call.S, and I'd expected we'd add the
full fat SMCCCv1.2 variants there.
FWIW, here's an inline assembly version that I have working. On the plus
side, gcc does a decent job of optimizing. It doesn't store to memory any
result registers that aren't consumed by the caller. On the downside, it's
limited to 15 args and 15 results as noted previously. So it doesn't meet
your goal of fully implementing the v1.2 spec. Also, all 15 input arguments
must be initialized or gcc complains about using uninitialized values.
This version should handle both the 32-bit and 64-bit worlds, though I've
only tested in the 64-bit world.
I've made the input and output structures be arrays rather than listing
each register as a separate field. Either approach should work, and I'm not
sure what the tradeoffs are.
But if building on what Sudeep Holla has already done with raw assembly
is preferred, I'm OK with that as well.
Michael
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on
ARM64, causing the Hyper-V specific code to be built.
Signed-off-by: Michael Kelley <redacted>
Reviewed-by: Sunil Muthuswamy <redacted>
---
drivers/hv/Kconfig | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
The Hyper-V frame buffer driver may be built as a module, and
it needs access to screen_info. So export screen_info.
Signed-off-by: Michael Kelley <redacted>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/kernel/efi.c | 1 +
1 file changed, 1 insertion(+)
@@ -55,6 +55,7 @@ static __init pteval_t create_mapping_protection(efi_memory_desc_t *md)/* we will fill this structure from the stub, so don't put it in .bss */structscreen_infoscreen_info__section(".data");+EXPORT_SYMBOL(screen_info);int__initefi_create_mapping(structmm_struct*mm,efi_memory_desc_t*md){
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
Add ARM64-specific code to initialize the Hyper-V
hypervisor when booting as a guest VM. Provide functions
and data structures indicating hypervisor status that
are needed by VMbus driver.
This code is built only when CONFIG_HYPERV is enabled.
Signed-off-by: Michael Kelley <redacted>
---
arch/arm64/hyperv/mshyperv.c | 119 ++++++++++++++++++++++++++++++++++++++
arch/arm64/include/asm/mshyperv.h | 6 ++
arch/arm64/kernel/setup.c | 4 ++
3 files changed, 129 insertions(+)
@@ -14,6 +14,125 @@#include<linux/types.h>#include<linux/export.h>#include<linux/ptrace.h>+#include<linux/errno.h>+#include<linux/acpi.h>+#include<linux/version.h>+#include<linux/cpuhotplug.h>+#include<linux/slab.h>+#include<linux/cpumask.h>+#include<asm/mshyperv.h>++staticboolhyperv_initialized;+structms_hyperv_infoms_hyperv__ro_after_init;+EXPORT_SYMBOL_GPL(ms_hyperv);++u32*hv_vp_index;+EXPORT_SYMBOL_GPL(hv_vp_index);++u32hv_max_vp_index;+EXPORT_SYMBOL_GPL(hv_max_vp_index);++staticinthv_cpu_init(unsignedintcpu)+{+hv_vp_index[cpu]=hv_get_vpreg(HV_REGISTER_VP_INDEX);+return0;+}++void__inithyperv_early_init(void)+{+structhv_get_vp_registers_outputresult;+u32a,b,c,d;+u64guest_id;++/*+*Ifwe'reinaVMonHyper-V,theACPIhypervisor_idfieldwill+*havethestring"MsHyperV".+*/+if(strncmp((char*)&acpi_gbl_FADT.hypervisor_id,"MsHyperV",8))+return;++/* Setup the guest ID */+guest_id=generate_guest_id(0,LINUX_VERSION_CODE,0);+hv_set_vpreg(HV_REGISTER_GUEST_OSID,guest_id);++/* Get the features and hints from Hyper-V */+hv_get_vpreg_128(HV_REGISTER_FEATURES,&result);+ms_hyperv.features=result.as32.a;+ms_hyperv.misc_features=result.as32.c;++hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS,&result);+ms_hyperv.hints=result.as32.a;++pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",+ms_hyperv.features,ms_hyperv.hints,ms_hyperv.misc_features);++/*+*IfHyper-Vhascrashnotifications,setcrash_kexec_post_notifiers+*sothatwewillreportthepanictoHyper-Vbeforerunningkdump.+*/+if(ms_hyperv.misc_features&HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)+crash_kexec_post_notifiers=true;++/* Get information about the Hyper-V host version */+hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION,&result);+a=result.as32.a;+b=result.as32.b;+c=result.as32.c;+d=result.as32.d;+pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",+b>>16,b&0xFFFF,a,d&0xFFFFFF,c,d>>24);++hyperv_initialized=true;+}++staticint__inithyperv_init(void)+{+inti;++/* Allocate and initialize percpu VP index array */+hv_max_vp_index=num_possible_cpus();+hv_vp_index=kmalloc_array(hv_max_vp_index,sizeof(*hv_vp_index),+GFP_KERNEL);+if(!hv_vp_index){+hv_max_vp_index=0;+return-ENOMEM;+}++for(i=0;i<hv_max_vp_index;i++)+hv_vp_index[i]=VP_INVAL;++if(cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,"arm64/hyperv_init:online",+hv_cpu_init,NULL)<0){+hv_max_vp_index=0;+kfree(hv_vp_index);+hv_vp_index=NULL;+return-EINVAL;+}++return0;+}++early_initcall(hyperv_init);++/* This routine is called before kexec/kdump. It does required cleanup. */+voidhyperv_cleanup(void)+{+hv_set_vpreg(HV_REGISTER_GUEST_OSID,0);++}+EXPORT_SYMBOL_GPL(hyperv_cleanup);++boolhv_is_hyperv_initialized(void)+{+returnhyperv_initialized;+}+EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);++boolhv_is_hibernation_supported(void)+{+returnfalse;+}+EXPORT_SYMBOL_GPL(hv_is_hibernation_supported);/**TheVMbushandlerfunctionsareno-opsonARM64because
@@ -340,6 +341,9 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)if(acpi_disabled)unflatten_device_tree();+/* Do after acpi_boot_table_init() so local FADT is available */+hyperv_early_init();+bootmem_init();kasan_init();
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
@@ -340,6 +341,9 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) if (acpi_disabled) unflatten_device_tree();+ /* Do after acpi_boot_table_init() so local FADT is available */+ hyperv_early_init();+ bootmem_init(); kasan_init();--
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
Add function to inform Hyper-V about a guest panic.
Also add functions to set up and remove kexec and panic
handlers, which are currently unused on ARM64 but are
called from architecture independent code in the VMbus
driver.
This code is built only when CONFIG_HYPERV is enabled.
Signed-off-by: Michael Kelley <redacted>
Reviewed-by: Sunil Muthuswamy <redacted>
---
arch/arm64/hyperv/Makefile | 2 +-
arch/arm64/hyperv/hv_core.c | 52 ++++++++++++++++++++++++++++++++++++++++++
arch/arm64/hyperv/mshyperv.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 107 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/hyperv/mshyperv.c
@@ -124,3 +124,55 @@ u64 hv_get_vpreg(u32 msr)returnoutput.as64.low;}EXPORT_SYMBOL_GPL(hv_get_vpreg);++/*+*hyperv_report_panic-reportapanictoHyper-V.Thisfunctionuses+*theolderversionoftheHyper-Vinterfacethatadmittedlydoesn't+*passenoughinformationtobeusefulbeyondjustrecordingthe+*occurrenceofapanic.Theparallelhv_kmsg_dump()usesthe+*newinterfacethatallowsreporting4Kbytesofdata,whichismuch+*moreuseful.Hyper-VonARM64alwayssupportsthenewerinterface,but+*weretainsupportfortheolderversionbecausethesysadminisallowed+*todisablethenewerversionviasysctlincaseofinformationsecurity+*concernsaboutthemoreverboseversion.+*/+voidhyperv_report_panic(structpt_regs*regs,longerr,boolin_die)+{+staticboolpanic_reported;+u64guest_id;++/* Don't report a panic to Hyper-V if we're not going to panic */+if(in_die&&!panic_on_oops)+return;++/*+*Weprefertoreportpanicon'die'chainaswehaveproper+*registerstoreport,butifwemissit(e.g.onBUG())weneed+*toreportiton'panic'.+*+*Callingcodeinthe'die'and'panic'pathsensuresthatonly+*oneCPUisrunningthiscode,sonoatomicityisneeded.+*/+if(panic_reported)+return;+panic_reported=true;++guest_id=hv_get_vpreg(HV_REGISTER_GUEST_OSID);++/*+*Hyper-Vprovidestheabilitytostoreonly5values.+*Pickthepassedinerrorvalue,theguest_id,andthePC.+*Thefirsttwogeneralregistersareaddedarbitrarily.+*/+hv_set_vpreg(HV_REGISTER_CRASH_P0,err);+hv_set_vpreg(HV_REGISTER_CRASH_P1,guest_id);+hv_set_vpreg(HV_REGISTER_CRASH_P2,regs->pc);+hv_set_vpreg(HV_REGISTER_CRASH_P3,regs->regs[0]);+hv_set_vpreg(HV_REGISTER_CRASH_P4,regs->regs[1]);++/*+*LetHyper-Vknowthereiscrashdataavailable+*/+hv_set_vpreg(HV_REGISTER_CRASH_CTL,HV_CRASH_CTL_CRASH_NOTIFY);+}+EXPORT_SYMBOL_GPL(hyperv_report_panic);
From: Michael Kelley <hidden> Date: 2021-03-08 19:58:48
hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level
Functional Spec (TLFS), and #includes the architecture-independent
part of hyperv-tlfs.h in include/asm-generic. The published TLFS
is distinctly oriented to x86/x64, so the ARM64-specific
hyperv-tlfs.h includes information for ARM64 that is not yet formally
published. The TLFS is available here:
docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs
mshyperv.h defines Linux-specific structures and routines for
interacting with Hyper-V on ARM64, and #includes the architecture-
independent part of mshyperv.h in include/asm-generic.
Use these definitions to provide utility functions to make
Hyper-V hypercalls and to get and set Hyper-V provided
registers associated with a virtual processor.
Signed-off-by: Michael Kelley <redacted>
---
MAINTAINERS | 3 +
arch/arm64/Kbuild | 1 +
arch/arm64/hyperv/Makefile | 2 +
arch/arm64/hyperv/hv_core.c | 126 +++++++++++++++++++++++++++++++++++
arch/arm64/include/asm/hyperv-tlfs.h | 69 +++++++++++++++++++
arch/arm64/include/asm/mshyperv.h | 54 +++++++++++++++
6 files changed, 255 insertions(+)
create mode 100644 arch/arm64/hyperv/Makefile
create mode 100644 arch/arm64/hyperv/hv_core.c
create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
create mode 100644 arch/arm64/include/asm/mshyperv.h
From: Michael Kelley <hidden> Date: 2021-03-24 15:55:44
From: Michael Kelley <redacted> Sent: Monday, March 8, 2021 11:57 AM
This series enables Linux guests running on Hyper-V on ARM64
hardware. New ARM64-specific code in arch/arm64/hyperv initializes
Hyper-V, including its interrupts and hypercall mechanism.
Existing architecture independent drivers for Hyper-V's VMbus and
synthetic devices just work when built for ARM64. Hyper-V code is
built and included in the image and modules only if CONFIG_HYPERV
is enabled.
ARM64 maintainers --
What are the prospects for getting your review and Ack on this patch set?
We're wanting to get the Hyper-V support on ARM64 finally accepted upstream.
Previous comments should be addressed in this revision, with perhaps a
remaining discussion point around the alternate SMCCC hypercall interface
in Patch 1 that makes use of changes in v1.2 of the SMCCC spec. There are
several viable approaches that I've noted in the patch, depending on
your preferences.
Michael
The seven patches are organized as follows:
1) Update include/linux/arm-smccc.h to provide an HVC wrapper
variant that returns results in other than X0 thru X3
2) Add definitions and functions for making Hyper-V hypercalls
and getting/setting virtual processor registers provided by
Hyper-V
3) Add architecture specific definitions needed by the
architecture independent Hyper-V clocksource driver in
drivers/clocksource/hyperv_timer.c. Update the clocksource
driver to be initialized on ARM64.
4) Add functions needed by the arch independent VMbus driver
for reporting a panic to Hyper-V and as stubs for the kexec
and crash handlers.
5) Add Hyper-V initialization code and utility functions that
report Hyper-v status.
6) Export screen_info so it may be used by the Hyper-V frame buffer
driver built as a module. It is already exported for x86,
powerpc, and alpha architectures.
7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64.
Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests
with 4K/16K/64K page size. Linux guests with this ARM64 enablement
code work with all three supported ARM64 page sizes.
The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has
x86/x64-specific code and is not being built for ARM64. Fixing
this driver to enable vPCI devices on ARM64 will be done later.
In a few cases, terminology from the x86/x64 world has been carried
over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the
x86/x64 terminology and has not replaced it with something more
generic, so the code uses the Hyper-V terminology. This will be
fixed when Hyper-V updates the usage in the TLFS.
This patch set is based on the hyperv-next branch of the code tree
https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/
Changes in v9:
* Added Patch 1 to enable making an SMCCC compliant hypercall
that returns results in other than registers X0 thru X3, per
version 1.2 and later of the SMCCC spec.
* Using the ability to return results in registers X6 and X7,
converted hv_get_vpreg_128() to use a "fast" hypercall that
passes inputs and outputs in registers, and in doing so eliminated
a lot of memory allocation complexity.
* Cleaned up some extra blank lines and use of spaces in aligning
local variables. [Sunil Muthuswamy]
* Based on discussion about future directions, reverted the
population of hv_vp_index array to use a cpuhp state instead
of a hypercall, which is like it was in v7 and earlier.
Changes in v8:
* Removed a lot of code based on refactoring the boundary between
arch independent and arch dependent code for Hyper-V, per comments
from Arnd Bergmann. The removed code was either duplicated on
the x86 side, or has been folded into architecture independent
code as not really being architecture dependent.
* Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann]
* Reworked the approach to Hyper-V initialization. The functionality
is the same, but is now structured like the Xen code with an early
init function called in setup_arch() and an early initcall to
finish the initialization. [Arnd Bergmann]
Changes in v7:
* Separately upstreamed split of hyperv-tlfs.h into arch dependent
and independent versions. In this patch set, update the ARM64
hyperv-tlfs.h to include architecture independent definitions.
This approach eliminates a lot of lines of otherwise duplicated
code on the ARM64 side.
* Break ARM64 mshyperv.h into smaller pieces. Have an initial
baseline, and add code along with patches for a particular
functional area. [Marc Zyngier]
* In mshyperv.h, use static inline functions instead of #defines
where possible. [Arnd Bergmann]
* Use VMbus INTID obtained from ACPI DSDT instead of hardcoding.
The STIMER INTID is still hardcoded because it is needed
before Linux has initialized the ACPI subsystem, so it can't
be obtained from the DSDT. Wedging it into the GTDT seems
dubious, so was not done. [Marc Zyngier]
* Update Hyper-V page size allocation functions to use
alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd
Bergmann]
* Various other minor changes based on feedback and to rebase
to latest linux-next [Marc Zyngier and Arnd Bergmann]
Changes in v6:
* Use SMCCC hypercall interface instead of direct invocation
of HVC instruction and the Hyper-V hypercall interface
[Marc Zyngier]
* Reimplemented functions to alloc/free Hyper-V size pages
using kmalloc/kfree since kmalloc now guarantees alignment of
power of 2 size allocations [Marc Zyngier]
* Export screen_info in arm64 architecture so it can be used
by the Hyper-V buffer driver built as a module
* Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c
to better reflect its content
* Fixed the bit position of certain feature flags presented by
Hyper-V to the guest. The bit positions on ARM64 don't match
the position on x86 like originally thought.
* Minor fixups to rebase to 5.6-rc5 linux-next
Changes in v5:
* Minor fixups to rebase to 5.4-rc1 linux-next
Changes in v4:
* Moved clock-related code into an architecture independent
Hyper-V clocksource driver that is already upstream. Clock
related code is removed from this patch set except for the
ARM64 specific interrupt handler. [Marc Zyngier]
* Separately upstreamed the split of mshyperv.h into arch independent
and arch dependent portions. The arch independent portion has been
removed from this patch set.
* Divided patch #2 of the series into multiple smaller patches
[Marc Zyngier]
* Changed a dozen or so smaller things based on feedback
[Marc Zyngier, Will Deacon]
* Added functions to alloc/free Hyper-V size pages for use by
drivers for Hyper-V synthetic devices when updated to not assume
guest page size and Hyper-v page size are the same
Changes in v3:
* Added initialization of hv_vp_index array like was recently
added on x86 branch [KY Srinivasan]
* Changed Hyper-V ARM64 register symbols to be all uppercase
instead of mixed case [KY Srinivasan]
* Separated mshyperv.h into two files, one architecture
independent and one architecture dependent. After this code
is upstream, will make changes to the x86 code to use the
architecture independent file and remove duplication. And
once we have a multi-architecture Hyper-V TLFS, will do a
separate patch to split hyperv-tlfs.h in the same way.
[KY Srinivasan]
* Minor tweaks to rebase to latest linux-next code
Changes in v2:
* Removed patch to implement slow_virt_to_phys() on ARM64.
Use of slow_virt_to_phys() in arch independent Hyper-V
drivers has been eliminated by commit 6ba34171bcbd
("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()")
* Minor tweaks to rebase to latest linux-next code
*** BLURB HERE ***
Michael Kelley (7):
smccc: Add HVC call variant with result registers other than 0 thru 3
arm64: hyperv: Add Hyper-V hypercall and register access utilities
arm64: hyperv: Add Hyper-V clocksource/clockevent support
arm64: hyperv: Add kexec and panic handlers
arm64: hyperv: Initialize hypervisor on boot
arm64: efi: Export screen_info
Drivers: hv: Enable Hyper-V code to be built on ARM64
MAINTAINERS | 3 +
arch/arm64/Kbuild | 1 +
arch/arm64/hyperv/Makefile | 2 +
arch/arm64/hyperv/hv_core.c | 178 +++++++++++++++++++++++++++++++++++
arch/arm64/hyperv/mshyperv.c | 173 ++++++++++++++++++++++++++++++++++
arch/arm64/include/asm/hyperv-tlfs.h | 69 ++++++++++++++
arch/arm64/include/asm/mshyperv.h | 72 ++++++++++++++
arch/arm64/kernel/efi.c | 1 +
arch/arm64/kernel/setup.c | 4 +
drivers/clocksource/hyperv_timer.c | 14 +++
drivers/hv/Kconfig | 3 +-
include/linux/arm-smccc.h | 29 ++++--
12 files changed, 542 insertions(+), 7 deletions(-)
create mode 100644 arch/arm64/hyperv/Makefile
create mode 100644 arch/arm64/hyperv/hv_core.c
create mode 100644 arch/arm64/hyperv/mshyperv.c
create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
create mode 100644 arch/arm64/include/asm/mshyperv.h
--
1.8.3.1
From: Michael Kelley <hidden> Date: 2021-04-05 17:45:57
From: Michael Kelley <redacted>
Sent: Wednesday, March 24, 2021 8:55 AM
To: will@kernel.org; catalin.marinas@arm.com; Mark Rutland <Mark.Rutland@arm.com>;
lorenzo.pieralisi@arm.com; sudeep.holla@arm.com
Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; linux-
hyperv@vger.kernel.org; linux-efi@vger.kernel.org; arnd@arndb.de; wei.liu@kernel.org;
ardb@kernel.org; daniel.lezcano@linaro.org; KY Srinivasan [off-list ref]
Subject: RE: [PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64
From: Michael Kelley <redacted> Sent: Monday, March 8, 2021 11:57 AM
quoted
This series enables Linux guests running on Hyper-V on ARM64
hardware. New ARM64-specific code in arch/arm64/hyperv initializes
Hyper-V, including its interrupts and hypercall mechanism.
Existing architecture independent drivers for Hyper-V's VMbus and
synthetic devices just work when built for ARM64. Hyper-V code is
built and included in the image and modules only if CONFIG_HYPERV
is enabled.
ARM64 maintainers --
What are the prospects for getting your review and Ack on this patch set?
We're wanting to get the Hyper-V support on ARM64 finally accepted upstream.
Previous comments should be addressed in this revision, with perhaps a
remaining discussion point around the alternate SMCCC hypercall interface
in Patch 1 that makes use of changes in v1.2 of the SMCCC spec. There are
several viable approaches that I've noted in the patch, depending on
your preferences.
Michael
Thanks, Mark, for jumping in on the SMCCC hypercall interface. But I'm still
looking for feedback or ACKs on the other patches in the series. There's only
one place in Patch 2 of the series that needs the SMCCC v1.2 interface, and I'd
like to be able to respond to any remaining issues with the other patches
while the SMCCC details are finished up.
Michael
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel