Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.
From: Steve Rutherford <hidden>
Date: 2021-02-05 00:58:12
Also in:
lkml
On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra [off-list ref] wrote:
quoted hunk ↗ jump to hunk
From: Ashish Kalra <ashish.kalra@amd.com> Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check for host-side support for SEV live migration. Also add a new custom MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration feature. Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> --- Documentation/virt/kvm/cpuid.rst | 5 +++++ Documentation/virt/kvm/msr.rst | 12 ++++++++++++ arch/x86/include/uapi/asm/kvm_para.h | 4 ++++ arch/x86/kvm/svm/sev.c | 13 +++++++++++++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/svm/svm.h | 2 ++ 6 files changed, 52 insertions(+)diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst index cf62162d4be2..0bdb6cdb12d3 100644 --- a/Documentation/virt/kvm/cpuid.rst +++ b/Documentation/virt/kvm/cpuid.rst@@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID 15 guest checks this feature bit before using extended destination ID bits in MSI address bits 11-5. +KVM_FEATURE_SEV_LIVE_MIGRATION 16 guest checks this feature bit before + using the page encryption state + hypercall to notify the page state + change + KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24 host will warn if no guest-side per-cpu warps are expected in kvmclockdiff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst index e37a14c323d2..020245d16087 100644 --- a/Documentation/virt/kvm/msr.rst +++ b/Documentation/virt/kvm/msr.rst@@ -376,3 +376,15 @@ data: write '1' to bit 0 of the MSR, this causes the host to re-scan its queue and check if there are more notifications pending. The MSR is available if KVM_FEATURE_ASYNC_PF_INT is present in CPUID. + +MSR_KVM_SEV_LIVE_MIGRATION: + 0x4b564d08 + + Control SEV Live Migration features. + +data: + Bit 0 enables (1) or disables (0) host-side SEV Live Migration feature, + in other words, this is guest->host communication that it's properly + handling the shared pages list. + + All other bits are reserved.diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 950afebfba88..f6bfa138874f 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h@@ -33,6 +33,7 @@ #define KVM_FEATURE_PV_SCHED_YIELD 13 #define KVM_FEATURE_ASYNC_PF_INT 14 #define KVM_FEATURE_MSI_EXT_DEST_ID 15 +#define KVM_FEATURE_SEV_LIVE_MIGRATION 16 #define KVM_HINTS_REALTIME 0@@ -54,6 +55,7 @@ #define MSR_KVM_POLL_CONTROL 0x4b564d05 #define MSR_KVM_ASYNC_PF_INT 0x4b564d06 #define MSR_KVM_ASYNC_PF_ACK 0x4b564d07 +#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08 struct kvm_steal_time { __u64 steal;@@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data { #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK #define KVM_PV_EOI_DISABLED 0x0 +#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0) + #endif /* _UAPI_ASM_X86_KVM_PARA_H */diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index b0d324aed515..93f42b3d3e33 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c@@ -1627,6 +1627,16 @@ int svm_page_enc_status_hc(struct kvm *kvm, unsigned long gpa, return ret; } +void sev_update_migration_flags(struct kvm *kvm, u64 data) +{ + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; + + if (!sev_guest(kvm)) + return;
This should assert that userspace wanted the guest to be able to make these calls (see more below).
quoted hunk ↗ jump to hunk
+ + sev->live_migration_enabled = !!(data & KVM_SEV_LIVE_MIGRATION_ENABLED); +} + int svm_get_shared_pages_list(struct kvm *kvm, struct kvm_shared_pages_list *list) {@@ -1639,6 +1649,9 @@ int svm_get_shared_pages_list(struct kvm *kvm, if (!sev_guest(kvm)) return -ENOTTY; + if (!sev->live_migration_enabled) + return -EINVAL; + if (!list->size) return -EINVAL;diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 58f89f83caab..43ea5061926f 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c@@ -2903,6 +2903,9 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) svm->msr_decfg = data; break; } + case MSR_KVM_SEV_LIVE_MIGRATION: + sev_update_migration_flags(vcpu->kvm, data); + break; case MSR_IA32_APICBASE: if (kvm_vcpu_apicv_active(vcpu)) avic_update_vapic_bar(to_svm(vcpu), data);@@ -3976,6 +3979,19 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) vcpu->arch.cr3_lm_rsvd_bits &= ~(1UL << (best->ebx & 0x3f)); } + /* + * If SEV guest then enable the Live migration feature. + */ + if (sev_guest(vcpu->kvm)) { + struct kvm_cpuid_entry2 *best; + + best = kvm_find_cpuid_entry(vcpu, KVM_CPUID_FEATURES, 0); + if (!best) + return; + + best->eax |= (1 << KVM_FEATURE_SEV_LIVE_MIGRATION); + } +
Looking at this, I believe the only way for this bit to get enabled is if userspace toggles it. There needs to be a way for userspace to identify if the kernel underneath them does, in fact, support SEV LM. I'm at risk for having misread these patches (it's a long series), but I don't see anything that communicates upwards. This could go upward with the other paravirt features flags in cpuid.c. It could also be an explicit KVM Capability (checked through check_extension). Userspace should then have a chance to decide whether or not this should be enabled. And when it's not enabled, the host should return a GP in response to the hypercall. This could be configured either through userspace stripping out the LM feature bit, or by calling a VM scoped enable cap (KVM_VM_IOCTL_ENABLE_CAP). I believe the typical path for a feature like this to be configured would be to use ENABLE_CAP.
quoted hunk ↗ jump to hunk
if (!kvm_vcpu_apicv_active(vcpu)) return;diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 066ca2a9f1e6..e1bffc11e425 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h@@ -79,6 +79,7 @@ struct kvm_sev_info { unsigned long pages_locked; /* Number of pages locked */ struct list_head regions_list; /* List of registered regions */ u64 ap_jump_table; /* SEV-ES AP Jump Table address */ + bool live_migration_enabled; /* List and count of shared pages */ int shared_pages_list_count; struct list_head shared_pages_list;@@ -592,6 +593,7 @@ int svm_unregister_enc_region(struct kvm *kvm, void pre_sev_run(struct vcpu_svm *svm, int cpu); void __init sev_hardware_setup(void); void sev_hardware_teardown(void); +void sev_update_migration_flags(struct kvm *kvm, u64 data); void sev_free_vcpu(struct kvm_vcpu *vcpu); int sev_handle_vmgexit(struct vcpu_svm *svm); int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in); --2.17.1