Hi Oliver,

Adding Andrew and Peter to the discussion.

On Tue, 24 Aug 2021 22:15:03 +0100,
Oliver Upton [off-list ref] wrote:

Hey folks,

Ricardo and I were discussing hypercall support in KVM/arm64 and
something seems to be a bit problematic. I do not see anywhere that KVM
requires opt-in from the VMM to expose new hypercalls to the guest. To
name a few, the TRNG and KVM PTP hypercalls are unconditionally provided
to the guest.

Exposing new hypercalls to guests in this manner seems very unsafe to
me. Suppose an operator is trying to upgrade from kernel N to kernel
N+1, which brings in the new 'widget' hypercall. Guests are live
migrated onto the N+1 kernel, but the operator finds a defect that
warrants a kernel rollback. VMs are then migrated from kernel N+1 -> N.
Any guests that discovered the 'widget' hypercall are likely going to
get fussy _very_ quickly on the old kernel.

This goes against what we decided to support for the *only* publicly
available VMM that cares about save/restore, which is that we only
move forward and don't rollback. Hypercalls are the least of your
worries, and there is a whole range of other architectural features
that will have also appeared/disappeared (your own CNTPOFF series is a
glaring example of this).

I appreciate that you may have different considerations, but I felt
that it was important to state *why* this is the way it is.

Really, we need to ensure that the exposed guest ABI is
backwards-compatible. Running a VMM that is blissfully unaware of the
'widget' hypercall should not implicitly expose it to its guest on a new
kernel.

This conversation was in the context of devising a new UAPI that allows
VMMs to trap hypercalls to userspace. While such an interface would
easily work around the issue, the onus of ABI compatibility lies with
the kernel.

So, this is all a long-winded way of asking: how do we dig ourselves out
of this situation, and how to we avoid it happening again in the future?
I believe the answer to both is to have new VM capabilities for sets of
hypercalls exposed to the guest. Unfortunately, the unconditional
exposure of TRNG and PTP hypercalls is ABI now, so we'd have to provide
an opt-out at this point. For anything new, require opt-in from the VMM
before we give it to the guest.

Have I missed something blatantly obvious, or do others see this as an
issue as well? I'll reply with an example of adding opt-out for PTP.
I'm sure other hypercalls could be handled similarly.

Why do we need this? For future hypercalls, we could have some buy-in
capabilities. For existing ones, it is too late, and negative features
are just too horrible.

For KVM-specific hypercalls, we could get the VMM to save/restore the
bitmap of supported functions. That would be "less horrible". This
could be implemented using extra "firmware pseudo-registers" such as
the ones described in Documentation/virt/kvm/arm/psci.rst.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel