On 01/10/21 12:32, Marcelo Tosatti wrote:

quoted

+1. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_0), +
kvmclock nanoseconds (k_0), and realtime nanoseconds (r_0). + [...]
 +4. Invoke the KVM_SET_CLOCK ioctl, providing the kvmclock
nanoseconds +   (k_0) and realtime nanoseconds (r_0) in their
respective fields. +   Ensure that the KVM_CLOCK_REALTIME flag is
set in the provided +   structure. KVM will advance the VM's
kvmclock to account for elapsed +   time since recording the clock
values.

You can't advance both kvmclock (kvmclock_offset variable) and the
TSCs, which would be double counting.

So you have to either add the elapsed realtime (1) between
KVM_GET_CLOCK to kvmclock (which this patch is doing), or to the
TSCs. If you do both, there is double counting. Am i missing
something?

Probably one of these two (but it's worth pointing out both of them):

1) the attribute that's introduced here *replaces*
KVM_SET_MSR(MSR_IA32_TSC), so the TSC is not added.

2) the adjustment formula later in the algorithm does not care about how
much time passed between step 1 and step 4.  It just takes two well
known (TSC, kvmclock) pairs, and uses them to ensure the guest TSC is
the same on the destination as if the guest was still running on the
source.  It is irrelevant that one of them is before migration and one
is after, all it matters is that one is on the source and one is on the
destination.

Perhaps we can add to step 6 something like:

+6. Adjust the guest TSC offsets for every vCPU to account for (1)
time +   elapsed since recording state and (2) difference in TSCs
between the +   source and destination machine: + +   new_off_n = t_0
+ off_n + (k_1 - k_0) * freq - t_1 +

"off + t - k * freq" is the guest TSC value corresponding to a time of 0
in kvmclock.  The above formula ensures that it is the same on the
destination as it was on the source.

Also, the names are a bit hard to follow.  Perhaps

	t_0		tsc_src
	t_1		tsc_dest
	k_0		guest_src
	k_1		guest_dest
	r_0		host_src
	off_n		ofs_src[i]
	new_off_n	ofs_dest[i]

Paolo


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel