On Mon, Mar 14, 2016 at 06:20:36PM +0000, Andre Przywara wrote:

Hi,

On 14/03/16 17:54, Marc Zyngier wrote:

quoted

On 14/03/16 17:29, Peter Maydell wrote:

quoted

On 14 March 2016 at 11:13, Andre Przywara [off-list ref] wrote:

quoted

So I see two ways to fix this:
1.) we find a KVM specific way of letting userland save and restore the
ITS tables directly
2.) we implement the BASER<n> registers, but still use our "cache" for
normal operations. On demand we would serialize KVM's virtual ITS data
structures and put them into the guest's memory, so they could be
saved/restored from there.

I feel like we're rehashing a bunch of design choices we talked
through way back in the last-but-one Connect. I don't suppose
anybody wrote down our rationales from back then?

(In particular I forget whether we decided the ITS tables were
large enough to need to allow some sort of before-the-VM-stops
migration of the data, which would be relatively doable with
option 2 but painful under option 1.)

I think only option 2 is valid here, and we must be able to shove most
of the routing information in the device/collection/IT tables. Common HW
seems to use 64bit of data per entry per table, so we should be able to
do the same with KVM.

All right, just skimmed over this and it looks doable.
For the collection table we will most likely even get away with 32 bits
per entry (compressed MPIDR or even VCPUIDs).
Would the IPA of the ITTE suffice for each device table entry?

I will work out the details later.

quoted

quoted

quoted

quoted

Only caveat there I think was that we had to decide on a storage format
in those memory regions, to allow QEMU to understand the state and to
ensure back/forwards compatibility between KVM versions.

Do we need QEMU to actually understand this? Can't we just leave this
all to the kernel and QEMU just passes on the data? That would still
require some ABI stability between kernel versions in this respect, but
it's less problematic than exposing the data format to userland at all.

This would preclude ever being able to migrate a VM from KVM to
TCG QEMU, which seems a shame. (That doesn't work right now, but
I'm a bit wary of shutting the door to it forever.)

If the format of the migrated tables becomes ABI for KVM, it also
becomes ABI for userspace (anything that comes out of the kernel *is*
ABI). Andre, can you please explain what you mean?

Well, probably there is not so much difference. I was just wondering if
it would be easier to treat that data as an opaque blob.
But you are probably right that it would just mean the difference
between documenting the format or not.

Even ignoring the migrate-to-TCG case, you cannot treat it as a blob,
because you want to be able to migrate between KVM on kernel version X
and version Y.

-Christoffer