Re: Runtime Memory Validation in Intel-TDX and AMD-SNP

On Sun, Jul 25, 2021 at 12:16:45PM +0300, Mike Rapoport wrote:

On Fri, Jul 23, 2021 at 07:29:59PM +0300, Kirill A. Shutemov wrote:

quoted

On Fri, Jul 23, 2021 at 06:23:39PM +0300, Mike Rapoport wrote:

quoted

quoted

@@ -1318,9 +1327,14 @@ void __init e820__memblock_setup(void)
 		if (entry->type == E820_TYPE_SOFT_RESERVED)
 			memblock_reserve(entry->addr, entry->size);
 
-		if (entry->type != E820_TYPE_RAM && entry->type != E820_TYPE_RESERVED_KERN)
+		if (entry->type != E820_TYPE_RAM &&
+		    entry->type != E820_TYPE_RESERVED_KERN &&
+		    entry->type != E820_TYPE_UNACCEPTED)
 			continue;

If I understand correctly, you assume that

* E820_TYPE_RAM and E820_TYPE_RESERVED_KERN regions are already accepted by
  firmware/booloader
* E820_TYPE_UNACCEPTED would have been E820_SYSTEM_RAM if we'd disabled
  encryption

What happens with other types? Particularly E820_TYPE_ACPI and
E820_TYPE_NVS that may reside in memory and might have been accepted by
BIOS.

Any accessible memory that not marked as UNACCEPTED has to be accepted
before kernel gets control.

Hmm, that would mean that everything that runs before the kernel must
maintain precise E820 map. If we use 2M chunk as basic unit for accepting
memory, the firmware must also use the same basic unit. E.g. we can't have
an ACPI table squeezed between E820_TYPE_UNACCEPTED.

No. See mark_unaccepted(). Any chunks that cannot be accepted with 2M, get
accepted upfront, so we will not need to track them.

(I've just realized that mark_unaccepted() is buggy if 'start' and 'end'
are in the same 2M. Will fix.)

Using e820 table would also mean that bootloader must be able to modify
e820 and it also must follow the 2M rule.

I think that using a dedicated data structure would be more robust than
hooking into e820 table.

Maybe. We can construct the bitmap in the decompresser and translate
EFI_UNACCEPTED_MEMORY to E820_TYPE_RAM. I will look into this.

quoted

quoted

quoted

+		if (entry->type == E820_TYPE_UNACCEPTED)
+			mark_unaccepted(entry->addr, end);
+
 		memblock_add(entry->addr, entry->size);
 	}
 

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 72920af0b3c0..db9d1bcac9ed 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c

@@ -944,6 +944,8 @@ void __init setup_arch(char **cmdline_p)
 	if (movable_node_is_enabled())
 		memblock_set_bottom_up(true);
 #endif
+	/* TODO: make conditional */
+	memblock_set_bottom_up(true);

  
If memory is accepted during memblock allocations this should not really
matter.
Bottom up would be preferable if we'd like to reuse as much of already
accepted memory as possible before page allocator is up.

One of the main reason for this feature is to speed up boot time and
re-usinging preaccepted memory fits the goal.

Using bottom up also means that early allocations end up in DMA zones,
which probably not a problem for VMs in general, but who knows what path
through devices people would want to use...

Good point. Maybe we can drop it. Will see based on performance
evaluation.

quoted

quoted

quoted

--- a/mm/memblock.c
+++ b/mm/memblock.c

@@ -814,6 +814,7 @@ int __init_memblock memblock_reserve(phys_addr_t base, phys_addr_t size)
 	memblock_dbg("%s: [%pa-%pa] %pS\n", __func__,
 		     &base, &end, (void *)_RET_IP_);
 
+	accept_pages(base, base + size);

Hmm, I'm not sure memblock_reserve() is the right place to accept pages. It
can be called to reserve memory owned by firmware which not necessarily
would be encrypted. Besides, memblock_reserve() may be called for absent
memory, could be it'll confuse TDX/SEV?

Such memory will not be marked as unaccepted and accept_pages() will do
nothing.

quoted

Ideally, the call to accept_pages() should live in
memblock_alloc_range_nid(), but unfortunately there still stale
memblock_find_in_range() + memblock_reserve() pairs in x86 setup code.

memblock_reserve() is the root of memory allocation in the early boot and
it is natual place to do the trick. Unless we have a good reason to move
it somewhere I would keep it here.

I think it is better to accept memory that is actually allocated rather
than marked as being used. It'll make it more robust against future changes
in memblock_reserve() callers and in what is accept_pages() in your patch. 

I disagree.

If we move accept_pages() up to callers we will make less robust: any new
user of memblock_reserve() has to consider if accept_pages() is needed and
like would ignore it since it's not essential for any non-TDX/non-SEV use
case.

-- 
 Kirill A. Shutemov