Hi, David,

Thanks a lot for comments!

David Hildenbrand [off-list ref] writes:

On 30.09.24 07:51, Huang Ying wrote:

quoted

On systems with TDX (Trust Domain eXtensions) enabled, memory ranges
hot-added must be checked for compatibility by TDX.  This is currently
implemented through memory hotplug notifiers for each memory_block.
If a memory range which isn't TDX compatible is hot-added, for
example, some CXL memory, the command line as follows,
   $ echo 1 > /sys/devices/system/node/nodeX/memoryY/online
will report something like,
   bash: echo: write error: Operation not permitted
If pr_debug() is enabled, the error message like below will be shown
in the kernel log,
   online_pages [mem 0xXXXXXXXXXX-0xXXXXXXXXXX] failed
Both are too general to root cause the problem.  This will confuse
users.  One solution is to print some error messages in the TDX memory
hotplug notifier.  However, memory hotplug notifiers are called for
each memory block, so this may lead to a large volume of messages in
the kernel log if a large number of memory blocks are onlined with a
script or automatically.  For example, the typical size of memory
block is 128MB on x86_64, when online 64GB CXL memory, 512 messages
will be logged.

ratelimiting would likely help here a lot, but I agree that it is
suboptimal.

quoted

Therefore, in this patch, the whole hot-adding memory range is
checked
for TDX compatibility through a newly added architecture specific
function (arch_check_hotplug_memory_range()).  If rejected, the memory
hot-adding will be aborted with a proper kernel log message.  Which
looks like something as below,
   virt/tdx: Reject hot-adding memory range: 0xXXXXXXXX-0xXXXXXXXX
for TDX compatibility.

quoted

The target use case is to support CXL memory on TDX enabled systems.

If the CXL memory isn't compatible with TDX, the whole CXL memory
range hot-adding will be rejected.  While the CXL memory can still be
used via devdax interface.

I'm curious, why can that memory be used through devdax but not
through the buddy? I'm probably missing something important :)

Because only TDX compatible memory can be used for TDX guest.  The buddy
is used to allocate memory for TDX guest.  While devdax will not be used
for that.

quoted

This also makes the original TDX memory hotplug notifier useless, so
delete it.

The online-notifier would even be too late when used with the
memmap-on-memory feature I assume, as we might be touching that memory
even before being able to call memory online notifiers.

This should be OK.  Because we will not use the memory for TDX guest in
this way.

One way to handle that would be to switch to the MEM_PREPARE_ONLINE
notifier, but it's still called per-memory block.

Nothing jumped at me, so

Acked-by: David Hildenbrand <redacted>

Thank you very much!

--
Best Regards,
Huang, Ying