Re: [PATCH v16 00/45] arm64: Support for Arm CCA in KVM

On 12/08/2026 11:35, Gavin Shan wrote:

Hi Alper and Suzuki,

On 8/12/26 4:04 PM, Suzuki K Poulose wrote:

quoted

Hi Alper, Gavin

On 12/08/2026 04:25, Alper Gun wrote:

quoted

On Tue, Aug 11, 2026 at 8:08 PM Gavin Shan [off-list ref] wrote:

quoted

As the following calltrace indicates, -EAGAIN is returned from tf- 
rmm::update_ripas()
because true is returned from s2tte_drain_pending() for the S2TTE 
corresponding to
IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) 
in ripas_change().
Upon this specific error and the IPA range [0x80000000 0x90000000], 
find_map_level()
returns level of 2, and realm_create_rtt_levels() returns 0 without 
populating any
RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above 
loop starts over
again.

    Linux host
    ==========
    kvm_arch_vcpu_ioctl_run                     // cca/host-v16
      check_vcpu_requests
        kvm_check_request
          kvm_rec_handle_request
            kvm_complete_ripas_change
              realm_set_ipa_state
                ripas_change
                  rmi_rtt_set_ripas
                    SMC_RMI_RTT_SET_RIPAS

    TF-RMM
    ======
    SMC_RMI_RTT_SET_RIPAS                      // tf-rmm/topics/rmm- 
v2.0-poc_3
      smc_rtt_set_ripas
        s2tt_walk_lock_unlock
        rtt_set_ripas_range
          update_ripas
            s2tte_drain_pending                // true, returns -EAGAIN

The problem is the pending-bit for RTE corresponding IPA address 
0x80000000 isn't cleared
when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this 
bit is set and why
it's not cleared in time.

Thanks for the details.

quoted

quoted

Hi Gavin, Suzuki,

I think I ran into a similar issue on rmm-v2.0-poc_3 last week.
This looks like a potential RMM bug: could bit 32 be part of the 
physical
Address (if PA >= 4 GiB)?

It seems s2tte_drain_pending() in lib/s2tt/src/s2tt.c checks bit 32 
without
checking whether the descriptor is valid or invalid.

In my testing, guarding the drain checks with a check for 
S2TTE_INVALID seemed
to resolve the boot hang:

--- a/lib/s2tt/src/s2tt.c
+++ b/lib/s2tt/src/s2tt.c

@@ -1701,6 +1701,10 @@ unsigned long

s2tte_clear_drain_pending(unsigned long s2tte)

  bool s2tte_drain_pending(unsigned long s2tte)
  {
+ if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
+ return false;

We should use also consider cases where the entry is INVALID, but
has HIPAS=ASSIGNED/ASSIGNED_DEV to make it tighter. So, I think
it is better to use :

s2tte_is_unassigned() or in the library stick to :

if (!s2tte_has_hipas(s2tte, S2TTE_INVALID_HIPAS_UNASSIGNED))
     return false;

May be we should assert this and make the caller responsible for
checking the bit. I will leave it to the tf-RMM team to fix.

But for now, please use the above fix.

Both worked for me. With the extra check in place, the realm guest can boot
up successfully.

FYI, The below additional checks in s2tte_tlbi_pending() and 
s2tte_drain_handle()
aren't needed because they're always guarded by s2tte_drain_pending() in 
all
calling sites.

fyi, the tf-RMM patch is out for review here : (Thanks Javier)

https://review.trustedfirmware.org/c/TF-RMM/tf-rmm/+/53531

Please feel free to cherry-pick that one

Cheers
Suzuki