Thread (28 messages) 28 messages, 4 authors, 2021-05-17

Re: [PATCH v2 1/2] optee: fix tee out of memory failure seen during kexec reboot

From: Allen Pais <hidden>
Date: 2021-05-06 07:29:47
Also in: lkml, op-tee
Subsystem: op-tee driver, tee subsystem, the rest · Maintainers: Jens Wiklander, Linus Torvalds

quoted
quoted
quoted
quoted
quoted
quoted
quoted
[    0.368428] tee_bnxt_fw optee-clnt0: tee_shm_alloc failed
[    0.368461] tee_bnxt_fw: probe of optee-clnt0 failed with error -22

tee_shm_release() is not invoked on dma shm buffer.

Implement .shutdown() method to handle the release of the buffers
correctly.

More info:
https://github.com/OP-TEE/optee_os/issues/3637

Signed-off-by: Allen Pais <redacted>
---
drivers/tee/optee/core.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
This looks good to me. Do you have a practical way of testing this on
QEMU for instance?
Jens,

I could not reproduce nor create a setup using QEMU, I could only
do it on a real h/w.

I have extensively tested the fix and I don't see any issues.
I did a few test runs too, seems OK.
I carried these changes and have not run into any issues with Kexec so far.
Last week, while trying out kdump, we ran into a crash(this is when the
Kdump kernel reboots).

$echo c > /proc/sysrq-trigger

Leads to:

[   18.004831] Unable to handle kernel paging request at virtual address ffff0008dcef6758
[   18.013002] Mem abort info:
[   18.015885]   ESR = 0x96000005
[   18.019034]   EC = 0x25: DABT (current EL), IL = 32 bits
[   18.024516]   SET = 0, FnV = 0
[   18.027667]   EA = 0, S1PTW = 0
[   18.030905] Data abort info:
[   18.033877]   ISV = 0, ISS = 0x00000005
[   18.037835]   CM = 0, WnR = 0
[   18.040896] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000970a78000
[   18.047811] [ffff0008dcef6758] pgd=000000097fbf9003, pud=0000000000000000
[   18.054819] Internal error: Oops: 96000005 [#1] SMP
[   18.059850] Modules linked in: bnxt_en pcie_iproc_platform pcie_iproc diagbe(O)
[   18.067395] CPU: 3 PID: 1 Comm: systemd-shutdow Tainted: G           O      5.4.83-microsoft-standard #1
[   18.077174] Hardware name: Overlake (DT)
[   18.081219] pstate: 80400005 (Nzcv daif +PAN -UAO)
[   18.086170] pc : tee_shm_free+0x18/0x48
[   18.090126] lr : optee_disable_shm_cache+0xa4/0xf0
[   18.095066] sp : ffff80001005bb90
[   18.098484] x29: ffff80001005bb90 x28: ffff000037e20000
[   18.103962] x27: 0000000000000000 x26: ffff00003ed10490
[   18.109440] x25: ffffca760e975f90 x24: 0000000000000000
[   18.114918] x23: ffffca760ed79808 x22: ffff00003ec66e18
[   18.120396] x21: ffff80001005bc08 x20: 00000000b200000a
[   18.125874] x19: ffff0008dcef6700 x18: 0000000000000010
[   18.131352] x17: 0000000000000000 x16: 0000000000000000
[   18.136829] x15: ffffffffffffffff x14: ffffca760ed79808
[   18.142307] x13: ffff80009005b897 x12: ffff80001005b89f
[   18.147786] x11: ffffca760eda4000 x10: ffff80001005b820
[   18.153264] x9 : 00000000ffffffd0 x8 : ffffca760e59b2c0
[   18.158742] x7 : 0000000000000000 x6 : 0000000000000000
[   18.164220] x5 : 0000000000000000 x4 : 0000000000000000
[   18.169698] x3 : 0000000000000000 x2 : ffff0008dcef6700
[   18.175175] x1 : 00000000ffff0008 x0 : ffffca760e59ca04
[   18.180654] Call trace:
[   18.183176]  tee_shm_free+0x18/0x48
[   18.186773]  optee_disable_shm_cache+0xa4/0xf0
[   18.191356]  optee_shutdown+0x20/0x30
[   18.195135]  platform_drv_shutdown+0x2c/0x38
[   18.199538]  device_shutdown+0x180/0x298
[   18.203586]  kernel_restart_prepare+0x44/0x50
[   18.208078]  kernel_restart+0x20/0x68
[   18.211853]  __do_sys_reboot+0x104/0x258
[   18.215899]  __arm64_sys_reboot+0x2c/0x38
[   18.220035]  el0_svc_handler+0x90/0x138
[   18.223991]  el0_svc+0x8/0x208
[   18.227143] Code: f9000bf3 aa0003f3 aa1e03e0 d503201f (b9405a60)
[   18.233435] ---[ end trace 835d756cd66aa959 ]---
[   18.238621] Kernel panic - not syncing: Fatal exception
[   18.244014] Kernel Offset: 0x4a75fde00000 from 0xffff800010000000
[   18.250299] PHYS_OFFSET: 0xffff99c680000000
[   18.254613] CPU features: 0x0002,21806008
[   18.258747] Memory Limit: none
[   18.262310] ---[ end Kernel panic - not syncing: Fatal exception ]—

I see that before secure world returns OPTEE_SMC_RETURN_ENOTAVAIL(which
Should disable and clear all the cache) we run into the crash trying to free shm.

Thoughts?
It seems that the pointer is invalid, but the pointer doesn't look
like garbage. Could the kernel have unmapped the memory area covering
that address?
Yes, I am not entirely sure if the kernel had the time to unmap the memory.
Right after triggering the crash the kdump kernel is booted and I see the following

[ 2.050145] optee: probing for conduit method.
[ 2.054743] optee: revision 3.6 (f84427aa)
[ 2.054821] optee: dynamic shared memory is enabled
[ 2.066186] optee: initialized driver

Could this be previous un-released maps causing corruption?
Aha, yes, that could be it.
How about checking for the ptr?
diff --git a/drivers/tee/optee/call.c b/drivers/tee/optee/call.c
index aadedec3bfe7..8dc4fe9a1588 100644
--- a/drivers/tee/optee/call.c
+++ b/drivers/tee/optee/call.c
@@ -426,10 +426,12 @@ void optee_disable_shm_cache(struct optee *optee)
                if (res.result.status == OPTEE_SMC_RETURN_ENOTAVAIL)
                        break; /* All shm's freed */
                if (res.result.status == OPTEE_SMC_RETURN_OK) {
-                       struct tee_shm *shm;
+                       struct tee_shm *shm = NULL;
 
                        shm = reg_pair_to_ptr(res.result.shm_upper32,
                                              res.result.shm_lower32);
+                       if (IS_ERR(shm))
+                               return PTR_ERR(shm);
                        tee_shm_free(shm);
Thanks.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help