RE: [EXT] Re: [PATCH v19 0/7] firmware: imx: driver for NXP secure-enclave

From: Pankaj Gupta <pankaj.gupta@nxp.com>
Date: 2025-11-07 06:57:19
Also in: imx, linux-devicetree, linux-doc, lkml

quoted

On 25-10-27, Pankaj Gupta wrote:

quoted

On 25-09-27, Pankaj Gupta wrote:

quoted

Highlight:

There are long discussion at V18 with Marco Felsch.
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%
2Fl

ore.kernel.org%2Fimx%2FAM9PR04MB8604F5A7DD8B1E41FDAD09039501
A%40AM9

quoted

quoted
quoted

PR04MB&data=05%7C02%7Cpankaj.gupta%40nxp.com%7C853f73b919714
8e3c4ea

quoted

quoted
quoted

08de1b0c6148%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6
38977937

quoted

quoted
quoted

182676166%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUs
IlYiOiIwL

quoted

quoted
quoted

jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
%7C%

quoted

quoted
quoted

7C%7C&sdata=2LXQbwdgb4Sb3PaYMUhn3gmSaJZW3%2FNXAJOX6d%2Fh%
2BwI%3D&re

quoted

served=0
8604.eurprd04.prod.outlook.com/

The key point is about using OP-TEE OS.

Marco's questions:
=================
Q1. Both MU(s) from each world - Secure & Non-secure, cannot be used
    simultaneously. Why?
A1. ELE FW fix is available, where concurrent access to ELE from both the
    world is working. This fix is not required for the current patch-set for
    i.MX8ULP.

According
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2
Fgi
thub.com%2FOP-

TEE%2Foptee_os%2Fpull%2F7268%2Fcommits%2F83b516edc0270

quoted

quoted

ca8300ce524a0c3d560e67a0f48%23r1955899462&data=05%7C02%7Cpan
kaj.gupt

quoted

quoted

a%40nxp.com%7C853f73b9197148e3c4ea08de1b0c6148%7C686ea1d3bc2
b4c6fa92

quoted

quoted

cd99c5c301635%7C0%7C0%7C638977937182705340%7CUnknown%7CT
WFpbGZsb3d8e

quoted

quoted

yJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
oiT

quoted

quoted

WFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=k30VGVc%2BEYsHw3
QOABgFZw6

quoted

v%2F5oSZx7%2FekHCDaZjbH8%3D&reserved=0
there is no fix yet. Can you please point me to the fixed version?

It is not yet up-streamed to OPTEE-OS repo.

My intention of adding the above OP-TEE discussion link was to point 
out that an ELE-FW bug exists which needs to be fixed.

quoted

Fix version is part of NXP LF Q3 released on 30 September.
Commit can be found on this URL:

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
u
b.com%2Fnxp-imx%2Fimx-optee-os%2Fcommits%2Flf-
6.12.34_2.1.0%2F&data=05%7C02%7Cpankaj.gupta%40nxp.com%7C853f7
3b9197148e3c4ea08de1b0c6148%7C686ea1d3bc2b4c6fa92cd99c5c30163
5%7C0%7C0%7C638977937182721931%7CUnknown%7CTWFpbGZsb3d8e
yJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIj
oiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=U6Rh%2Be1CTBdJ
PGGyfm5hF4mqylkBcSQp6KNzE9cL%2FUo%3D&reserved=0.

I assume you meant the following commit:
 -
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
u
b.com%2Fnxp-imx%2Fimx-optee-
os%2Fcommit%2F039f15017cea38c401f908e0e1662ba8806ed4e3&data=0
5%7C02%7Cpankaj.gupta%40nxp.com%7C853f73b9197148e3c4ea08de1b
0c6148%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6389779
37182737561%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydW
UsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D
%3D%7C0%7C%7C%7C&sdata=5mxtHM9Fy0i94lRNn1itWJcTWU4Ca%2F0ZR
ZUDRTl9WxQ%3D&reserved=0

This adapts the timeout value to 100ms and seems more like an workaround.

There are additional fixes in OPTEE-OS, that will be part of LF Q4'25.

However, can NXP confirm that the ELE concurrent access is possible 
without a previous ELE FW update?

Fix in the ELE FW, released as part of LF Q3 2025,  is a must to include. OPTEE fixes are also needed.
OPTEE fixes will be up-streamed soon.

quoted

One more fix is coming in LF Q4. After that, these patches will 
up-streamed to OP-TEE-OS.

Then you will be able locate it in OPTEE-OS git repo.

Thanks for the sharing the links :)

quoted

Since the current Patch-set is for i.MX8ULP, where concurrent access 
from both the world is not possible.

quoted

In context to i.MX93, for OPTEE-OS patches, request you to please 
discuss in OPTEE community. Not here.

Sure, as said above, my intention was to point out that concurrent 
access is not possible with the current ELE-FW. At least my knowledge 
was that it was/is a ELE FW issue and not an OP-TEE issue.

quoted

Also, can you please outline why the fix is not required for i.MX8ULP?
Unfortunately I can't remember.

There is no Trust-MU for OPTEE-OS in i.MX8ULP. The ELE driver in 
OPTEE, is disabled for i.MX8ULP.

Hm.. I thought there will be always two communication MUs one for the 
secure-world and one for the non-secure world? Or what did you mean by:

"""
From i.MX9x & onwards, SoC(s) has at least one dedicated ELE MU(s) for 
each world - Linux(one or more) and OPTEE-OS (one or more).
"""

(ref.:https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%
2Fl ore.kernel.org%2Fall%2FAM9PR04MB8604C05882605EDB4913DA089549A
%40AM9PR04MB8604.eurprd04.prod.outlook.com%2F&data=05%7C02%7
Cpankaj.gupta%40nxp.com%7C853f73b9197148e3c4ea08de1b0c6148%7C
686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6389779371827527
25%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLj
AuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
7C%7C%7C&sdata=PhXXih6UnalGUPveMZPaS2EVAd2SMR%2FyblcjyuHRUrw
%3D&reserved=0)

Does this mean that all i.MX9x, i.MX10x and so on do have the the 
secure and non-secure MU setup? Or is it based on the SoC release date?
Because regarding the datasheet the i.MX8ULP is newer than the i.MX93, 
therefore I assumed that the i.MX8ULP has two MUs as well.

From i.MX93 and onward, there is only one RoT. Hence , it is designed to have dedicated MU for each world.

I checked the the NXP OP-TEE source code and found the following commit:

8<----------------
commit 44388d37e68000ee50a9b1d656e0a60ae6614977
Author: Sahil Malhotra [off-list ref]
Date:   Tue Apr 1 20:04:44 2025 +0200

    core: imx: disable ELE on i.MX8ULP by default

    On i.MX8ULP, there is only one MU to communicate with ELE,
    which cannot be dedicated on OP-TEE side all the time.
    There may be ELE services running on Linux side, which can
    cause conflict with OP-TEE, So disabling ELE by default.
    Moreover i.MX8ULP also has CAAM, so HUK and Random number
    are coming from CAAM.

    Signed-off-by: Sahil Malhotra [off-list ref]
    Acked-by: Jens Wiklander [off-list ref]
8<----------------

So it's possible to configure the XRDC (configured in the TF-A) in a 
way to map the ELE access to the secure world. If I got the TF-A and OP-TEE commits right.

To me this sound more like a NXP design decision to move the ELE to 
the non- secure and the CAAM to the secure world.

As per the i.MX8ULP boot-up sequence and ELE's initial role in boot-up, with CAAM co-exists, ELE is logical to be with Linux.
Another point here that CAAM has 4 JR(s) and hence CAAM can be shared between Linux and OPTEE-OS.

quoted

Moreover, there is CAAM in i.MX8ULP, which is used for security 
services, in OPTEE-OS.

i.MX8ULP is getting really interesting, though.

May I ask what RoT is used by this SoC if there are two?

ELE is Root of Trust during secure boot.
For OPTEE-OS, CAAM is RoT.

quoted

Q2. How is the secure and non-secure world talk to the ELE if there is only
    one MU?
A2. From i.MX9x & onwards, SoC(s) has at least one dedicated ELE 
MU(s)

for

quoted

    each world - Linux(one or more) and OP-TEE OS (one or more). 
Has

updated

quoted

    the commit message of 3/7.

quoted

Does apply to the i.MX8ULP SoC as well?

No. Since there is no trust-MU for OPTEE-OS, on i.MX8ULP, to 
communicate with ELE.

As said above it would be possible to move the ELE communication to 
the secure world.

quoted

The NXP name scheme isn't very easy to follow.

quoted

Q3. How exactly do the eFuse write access after the device was 
locked

down,

quoted

    will be provided with single path; for the following case?
    - For a verified boot setup, you need to burn an eFuse at some point,
    to tell the bootROM to boot only correct verified firmware images.
    - After this lockdown, it's no longer possible to burn eFuses from the
    REE, while the production line setup still requires the support.
A3. OP-TEE OS supports read fuses, currently as part of our 
quarterly

release.

quoted

    Write fuse will be in coming release. Please note that the current patch-
    set does not contain any read/write eFuse feature.

quoted

But one of your patches is mention it, right? So there will be 
still two paths.

Due to lack of trusted-mu, there is no way to read/write fuse on 
i.MX8ULP,

from OPTEE-OS.

quoted

For current patch-set, it is the single path.

How does the i.MX8ULP fuse flow work, after the LOCK_DOWN fuse is blown?

There is no such issue on i.MX8ULP

This was one of my main concers why having OP-TEE required in the 
first place, because the i.MX93 requires the that the fuse-request 
comes from the secure-world if the device is in LOCK_DOWN state.

Is this also the case for the i.MX8ULP?

No, this is not a valid case for i.MX8ULP.

quoted

Also according your IOCTL docuementation you want to expose the 
whole device to the user-space?

quoted

| What:          /dev/<se>_mu[0-9]+_ch[0-9]+
| Date:          Mar 2025
| KernelVersion: 6.8
| Contact:       linux-imx@nxp.com, pankaj.gupta@nxp.com
| Description:
|                NXP offers multiple hardware IP(s) for secure 
| enclaves like

EdgeLock-

quoted

|                Enclave(ELE), SECO. The character device file descriptors
|                /dev/<se>_mu*_ch* are the interface between 
| userspace NXP's

secure-

quoted

|                enclave shared library and the kernel driver.
|
|                The ioctl(2)-based ABI is defined and documented in
|                [include]<linux/firmware/imx/ele_mu_ioctl.h>.
|                ioctl(s) are used primarily for:
|                        - shared memory management
|                        - allocation of I/O buffers
|                        - getting mu info
|                        - setting a dev-ctx as receiver to 
| receive all the commands

from FW

quoted

|                        - getting SoC info
|                        - send command and receive command 
| response

quoted

This is a rather uncommon approach. The kernel has interfaces to 
abstract hardware. You completely bypass this if you expose 
everything to the userspace.

It is in-correct.
Not everything, just exposed file-operation. and ioctl(s) for 
DMA(eable)

buffer allocation from reserved lower memory region.

quoted

Things managed by Kernel:
* Send/receive path to ELE, is managed by Kernel.
* Receive/send patch to the ELE's slave NVM-manager, is managed by kernel.
* Low power management handled by kernel driver. In case of 
low-power

state, ELE driver re-init the V2X IP as part of resume.

quoted

* Other kernel management layers like NVMEM, kernel HWRNG, will use 
the

api(s) exposed by this driver.

But you also expose an uAPI which allows the user to bypass everything 
via sending arbitrary commands, right?

Yes. But it's not unusual at all. The pattern of userspace sending commands directly to the kernel, is quite common

quoted

* Loading the runtime FW.

quoted

Q4. What is the ELE behavior if the secure-world stresses the ELE? Is there
    a MU priority so the normal-world MU may get blocked (never

addressed)

quoted

    or are both MUs scheduled in round-robin?
A4. ELE will pick the message from Both MU(s), based on FIFO.

quoted

This doesn't answer my question. How gets the FIFO filled? If the 
one of both worlds aways keep the MU busy?

If both the world is keeping the MU busy. ELE will pick the first 
incoming message, let's say on MU#1, While acknowledge and queueing 
the incoming message on the other MU#2, to be picked once the 
handling of the current message at hand is completed. Despite the 
MU#1, is bombarded With next message, but it will process the 
message from MU#2,

while acking the incoming message from MU#1. And add it to its queue. 
This way FIFO is maintained by ELE.

:) Thanks for the explanation.

quoted

Q5. Why ELE performance matters?
    - On i.MX8M, which uses the CAAM (the ELE predecessor), It can be
    verified that the ARMv8 CE are much faster than the 
crypto-engine

itself.

quoted

    Therefore the CAAM is mostly unused.
A5. User looks for both type of implementation based on their usecase:
    - Certain users still offload to CAAM, to save their ARM CPU cycles for
    other priority tasks.

quoted

I have never seen such designs but okay.

I assure you, there are several such cases with our system-engineering team.

quoted

    - While certain customer choose ARM-CE for the crypto 
offloads for

better

quoted

    performance.
    As far as ELE is concerned, it is not a crypto accelerator like CAAM.
    Performance hungry crypto-ops are not recommended to be 
offloaded

to ELE.

quoted

I don't get this point. So NXP doesn't recommend to use the ELE 
for crypt-ops?

NXP does not recommend to offload the Crypto-Op to ELE, for 
performance

cases.

Good to know, thanks.

quoted

Q6. For the IOT-cases where real-time encrypted feed from 
sensors, will

take

quoted

    latency hit, Does the ELE support inline en-/decryption?
A6. Yes, ELE support Inline Encryption Engine (IEE), starting 
i.MX95

onwards.

quoted

May I ask, how is this connected and which interfaces are 
connected to the IEE? I would have checked it by myself but 
unfortunately NXP doesn't provide a TRM for the i.MX95 yet.

We can discuss this in greater depth on one-one email, where I will 
touch

upon the IEE and its interfaces.

quoted

But, like to inform that there is not IEE on i.MX8ULP or even on i.Mx93.

Sure, thanks.

quoted

Marco's Suggestions/NXP response:
================================
S1. Below listed ELE access requirements, can be done from OP-TEE 
OS in

all

quoted

    the Life-Cycle states. then Why to use Linux?
    - eFuse access (done via  OP-TEE OS i.MX specific PTA): OTP 
eFuse R/W

access

quoted

    after doing the LOCK_DOWN fuse is no longer possible without  
OP-TEE

OS. This

quoted

    involves general purpose (GP) eFuses too.
    - ELE ping every 1 day (kernel SMC WDG driver, OP-TEE OS wd driver).
    - HW-RNG (kernel HWRNG driver + OP-TEE OS HWRNG PTA).
R1. There are i.MX9* users, that are not using OP-TEE OS, but still want
    to use ELE features from Linux. So, these users are requesting for this
    driver in Linux.

quoted

Which ELE features?

All the features from Linux, when OPTEE-OS is not present like:
* Read/write efuses. (eg NVMEM Kernel driver). Where there is no

requirement to chenge the eFuse, during SoC in provision lifecycle.

Some features require the device to be in LOCK_DOWN mode, which 
requires secure-world eFuse write path only afterwards. But it seems 
like NXP really wants to maintain two write paths.

quoted

* Low power management at Linux driver.

The power-modes are selected via the ELE?

Voltage regulation for i.MX93 in Linux kernel, is done by ELE.
During Linux suspend-resume, Secure-enclave (V2X on i.MX95) part of wake-up domain, will be managed by secure-enclaves(ELE) part of always-on domain.

quoted

* Linux HWRNG.
* Loading the secondary runtime fw.

What is a secondary runtime-fw?

ELE FW size is larger than the size of ELE internal secure memory.
Hence FW is split into two.

Primary FW, is the FW, that enables features that helps for SoC boot-up.
Secondary runtime FW, is the FW, that enables features like HSM.

quoted

    As suggested by you, writing to eFuse after life-cycle closed state, the
    driver in OP-TEE OS is needed.
    Though there are other OP-TEE OS features too, that needs to 
be

offloaded

quoted

    to HSM, for security reasons, like:
    - Hardware Unique Key(HUK) getting generated.
    - OP-TEE OS's ASLR seed generation.

quoted

We're talking about features the normal world wants to have access 
too, since you said ".. but still want to use ELE feature from 
Linux ..". The HUK and the TEE ASLR seed is clearly not something 
Linux wants to have access too.

You misunderstood my point. I was stressing that there is a need for 
two paths, each from Linux and OPTEE-OS, in order to expose 
different feature-set required by respective worlds, where 1 or 2 
overlaps, while maximum are utilized by respective worlds.

quoted

S2. Having one implementation is beneficial in the following ways:
    - Multiple paths also requires more maintain- and testing effort.
    - FW files with having +1 FW for the new i.MX9*, where there are plenty
      already eg.: bootloader, TF-A, ele-fw, scu-fw (i.MX95), increases the
      maintenance on both the paths.
    - TEE code could be reused by other OSes.
    - TEE is used anyway for new projects due to CRA and friends 
R2. As conveyed in R1, there are Linux only users, that are 
dependent on

SE

quoted

    Linux kernel driver.

quoted

What difference does it make if these Linux-only users get the ELE 
access via OP-TEE? It's just one binary more.

If the User does not choose to include optional OPTEE-OS, then ELE 
access must be from Linux driver.

Okay, I think we're going in circles here.

quoted

S3. Having one implementation is beneficial in the following ways:
    - Adding the support for it to  OP-TEE OS do not requires ELE-FW.

quoted

I never said that this requires no ELE FW! I said it requires no 
ELE-FW update! Was the update already rolled out?

Yes, the updated FW is already release part of LF Q3 released on 
September

30, 2025.

quoted

It is not released as part any open repo.

So there is no ELE-FW update required to have both worlds running 
concurrently.

As mentioned below "But not for i.MX8ULP", concurrent access case is not a valid case for i.MX8ULP & current patch-set.

quoted

    - Concurrent access handling is done by the TEE core.
R3. As already mentioned that ELE FW fix is available, for 
concurrent

access.

quoted

    Please note that FW fix is not required for the current patch-set.

quoted

Can you provide us a link to the fix please? The fix should fix 
the concurrent normal-world <-> secure-world MU access issue. Why 
is this fix not required if I have a running OP-TEE?

The fix is very much required to run concurrent access for i.XM93 
and

onwards.

quoted

But not for i.MX8ULP and current patch-set is based on i.MX8ULP.

I see, the i.MX8ULP has only on MU. As said above my assumption was 
that all ELE SoCs do have at least 2-MUs."

As mentioned above "But not for i.MX8ULP", concurrent access case is not a valid case for 8ULP & current patch-set.

quoted

S4. This series causes more confusions till the ELE FW fix is available to
    allow concurrent access to ELE from both the worlds.
R4. FW fix is already available, and concurrent access is enabled by default
    in LF Q3'25, release. Please note that FW fix is not required for the
    current patch-set.

quoted

Please provide a link to the new FW and why is the fix nor 
required for this patchset?

Again. The fix is very much required to run concurrent access for 
i.XM93 and

onwards.

quoted

But not for i.MX8ULP and current patch-set is based on i.MX8ULP.

My bad, as said: my assumption was that all ELE SoCs do have at least 
two MUs.

quoted

S5. Suggesting to add the  OP-TEE OS support for the required 
features

e.g. eFuses,

quoted

    watchdog, HWRNG. The whole Linux part is mostly in place.
R5. Yes, the secure world features like eFuse read/write will be added
    to OP-TEE OS. Please note that this patch-set does not 
contain any

read/write

quoted

    eFuse feature.

quoted

I see, instead it expose the whole ELE to the userspace if I got 
it correct according to your IOCTL documentation. This would be even worse.

No, it is not a correct interpretation. As mentioned above, ELE 
driver abstract all the hw interaction to have easy and simple file bases operations.

But still, the user can send arbitrary commands to the ELE right?

To conclude this longly discussion:

My main concern was the concurrent access for the 2-MUs cases. This 
can be closed, as it turns out NXP fixed/workarounded this by 
increasing the ELE communication timeout to 100ms.

This will be not only the fix. There are other fixes like atomic write to trusted-MU.
Let's discuss in OPTEE-forum.

I still have mixed feeling about the fusing (including the 1-MU case), 
since it requires a secure-world OS in place once the LOCK_DOWN fuse was burned.
It's fine by me if NXP wants to have and wants to maintain a multi-path here.

Write fuse API will be added, to allow writing fuses from secure world too.

Last but least, the uAPI which can be used to send arbitrary ELE 
commands seems unusual. But I don't know how secure-enclaves are 
abstracted within the kernel, so these are just my two cents.

it's not unusual at all. The pattern of userspace sending commands directly to the kernel via a queue is quite common like:
GPUs: As you mentioned, userspace drivers (like those in Vulkan or CUDA) often build command buffers and submit them directly to the kernel or hardware.
Secure Enclaves: In systems like Intel SGX or AMD SEV, userspace applications interact with enclaves via ioctl or mmap interfaces, often sending structured commands or messages.

The uAPI (userspace API) for sending ELE (Enclave Lifecycle Engine?) commands likely follows this same model - abstracting the enclave as a device or subsystem and exposing a command queue or ioctl interface for userspace to interact with it.

I'm looking forward to the maintainers feedback.

Regards,
Marco

quoted

Regards,
Marco

Regards,
Pankaj

`h`	back out one level
`j`	next message in thread
`k`	previous message in thread
`l`	drill in
`Esc`	close help / fold thread tree
`?`	toggle this help