On 26 November 2014 at 01:39, Sudeep Holla [off-list ref] wrote:

On 25/11/14 16:43, Andy Green wrote:

quoted

On 25 November 2014 at 22:24, Sudeep Holla [off-list ref] wrote:

quoted

On 25/11/14 13:42, Andy Green wrote:

quoted

On 25 November 2014 at 19:48, Sudeep Holla [off-list ref] wrote:

quoted

On 20/11/14 12:35, Vincent Yang wrote:

quoted

The remote firmware(SCB) owns the SMP control. This MCPM driver gets
CPU/CLUSTER power up/down done by SCB over mailbox.

quoted

diff --git a/arch/arm/mach-mb86s7x/smc.S b/arch/arm/mach-mb86s7x/smc.S
new file mode 100644
index 0000000..a14330b
--- /dev/null
+++ b/arch/arm/mach-mb86s7x/smc.S

@@ -0,0 +1,27 @@
+/*
+ * SMC command interface to set secondary entry point
+ * Copyright: (C) 2013-2014 Fujitsu Semiconductor Limited
+ * Copyright: (C) 2014 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or

modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+.arch_extension sec
+
+/* void mb86s7x_cpu_entry(unsigned long secondary_entry); */
+ENTRY(mb86s7x_cpu_entry)
+       stmfd   sp!, {r1-r11, lr}
+       mov r1, r0
+       ldr r0, =1
+       mrc p15, 0, r3, c1, c0, 0
+       mov r4, r3
+       and r3, #0xbfffffff
+       mcr p15, 0, r3, c1, c0, 0
+       smc #0

Interesting, it looks like you have some secure entity running on your
platform.

Yes, we have a stub "secure firmware" that implements a few critical
functions to allow us to operate the kernel as nonsecure.  It's part
of the bootloader for this platform which is also GPL'd.

OK, thanks for clarifying.

quoted

quoted

1. While the CPU is powered down who is taking care of saving it's
     state as you are doing it in the Linux itself ?

Nothing.  The secure firmware is in a bootloader that is copied to and
runs from secure sram.  When the cpu is reset, he comes back up in
secure mode and gets initialized in the secure firmware, before
entering Non-secure mode and the kernel's secondary entry point.

So you do have live secure firmware stub on secure sram at any time,
right ? When the CPUs are powered down especially for low power states,
how is the secure state of the CPUs preserved ?

There isn't any 'secure state' like you are thinking.

Our secure firmware is a rudimentary stub that provides a handful of
stateless secure services to the kernel.

There's nothing to save or restore.

Ok that clarifies, but I assume some customers would like to extend that
beyond the stub.

Yes.  But currently the only implementation is the stub one.

quoted

quoted

quoted

quoted

2. Is Linux running in Secure or Non-secure mode ?

Another firmware (unfortunately not GPL) running on an on-die M3
informs the secure firmware on the AP whether he should set the AP cpu
to nonsecure or not before jumping to the kernel... basically it's
decided at runtime and the same kernel binary serves in both modes.

OK that's fine as along as you assume that kernel *always* runs in
*non-secure* mode and never attempts any *secure access*.

Yes that's how it's set up.

When this M3 tells the bootloader to make us nonsecure before going to
the kernel, he marks up all the IPs that have dynamic access control
to allow access in nonsecure.  That's because we have complete BSP
support out of tree and we want to prove that it can all work from
nonsecure directly.

When Fujitsu want to restrict an IP from nonsecure access for a
particular customer they will add support to manage the IP either on
the M3 or the AP secure firmware and disable the IP support in DT.
But that does not exist yet.

Yes this was what I was trying to tell. From what I understand now, if
the customer implements secure firmware which might need to save and
restore the state, then he needs to manage.(He might end up with

Yes.  But I think we'll have to implement and test management for that
when it exists.

duplication of this last man algorithm in the secure firmware too,
I was just trying to mention this)

Maybe... other patterns are also possible like whole core reservation
at the bootloader / secure firmware to run a different secure OS and
leave smp like it is.

Fujitsu's market is ASIC so they will implement whatever customization
their customer likes in hardware, firmware and software.

quoted

quoted

quoted

quoted

3. Why do you need this smc call for secondary boot only ?

The call sets the secondary entry point stored in the secure sram.

So IIUC, you run Linux in non-secure mode, PSCI would be more suitable
than MCPM when you start thinking/implementing CPUIdle otherwise I think
you will end up duplicating some logic(last man and race management)
both in Linux as well as your secure firmware.

As I mentioned our secure firmware is not what you're imagining.

Understood, that?s about the current stub secure firmware and I was
talking about non-existent full blown secure firmware. If that's not the
case, then it's fine.

OK

quoted

MCPM works very well for us.  Are you saying we can't use MCPM and
must reimplement with PSCI on both sides?

No I never said that, I wanted to understand what's the scope of your
or any secure firmware of your customer.

Good thanks.

quoted

quoted

quoted

The bootloader heuristic is if that's unset (0), and it's what the

This could be problem as when the CPU is hotplugged out, ideally it
should be set to 0 to avoid spurious wakeup and entry into Linux.

... I think you are talking about something else.

This is a PA stored in secure SRAM that holds the kernel's secondary
entry point.  It's 0 when the bootloader is copied to the secure SRAM
by the M3 at cold reset.  If anybody set it, it means Linux booted.

The M3 manages AP core state and either depowers the cluster or holds
them in reset. So there can be no 'spurious wakeup and entry into
Linux'.

Ok, good that you can guarantee that.

quoted

quoted

Yes MCPM does manage it, but IMO you are mixing up secure and non-secure
methods which might become issue later when implementing low power
CPU states.

Can you explain where you see the problem?  All we are doing is
storing the kernel secondary entry point.  The bootloader / secure
firmware always receives any started core first and decides when and
how to jump to it.

quoted

quoted

bootloader decided should be regarded as the primary cpu, then we do
the real onetime cold boot flow, load the kernel etc.  Non-primary
cpus wait at WFI in the bootloader.  When the primary cpu runs the
code above, he sets the secondary entry point, and later starts to

I assume it's not done when primary boots, but by primary cpu when
bringing up the secondaries.

Yes the point is the primary sets the secondary entry at the secure
firmware using the SMC and then the IPIs can start flying.

Only if GIC is properly configured for that though I don't see the point
in giving non-secure entity secure access. If any secure firmware wants
to change that in future that could be issue.

Well the point is there is no irq handling or smp management in the
secure firmware stub.

If it grows traditional secure world features like dealing with irqs
in secure world we'll have to cross / test that bridge when and if we
come to it.

quoted

quoted

quoted

bring up the other cores who jump to the secondary entry that was set.

I assume primary sends IPI to wake up secondaries, but if the SGIs are
configured as secure, then it will _not_ be delivered. If not it *might*

Well, it is coordinated between M3 and AP bootloader / secure firmware
if we should run nonsecure or not, and it configures a bunch of stuff
like GIC accordingly so it works.  And it does work.

quoted

work but I can't understand the need of running Linux non-secure with
all secure access given to it.

Fujitsu's goal with this was confirm the SoC + kernel can work in
nonsecure for the whole BSP stack.  Then as customers want particular
IPs secure-only, implement the behaviour they need in the AP secure
firmware or M3.  But as yet, no support for that is implemented.

I never said you *must implement* PSCI and *not use* MCPM. Since you

OK, good.

mentioned that you will be providing option to run Linux in non-secure mode,
which means there can be _secure_entity_ which does more than your
current stub, I just wanted to mention the possible issues you may need to
resolve then.

OK, thanks for your feedback.

-Andy

Regards,
Sudeep