From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:42
Anyone vaguely familiar with the ARMv8 architecture would quickly
understand that entering the kernel at EL2 without enabling the HVC
instruction is... living dangerously. But as it turns out [0], there
is a whole range of (*cough*) "high quality" (*cough*) Broadcom
systems out there configured exactly like that.
If you are speechless, I'm right with you.
These machines have stopped being able to boot an upstream kernel
since 5.12, where we changed the way we switch from nVHE to VHE, as
this relies on the HVC instruction being usable... It is also worth
noting that these systems have never been able to use KVM. Or kexec.
This small series addresses the issue by detecting an UNDEFing HVC in
a fairly controlled environment, and in this case pretend that we have
booted at EL1. It also documents the requirement for SCR_EL3.HCE to be
set to *1* if the kernel is entered at EL2. Turns out that we really
have to state the obvious.
This has been tested on a FVP model with a hacked-up boot-wrapper.
Note that I really don't think any of this is -stable material, except
maybe for the documentation. It isn't 5.14 material either. Best case,
this is 5.15, or maybe even later. If ever.
M. (drink required)
[0] https://lore.kernel.org/r/53f3a2d2-22f8-edee-2507-d41a4090dad7@gmail.com
Marc Zyngier (5):
arm64: Directly expand __init_el2_nvhe_prepare_eret where needed
arm64: Handle UNDEF in the EL2 stub vectors
arm64: Detect disabled HVC early
arm64: Warn on booting at EL2 with HVC disabled
arm64: Document the requirement for SCR_EL3.HCE
Documentation/arm64/booting.rst | 5 +++++
arch/arm64/include/asm/el2_setup.h | 6 ------
arch/arm64/include/asm/virt.h | 10 +++++++++
arch/arm64/kernel/head.S | 34 ++++++++++++++++++++++++++++++
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++-
arch/arm64/kernel/smp.c | 3 +++
6 files changed, 70 insertions(+), 7 deletions(-)
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:38
As we want to handle the silly case where HVC has been disabled
from EL3, augment our ability to handle exception at EL2.
Check for unknown exceptions (usually UNDEF) coming from EL2,
and treat them as a failing HVC call into the stubs. While
this isn't great and obviously doesn't catter for the gigantic
range of possible exceptions, it isn't any worse than what we
have today.
Just don't try and use it for anything else.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
From: Robin Murphy <robin.murphy@arm.com> Date: 2021-08-13 13:08:31
On 2021-08-12 20:02, Marc Zyngier wrote:
quoted hunk
As we want to handle the silly case where HVC has been disabled
from EL3, augment our ability to handle exception at EL2.
Check for unknown exceptions (usually UNDEF) coming from EL2,
and treat them as a failing HVC call into the stubs. While
this isn't great and obviously doesn't catter for the gigantic
range of possible exceptions, it isn't any worse than what we
have today.
Just don't try and use it for anything else.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
@@ -71,6 +80,14 @@ SYM_CODE_START_LOCAL(elx_sync) 9: mov x0, xzr eret++el2_undef:+ // Assumes this was a HVC that went really wrong...+ mrs x0, elr_el2+ add x0, x0, #4+ msr elr_el2, x0+ mov_q x0, HVC_STUB_ERR+ eret SYM_CODE_END(elx_sync) // nVHE? No way! Give me the real thing!
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-13 17:41:32
On Fri, 13 Aug 2021 14:08:23 +0100,
Robin Murphy [off-list ref] wrote:
On 2021-08-12 20:02, Marc Zyngier wrote:
quoted
As we want to handle the silly case where HVC has been disabled
from EL3, augment our ability to handle exception at EL2.
Check for unknown exceptions (usually UNDEF) coming from EL2,
and treat them as a failing HVC call into the stubs. While
this isn't great and obviously doesn't catter for the gigantic
range of possible exceptions, it isn't any worse than what we
have today.
Just don't try and use it for anything else.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
Hmm, might it be neater to check ESR_EL2.ISS to see if we landed here
for any reason *other* than a successfully-executed HVC?
We absolutely could. However, the sixpence question (yes, that's the
Brexit effect for you) is "what do you do with exceptions that are
neither UNDEF now HVC?".
We are taking a leap of faith by assuming that the only thing that
will UNDEF at EL2 while the stubs are installed is HVC. If anything
else occurs, I have no idea what to do with it. I guess we could always
ignore it instead of treating it as a HVC (as it is done at the
moment).
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Robin Murphy <robin.murphy@arm.com> Date: 2021-08-13 18:18:06
On 2021-08-13 18:41, Marc Zyngier wrote:
On Fri, 13 Aug 2021 14:08:23 +0100,
Robin Murphy [off-list ref] wrote:
quoted
On 2021-08-12 20:02, Marc Zyngier wrote:
quoted
As we want to handle the silly case where HVC has been disabled
from EL3, augment our ability to handle exception at EL2.
Check for unknown exceptions (usually UNDEF) coming from EL2,
and treat them as a failing HVC call into the stubs. While
this isn't great and obviously doesn't catter for the gigantic
range of possible exceptions, it isn't any worse than what we
have today.
Just don't try and use it for anything else.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
Hmm, might it be neater to check ESR_EL2.ISS to see if we landed here
for any reason *other* than a successfully-executed HVC?
We absolutely could. However, the sixpence question (yes, that's the
Brexit effect for you) is "what do you do with exceptions that are
neither UNDEF now HVC?".
We are taking a leap of faith by assuming that the only thing that
will UNDEF at EL2 while the stubs are installed is HVC. If anything
else occurs, I have no idea what to do with it. I guess we could always
ignore it instead of treating it as a HVC (as it is done at the
moment).
Right, I think that concern applies pretty much equally whichever way
you slice it. "Any exception other than an unknown from EL2 must imply
HVC" doesn't seem any less sketchy than "Any exception other than HVC
implies something is horribly wrong and abandoning EL2 might be wise" to
me, but it was primarily that the latter avoids having to faff with the
SPSR as well. No big deal either way, just one of my "I reckon this
could be shorter..." musings; it's been particularly Friday today :)
Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-14 09:38:37
On Fri, 13 Aug 2021 19:17:56 +0100,
Robin Murphy [off-list ref] wrote:
On 2021-08-13 18:41, Marc Zyngier wrote:
quoted
On Fri, 13 Aug 2021 14:08:23 +0100,
Robin Murphy [off-list ref] wrote:
quoted
On 2021-08-12 20:02, Marc Zyngier wrote:
quoted
As we want to handle the silly case where HVC has been disabled
from EL3, augment our ability to handle exception at EL2.
Check for unknown exceptions (usually UNDEF) coming from EL2,
and treat them as a failing HVC call into the stubs. While
this isn't great and obviously doesn't catter for the gigantic
range of possible exceptions, it isn't any worse than what we
have today.
Just don't try and use it for anything else.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/hyp-stub.S | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
Hmm, might it be neater to check ESR_EL2.ISS to see if we landed here
for any reason *other* than a successfully-executed HVC?
We absolutely could. However, the sixpence question (yes, that's the
Brexit effect for you) is "what do you do with exceptions that are
neither UNDEF now HVC?".
We are taking a leap of faith by assuming that the only thing that
will UNDEF at EL2 while the stubs are installed is HVC. If anything
else occurs, I have no idea what to do with it. I guess we could always
ignore it instead of treating it as a HVC (as it is done at the
moment).
Right, I think that concern applies pretty much equally whichever way
you slice it. "Any exception other than an unknown from EL2 must imply
HVC" doesn't seem any less sketchy than "Any exception other than HVC
implies something is horribly wrong and abandoning EL2 might be wise"
to me, but it was primarily that the latter avoids having to faff with
the SPSR as well.
Actually, that's not a bad idea at all. Here's my take on the theme,
completely untested:
@@ -46,6 +46,23 @@ SYM_CODE_END(__hyp_stub_vectors).align11SYM_CODE_START_LOCAL(elx_sync)+//tpidr_el2isn't used for anything while the stubs are+//installed,souseittosavex0whileweguessthe+//exceptiontype.No,wedon't have a stack...+msrtpidr_el2,x0+mrsx0,esr_el2+ubfxx0,x0,#26, #6+cmpx0,#ESR_ELx_EC_HVC64+b.eqelx_hvc+cbzx0,elx_unknown++//Foranythingelse,wehavenoreasonablewaytohandle+//theexception.Gobacktothefaultinginstruction...+mrsx0,tpidr_el2+eret++elx_hvc:+mrsx0,tpidr_el2cmpx0,#HVC_SET_VECTORSb.ne1fmsrvbar_el2,x1
No big deal either way, just one of my "I reckon this could be
shorter..." musings; it's been particularly Friday today :)
Well, I just made it a lot longer! :D Let me know what you think.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:43
As we are about to enable taking exceptions extremely early,
it becomes pointless to pre-populate SPSR_EL2 in init_el2_state.
We already have the stuck-in-VHE handling code that needs some
special casing, so let's bite the bullet and get rid of the macros
by expanding them where needed.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/el2_setup.h | 6 ------
arch/arm64/kernel/head.S | 2 ++
2 files changed, 2 insertions(+), 6 deletions(-)
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:46
Having HVC disabled from EL3 while the kernel is entered at EL2
is a complete nightmare.
We end-up taking an UNDEF at the worse possible moment (checking
for VHE) and even if we didn't, having KVM enabled would signify
the premature end of the kernel.
Instead, try and detect this stupid case by issuing a HVC
for HVC_RESET_VECTORS, which does nothing when the stubs
are live. If we get HVC_STUB_ERR back, that's because the
UNDEF handler has kicked in.
In this situation, close your eyes, block your nose, and gracefully
pretend we have booted at EL1.
Reported-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/head.S | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
Having HVC disabled from EL3 while the kernel is entered at EL2
is a complete nightmare.
We end-up taking an UNDEF at the worse possible moment (checking
for VHE) and even if we didn't, having KVM enabled would signify
the premature end of the kernel.
Instead, try and detect this stupid case by issuing a HVC
for HVC_RESET_VECTORS, which does nothing when the stubs
are live. If we get HVC_STUB_ERR back, that's because the
UNDEF handler has kicked in.
In this situation, close your eyes, block your nose, and gracefully
pretend we have booted at EL1.
Reported-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
This does the trick.
Tested-by: Rafał Miłecki <rafal@milecki.pl>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Will Deacon <will@kernel.org> Date: 2021-08-13 09:05:49
On Thu, Aug 12, 2021 at 08:02:11PM +0100, Marc Zyngier wrote:
quoted hunk
Having HVC disabled from EL3 while the kernel is entered at EL2
is a complete nightmare.
We end-up taking an UNDEF at the worse possible moment (checking
for VHE) and even if we didn't, having KVM enabled would signify
the premature end of the kernel.
Instead, try and detect this stupid case by issuing a HVC
for HVC_RESET_VECTORS, which does nothing when the stubs
are live. If we get HVC_STUB_ERR back, that's because the
UNDEF handler has kicked in.
In this situation, close your eyes, block your nose, and gracefully
pretend we have booted at EL1.
Reported-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/head.S | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
This comment got me thinking...
.macro host_hvc0
mrs xzr, actlr_el1
.endm
then set HCR_EL2.TACR=1 while we still can and match the ISS against a
constant in the handler. Too awful?
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-13 17:33:55
On Fri, 13 Aug 2021 10:05:40 +0100,
Will Deacon [off-list ref] wrote:
On Thu, Aug 12, 2021 at 08:02:11PM +0100, Marc Zyngier wrote:
quoted
Having HVC disabled from EL3 while the kernel is entered at EL2
is a complete nightmare.
We end-up taking an UNDEF at the worse possible moment (checking
for VHE) and even if we didn't, having KVM enabled would signify
the premature end of the kernel.
Instead, try and detect this stupid case by issuing a HVC
for HVC_RESET_VECTORS, which does nothing when the stubs
are live. If we get HVC_STUB_ERR back, that's because the
UNDEF handler has kicked in.
In this situation, close your eyes, block your nose, and gracefully
pretend we have booted at EL1.
Reported-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/head.S | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
This comment got me thinking...
.macro host_hvc0
mrs xzr, actlr_el1
.endm
then set HCR_EL2.TACR=1 while we still can and match the ISS against a
constant in the handler. Too awful?
I had a similar idea, using an ID register instead (though ACTLR_EL1
is much neater). It would indeed allow the kernel to go back to EL2.
But for what purpose? Yes, we could enable KVM that way. However, the
guest would need to know about it as well, as even simple things like
PSCI wouldn't work. That's would be defining a new guest ABI, and I'm
not overly keen on it.
Unless you are thinking of something else?
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:47
Now that we are able to paper over the gigantic stupidity that
booting at EL2 with SCR_EL3.HCE==0 is, let's taint WARN_TAINT()
when detecting this situation.
Yes, this is *LOUD*.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/virt.h | 10 ++++++++++
arch/arm64/kernel/head.S | 10 ++++++++++
arch/arm64/kernel/smp.c | 3 +++
3 files changed, 23 insertions(+)
@@ -425,6 +425,9 @@ static void __init hyp_mode_check(void)elseif(is_hyp_mode_mismatched())WARN_TAINT(1,TAINT_CPU_OUT_OF_SPEC,"CPU: CPUs started in inconsistent modes");+elseif(__boot_cpu_mode_flags[0]&BOOT_CPU_MODE_DOWNGRADED)+WARN_TAINT(1,TAINT_CPU_OUT_OF_SPEC,+"CPU: CPUs downgraded to EL1, HVC disabled");elsepr_info("CPU: All CPU(s) started at EL1\n");if(IS_ENABLED(CONFIG_KVM)&&!is_kernel_in_hyp_mode()){
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Now that we are able to paper over the gigantic stupidity that
booting at EL2 with SCR_EL3.HCE==0 is, let's taint WARN_TAINT()
when detecting this situation.
Yes, this is *LOUD*.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Tested-by: Rafał Miłecki <rafal@milecki.pl>
This replaces:
CPU: All CPU(s) started at EL1
with:
------------[ cut here ]------------
CPU: CPUs downgraded to EL1, HVC disabled
WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/smp.c:429 smp_cpus_done+0xac/0xe8
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-rc5-g86fc10657896 #41
Hardware name: Asus GT-AC5300 (DT)
pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
pc : smp_cpus_done+0xac/0xe8
lr : smp_cpus_done+0xac/0xe8
sp : ffffffc01002be00
x29: ffffffc01002be00 x28: 0000000000000000 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
x23: ffffffc010ab4000 x22: 0000000000000000 x21: 0000000000000000
x20: ffffffc0107b7e74 x19: ffffffc010a78000 x18: 0000000000000001
x17: ffffffc010a9ee40 x16: 0000000000000000 x15: 000042496b0a18f2
x14: fffffffffffc0f87 x13: 0000000000000039 x12: ffffff80010b03b0
x11: 00000000ffffffea x10: ffffffc010a5eb50 x9 : 0000000000000001
x8 : 0000000000000001 x7 : 0000000000017fe8 x6 : c0000000ffffefff
x5 : 0000000000057fa8 x4 : 0000000000000000 x3 : 0000000000000000
x2 : 00000000ffffffff x1 : a8d68d1fd96fdc00 x0 : 0000000000000000
Call trace:
smp_cpus_done+0xac/0xe8
smp_init+0x68/0x78
kernel_init_freeable+0xd0/0x214
kernel_init+0x24/0x120
ret_from_fork+0x10/0x18
---[ end trace a7d4af835e9d6e6b ]---
BEFORE:
smp: Bringing up secondary CPUs ...
Detected VIPT I-cache on CPU1
CPU1: Booted secondary processor 0x0000000001 [0x420f1000]
Detected VIPT I-cache on CPU2
CPU2: Booted secondary processor 0x0000000002 [0x420f1000]
Detected VIPT I-cache on CPU3
CPU3: Booted secondary processor 0x0000000003 [0x420f1000]
smp: Brought up 1 node, 4 CPUs
SMP: Total of 4 processors activated.
CPU features: detected: 32-bit EL0 Support
CPU features: detected: 32-bit EL1 Support
CPU features: detected: CRC32 instructions
CPU: All CPU(s) started at EL1
AFTER:
smp: Bringing up secondary CPUs ...
Detected VIPT I-cache on CPU1
CPU1: Booted secondary processor 0x0000000001 [0x420f1000]
Detected VIPT I-cache on CPU2
CPU2: Booted secondary processor 0x0000000002 [0x420f1000]
Detected VIPT I-cache on CPU3
CPU3: Booted secondary processor 0x0000000003 [0x420f1000]
smp: Brought up 1 node, 4 CPUs
SMP: Total of 4 processors activated.
CPU features: detected: 32-bit EL0 Support
CPU features: detected: 32-bit EL1 Support
CPU features: detected: CRC32 instructions
------------[ cut here ]------------
CPU: CPUs downgraded to EL1, HVC disabled
WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/smp.c:429 smp_cpus_done+0xac/0xe8
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.14.0-rc5-g86fc10657896 #41
Hardware name: Asus GT-AC5300 (DT)
pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
pc : smp_cpus_done+0xac/0xe8
lr : smp_cpus_done+0xac/0xe8
sp : ffffffc01002be00
x29: ffffffc01002be00 x28: 0000000000000000 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
x23: ffffffc010ab4000 x22: 0000000000000000 x21: 0000000000000000
x20: ffffffc0107b7e74 x19: ffffffc010a78000 x18: 0000000000000001
x17: ffffffc010a9ee40 x16: 0000000000000000 x15: 000042496b0a18f2
x14: fffffffffffc0f87 x13: 0000000000000039 x12: ffffff80010b03b0
x11: 00000000ffffffea x10: ffffffc010a5eb50 x9 : 0000000000000001
x8 : 0000000000000001 x7 : 0000000000017fe8 x6 : c0000000ffffefff
x5 : 0000000000057fa8 x4 : 0000000000000000 x3 : 0000000000000000
x2 : 00000000ffffffff x1 : a8d68d1fd96fdc00 x0 : 0000000000000000
Call trace:
smp_cpus_done+0xac/0xe8
smp_init+0x68/0x78
kernel_init_freeable+0xd0/0x214
kernel_init+0x24/0x120
ret_from_fork+0x10/0x18
---[ end trace a7d4af835e9d6e6b ]---
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-12 19:02:48
It is amazing that we never documented this absolutely basic
requirement: if you boot the kernel at EL2, you'd better
enable the HVC instruction from EL3.
Really, just do it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
Documentation/arm64/booting.rst | 5 +++++
1 file changed, 5 insertions(+)
@@ -212,6 +212,11 @@ Before jumping into the kernel, the following conditions must be met:- The value of SCR_EL3.FIQ must be the same as the one present at boot time whenever the kernel is executing.+ For all systems:+- If EL3 is present and the kernel is entered at EL2:++- SCR_EL3.HCE (bit 8) must be initialised to 0b1.+ For systems with a GICv3 interrupt controller to be used in v3 mode:- If EL3 is present:
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Aug 12, 2021 at 08:02:13PM +0100, Marc Zyngier wrote:
quoted hunk
It is amazing that we never documented this absolutely basic
requirement: if you boot the kernel at EL2, you'd better
enable the HVC instruction from EL3.
Really, just do it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
Documentation/arm64/booting.rst | 5 +++++
1 file changed, 5 insertions(+)
@@ -212,6 +212,11 @@ Before jumping into the kernel, the following conditions must be met:- The value of SCR_EL3.FIQ must be the same as the one present at boot time whenever the kernel is executing.+ For all systems:+- If EL3 is present and the kernel is entered at EL2:++- SCR_EL3.HCE (bit 8) must be initialised to 0b1.+ For systems with a GICv3 interrupt controller to be used in v3 mode:- If EL3 is present:
I'll queue this patch only for now.
A nitpick, I think we should move "For all systems" and "If EL3 is
present..." above the lines describing the SCR_EL3.FIQ requirement (I
can make the change locally).
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Mark Rutland <mark.rutland@arm.com> Date: 2021-08-24 10:52:10
On Tue, Aug 24, 2021 at 11:49:01AM +0100, Catalin Marinas wrote:
On Thu, Aug 12, 2021 at 08:02:13PM +0100, Marc Zyngier wrote:
quoted
It is amazing that we never documented this absolutely basic
requirement: if you boot the kernel at EL2, you'd better
enable the HVC instruction from EL3.
Really, just do it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
Documentation/arm64/booting.rst | 5 +++++
1 file changed, 5 insertions(+)
@@ -212,6 +212,11 @@ Before jumping into the kernel, the following conditions must be met:- The value of SCR_EL3.FIQ must be the same as the one present at boot time whenever the kernel is executing.+ For all systems:+- If EL3 is present and the kernel is entered at EL2:++- SCR_EL3.HCE (bit 8) must be initialised to 0b1.+ For systems with a GICv3 interrupt controller to be used in v3 mode:- If EL3 is present:
I'll queue this patch only for now.
A nitpick, I think we should move "For all systems" and "If EL3 is
present..." above the lines describing the SCR_EL3.FIQ requirement (I
can make the change locally).
FWIW, with that:
Acked-by: Mark Rutland <mark.rutland@arm.com>
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Anyone vaguely familiar with the ARMv8 architecture would quickly
understand that entering the kernel at EL2 without enabling the HVC
instruction is... living dangerously. But as it turns out [0], there
is a whole range of (*cough*) "high quality" (*cough*) Broadcom
systems out there configured exactly like that.
Some Broadcom systems, namely the 4908 and all of those using CFE, they
later switched to u-boot and ATF and got it right.
If you are speechless, I'm right with you.
These machines have stopped being able to boot an upstream kernel
since 5.12, where we changed the way we switch from nVHE to VHE, as
this relies on the HVC instruction being usable... It is also worth
noting that these systems have never been able to use KVM. Or kexec.
This small series addresses the issue by detecting an UNDEFing HVC in
a fairly controlled environment, and in this case pretend that we have
booted at EL1. It also documents the requirement for SCR_EL3.HCE to be
set to *1* if the kernel is entered at EL2. Turns out that we really
have to state the obvious.
This has been tested on a FVP model with a hacked-up boot-wrapper.
Note that I really don't think any of this is -stable material, except
maybe for the documentation. It isn't 5.14 material either. Best case,
this is 5.15, or maybe even later. If ever.
While I am very appreciative of the work you have done here to try to
get the dysfunctional systems to warn and continue to boot, I would
rather we try to load a minimal shim at EL3 capable of fixing up any
incorrect EL3 register setting ahead of loading the kernel provided this
is possible at all on a commercially available system. Rafal, is this
something that CFE allows you to do (as I could not get a straight
answer from that team), if so have you tried it?
--
Florian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
From: Marc Zyngier <maz@kernel.org> Date: 2021-08-15 09:27:36
On Sun, 15 Aug 2021 08:28:47 +0100,
Florian Fainelli [off-list ref] wrote:
On 8/12/2021 9:02 PM, Marc Zyngier wrote:
quoted
Anyone vaguely familiar with the ARMv8 architecture would quickly
understand that entering the kernel at EL2 without enabling the HVC
instruction is... living dangerously. But as it turns out [0], there
is a whole range of (*cough*) "high quality" (*cough*) Broadcom
systems out there configured exactly like that.
Some Broadcom systems, namely the 4908 and all of those using CFE,
they later switched to u-boot and ATF and got it right.
Do we have a list of the affected systems?
quoted
If you are speechless, I'm right with you.
These machines have stopped being able to boot an upstream kernel
since 5.12, where we changed the way we switch from nVHE to VHE, as
this relies on the HVC instruction being usable... It is also worth
noting that these systems have never been able to use KVM. Or kexec.
This small series addresses the issue by detecting an UNDEFing HVC in
a fairly controlled environment, and in this case pretend that we have
booted at EL1. It also documents the requirement for SCR_EL3.HCE to be
set to *1* if the kernel is entered at EL2. Turns out that we really
have to state the obvious.
This has been tested on a FVP model with a hacked-up boot-wrapper.
Note that I really don't think any of this is -stable material, except
maybe for the documentation. It isn't 5.14 material either. Best case,
this is 5.15, or maybe even later. If ever.
While I am very appreciative of the work you have done here to try to
get the dysfunctional systems to warn and continue to boot, I would
rather we try to load a minimal shim at EL3 capable of fixing up any
incorrect EL3 register setting ahead of loading the kernel provided
this is possible at all on a commercially available system.
That would be the best thing to do, and would make the machine fully
usable. I still think we need to have something in the kernel to at
least let the user know that their system is misconfigured though.
If CFE allows a payload to be loaded at EL3 and executed on all CPUs,
that would be absolutely awesome. It would even allow switching over
to ATF...
Thanks,
M.
Rafal, is this something that CFE allows you to do (as I could not
get a straight answer from that team), if so have you tried it?
--
Florian
On Sun, 15 Aug 2021 08:28:47 +0100,
Florian Fainelli [off-list ref] wrote:
quoted
On 8/12/2021 9:02 PM, Marc Zyngier wrote:
quoted
Anyone vaguely familiar with the ARMv8 architecture would quickly
understand that entering the kernel at EL2 without enabling the HVC
instruction is... living dangerously. But as it turns out [0], there
is a whole range of (*cough*) "high quality" (*cough*) Broadcom
systems out there configured exactly like that.
Some Broadcom systems, namely the 4908 and all of those using CFE,
they later switched to u-boot and ATF and got it right.
Do we have a list of the affected systems?
That is going to be hard to come up with given that OEMs such as Asus,
TP-Link, or Netgear and multiple devices are affected. I would start
with this list which is far from exhaustive: Netgear R8000P, TP-Link
Archer C2300, Asus GT AC5300.
quoted
quoted
If you are speechless, I'm right with you.
These machines have stopped being able to boot an upstream kernel
since 5.12, where we changed the way we switch from nVHE to VHE, as
this relies on the HVC instruction being usable... It is also worth
noting that these systems have never been able to use KVM. Or kexec.
This small series addresses the issue by detecting an UNDEFing HVC in
a fairly controlled environment, and in this case pretend that we have
booted at EL1. It also documents the requirement for SCR_EL3.HCE to be
set to *1* if the kernel is entered at EL2. Turns out that we really
have to state the obvious.
This has been tested on a FVP model with a hacked-up boot-wrapper.
Note that I really don't think any of this is -stable material, except
maybe for the documentation. It isn't 5.14 material either. Best case,
this is 5.15, or maybe even later. If ever.
While I am very appreciative of the work you have done here to try to
get the dysfunctional systems to warn and continue to boot, I would
rather we try to load a minimal shim at EL3 capable of fixing up any
incorrect EL3 register setting ahead of loading the kernel provided
this is possible at all on a commercially available system.
That would be the best thing to do, and would make the machine fully
usable. I still think we need to have something in the kernel to at
least let the user know that their system is misconfigured though.
If CFE allows a payload to be loaded at EL3 and executed on all CPUs,
that would be absolutely awesome. It would even allow switching over
to ATF...
Supposedly there are GPL tarballs containing an u-boot that is suitable
for the 4908, however I have no idea where to find those. In premise you
could chainload those to CFE and a get seemingly functional system.
--
Florian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, 12 Aug 2021 20:02:08 +0100, Marc Zyngier wrote:
Anyone vaguely familiar with the ARMv8 architecture would quickly
understand that entering the kernel at EL2 without enabling the HVC
instruction is... living dangerously. But as it turns out [0], there
is a whole range of (*cough*) "high quality" (*cough*) Broadcom
systems out there configured exactly like that.
If you are speechless, I'm right with you.
[...]