powerpc: don't select KFENCE on platform PPC_FSL_BOOK3E

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

    BUG: Kernel NULL pointer dereference on read at 0x00000000
    Faulting instruction address: 0xc01de598
    Oops: Kernel access of bad area, sig: 11 [#1]
    BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
    Dumping ftrace buffer:
       (ftrace buffer empty)
    Modules linked in:
    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
    NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
    REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
    MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
    DEAR: 00000000 ESR: 00000000
    GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
    GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
    GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
    GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
    NIP [c01de598] kfence_protect+0x44/0x6c
    LR [c08ae9c4] kfence_init+0xfc/0x2a4
    Call Trace:
    [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
    [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
    [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
    [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
    Instruction dump:
    7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
    41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
    random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
    ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
 arch/powerpc/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index d46db0bfb998..cffd57bcb5e4 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig

@@ -185,7 +185,7 @@ config PPC
 	select HAVE_ARCH_KASAN			if PPC32 && PPC_PAGE_SHIFT <= 14
 	select HAVE_ARCH_KASAN_VMALLOC		if PPC32 && PPC_PAGE_SHIFT <= 14
 	select HAVE_ARCH_KGDB
-	select HAVE_ARCH_KFENCE			if PPC32
+	select HAVE_ARCH_KFENCE			if PPC32 && !PPC_FSL_BOOK3E
 	select HAVE_ARCH_MMAP_RND_BITS
 	select HAVE_ARCH_MMAP_RND_COMPAT_BITS	if COMPAT
 	select HAVE_ARCH_NVRAM_OPS

-- 
2.18.0.huawei.25

Le 24/09/2021 à 08:39, Liu Shixin a écrit :

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

Could you please explain a bit more what the problem is ?

KFENCE has been implemented with the same logic as DEBUG_PAGEALLOC.

DEBUG_PAGEALLOC is enabled on FSL_BOOK3E.

In MMU_setup(), __map_without_ltlbs is set to 1 when KFENCE is enabled.

__map_without_ltlbs should disable the use of tlbcam.


So what's wrong really ?

Does DEBUG_PAGEALLOC work on FSL_BOOK3E ?

Thanks
Christophe

9 more quoted lines

On 2021/9/24 14:41, Christophe Leroy wrote:

Le 24/09/2021 à 08:39, Liu Shixin a écrit :

quoted

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

Could you please explain a bit more what the problem is ?

KFENCE has been implemented with the same logic as DEBUG_PAGEALLOC.

DEBUG_PAGEALLOC is enabled on FSL_BOOK3E.

In MMU_setup(), __map_without_ltlbs is set to 1 when KFENCE is enabled.

__map_without_ltlbs should disable the use of tlbcam.


So what's wrong really ?

Does DEBUG_PAGEALLOC work on FSL_BOOK3E ?

Thanks
Christophe

hi Christophe,

The phenomenon is that kernel panic in the kfence_protect_page function because
__kfence_pool is not mapped with page granularity.

The problem is that in the mapin_ram function, the return value(i.e base) of mmu_mapin_ram
is equal to top. As a result, no level-2 page table is created for [base, top]. It seems that
__map_without_ltlbs didn't diable the use of tlbcam.

I have tried to force page table for all lowmem, then this problem will go away
but the kfence_test failed, which could be explained by the fact that tlbcam is still used.

By the way, DEBUG_PAGEALLOC works well on FSL_BOOK3E without level-2 page table.

Thanks,

quoted

     BUG: Kernel NULL pointer dereference on read at 0x00000000
     Faulting instruction address: 0xc01de598
     Oops: Kernel access of bad area, sig: 11 [#1]
     BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
     Dumping ftrace buffer:
        (ftrace buffer empty)
     Modules linked in:
     CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
     NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
     REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
     MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
     DEAR: 00000000 ESR: 00000000
     GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
     GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
     GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
     GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
     NIP [c01de598] kfence_protect+0x44/0x6c
     LR [c08ae9c4] kfence_init+0xfc/0x2a4
     Call Trace:
     [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
     [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
     [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
     [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
     Instruction dump:
     7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
     41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
     random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
     ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
  arch/powerpc/Kconfig | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index d46db0bfb998..cffd57bcb5e4 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig

@@ -185,7 +185,7 @@ config PPC

11 more quoted lines

On 2021/9/24 14:41, Christophe Leroy wrote:

Le 24/09/2021 à 08:39, Liu Shixin a écrit :

quoted

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

Could you please explain a bit more what the problem is ?

KFENCE has been implemented with the same logic as DEBUG_PAGEALLOC.

DEBUG_PAGEALLOC is enabled on FSL_BOOK3E.

In MMU_setup(), __map_without_ltlbs is set to 1 when KFENCE is enabled.

__map_without_ltlbs should disable the use of tlbcam.


So what's wrong really ?

Does DEBUG_PAGEALLOC work on FSL_BOOK3E ?

Thanks
Christophe

hi Christophe,

In the last e-mail I have said that DEBUG_PAGEALLOC works well on FSL_BOOK3E, that is wrong.
Actually, I add some printed information and find DEBUG_PAGEALLOC Skip all pages.
So neither DEBUG_PAGEALLOC nor kfence is really working.

diff --git a/mm/memory.c b/mm/memory.c
index 25fc46e87214..37fb1b14bc3f 100644
--- a/mm/memory.c
+++ b/mm/memory.c

@@ -2564,8 +2564,10 @@ static int apply_to_pmd_range(struct mm_struct *mm, pud_t *pud,
        }
        do {
                next = pmd_addr_end(addr, end);
-               if (pmd_none(*pmd) && !create)
+               if (pmd_none(*pmd) && !create) {
+                       pr_info("---------pmd_none----------\n");
                        continue;
+               }
                if (WARN_ON_ONCE(pmd_leaf(*pmd)))
                        return -EINVAL;
                if (!pmd_none(*pmd) && WARN_ON_ONCE(pmd_bad(*pmd))) {

quoted

     BUG: Kernel NULL pointer dereference on read at 0x00000000
     Faulting instruction address: 0xc01de598
     Oops: Kernel access of bad area, sig: 11 [#1]
     BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
     Dumping ftrace buffer:
        (ftrace buffer empty)
     Modules linked in:
     CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
     NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
     REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
     MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
     DEAR: 00000000 ESR: 00000000
     GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
     GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
     GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
     GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
     NIP [c01de598] kfence_protect+0x44/0x6c
     LR [c08ae9c4] kfence_init+0xfc/0x2a4
     Call Trace:
     [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
     [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
     [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
     [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
     Instruction dump:
     7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
     41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
     random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
     ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
  arch/powerpc/Kconfig | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index d46db0bfb998..cffd57bcb5e4 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig

12 more quoted lines

kindly ping.


On 2021/9/24 14:39, Liu Shixin wrote:

11 more quoted lines

Le 12/10/2021 à 03:43, Liu Shixin a écrit :

kindly ping.

Hi

Based on the discussion we had, this patch is not enough. It should at 
least also de-activate DEBUG_PAGEALLOC,

However I'm looking at fixing it the other way round. Give me one week 
or two.

Christophe

On 2021/9/24 14:39, Liu Shixin wrote:

quoted

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

     BUG: Kernel NULL pointer dereference on read at 0x00000000
     Faulting instruction address: 0xc01de598
     Oops: Kernel access of bad area, sig: 11 [#1]
     BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
     Dumping ftrace buffer:
        (ftrace buffer empty)
     Modules linked in:
     CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
     NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
     REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
     MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
     DEAR: 00000000 ESR: 00000000
     GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
     GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
     GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
     GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
     NIP [c01de598] kfence_protect+0x44/0x6c
     LR [c08ae9c4] kfence_init+0xfc/0x2a4
     Call Trace:
     [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
     [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
     [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
     [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
     Instruction dump:
     7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
     41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
     random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
     ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
  arch/powerpc/Kconfig | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

15 more quoted lines

Liu Shixin [off-list ref] writes:

kindly ping.

I was under the impression you were trying to debug why it wasn't
working with Christophe.

cheers

On 2021/9/24 14:39, Liu Shixin wrote:

quoted

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

    BUG: Kernel NULL pointer dereference on read at 0x00000000
    Faulting instruction address: 0xc01de598
    Oops: Kernel access of bad area, sig: 11 [#1]
    BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
    Dumping ftrace buffer:
       (ftrace buffer empty)
    Modules linked in:
    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
    NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
    REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
    MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
    DEAR: 00000000 ESR: 00000000
    GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
    GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
    GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
    GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
    NIP [c01de598] kfence_protect+0x44/0x6c
    LR [c08ae9c4] kfence_init+0xfc/0x2a4
    Call Trace:
    [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
    [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
    [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
    [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
    Instruction dump:
    7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
    41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
    random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
    ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
 arch/powerpc/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig

12 more quoted lines

Le 12/10/2021 à 08:24, Michael Ellerman a écrit :

Liu Shixin [off-list ref] writes:

quoted

kindly ping.

I was under the impression you were trying to debug why it wasn't
working with Christophe.

The investigation was a bit dormant to be honest since Liu confirmed that neither KFENCE not DEBUG_PAGEALLOC works.

I now looked at the effort to make it work, and it is not trivial.
At the time being, all linear space is mapped with pinned TLBs and everything is setup for space 0, with space 1 being used temporarily when doing heavy changes to space 0.

We can't use standard pages for linear space on space 0 because we need memory mapped at all time for exceptions (on booke exception run with MMU on in space 0).

In order to use standard pages, we'd need to reorganise the kernel to have it run mostly in space 1 (for data at least) where we would map almost everything with standard pages, and keep pinned TLB to map linear space on space 0 for TLB miss exceptions. Then we'd do more or less like book3s/32 and switch back into space 1 into other exceptions prolog.

That could be good to do it as we could maybe have more code in common with non booke 32 bits, but it is not a trivial job.

So I suggest that for now, we just make KFENCE and DEBUG_PAGEALLOC unselectable for booke/32 (e500 and 44x).

Christophe

cheers

quoted

On 2021/9/24 14:39, Liu Shixin wrote:

quoted

On platform PPC_FSL_BOOK3E, all lowmem is managed by tlbcam. That means
we didn't really map the kfence pool with page granularity. Therefore,
if KFENCE is enabled, the system will hit the following panic:

     BUG: Kernel NULL pointer dereference on read at 0x00000000
     Faulting instruction address: 0xc01de598
     Oops: Kernel access of bad area, sig: 11 [#1]
     BE PAGE_SIZE=4K SMP NR_CPUS=4 MPC8544 DS
     Dumping ftrace buffer:
        (ftrace buffer empty)
     Modules linked in:
     CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.12.0-rc3+ #298
     NIP:  c01de598 LR: c08ae9c4 CTR: 00000000
     REGS: c0b4bea0 TRAP: 0300   Not tainted  (5.12.0-rc3+)
     MSR:  00021000 <CE,ME>  CR: 24000228  XER: 20000000
     DEAR: 00000000 ESR: 00000000
     GPR00: c08ae9c4 c0b4bf60 c0ad64e0 ef720000 00021000 00000000 00000000 00000200
     GPR08: c0ad5000 00000000 00000000 00000004 00000000 008fbb30 00000000 00000000
     GPR16: 00000000 00000000 00000000 00000000 c0000000 00000000 00000000 00000000
     GPR24: c08ca004 c08ca004 c0b6a0e0 c0b60000 c0b58f00 c0850000 c08ca000 ef720000
     NIP [c01de598] kfence_protect+0x44/0x6c
     LR [c08ae9c4] kfence_init+0xfc/0x2a4
     Call Trace:
     [c0b4bf60] [efffe160] 0xefffe160 (unreliable)
     [c0b4bf70] [c08ae9c4] kfence_init+0xfc/0x2a4
     [c0b4bfb0] [c0894d3c] start_kernel+0x3bc/0x574
     [c0b4bff0] [c0000470] set_ivor+0x14c/0x188
     Instruction dump:
     7c0802a6 8109d594 546a653a 90010014 54630026 39200000 7d48502e 2c0a0000
     41820010 554a0026 5469b53a 7d295214 <81490000> 38831000 554a003c 91490000
     random: get_random_bytes called from print_oops_end_marker+0x40/0x78 with crng_init=0
     ---[ end trace 0000000000000000 ]---

Signed-off-by: Liu Shixin <redacted>
---
  arch/powerpc/Kconfig | 2 +-

15 more quoted lines

Christophe Leroy [off-list ref] writes:

Le 12/10/2021 à 08:24, Michael Ellerman a écrit :

quoted

Liu Shixin [off-list ref] writes:

quoted

kindly ping.

I was under the impression you were trying to debug why it wasn't
working with Christophe.

The investigation was a bit dormant to be honest since Liu confirmed 
that neither KFENCE not DEBUG_PAGEALLOC works.

No worries. Sorry it fell to you to do the investigation.

I now looked at the effort to make it work, and it is not trivial.
At the time being, all linear space is mapped with pinned TLBs and 
everything is setup for space 0, with space 1 being used temporarily 
when doing heavy changes to space 0.

We can't use standard pages for linear space on space 0 because we need 
memory mapped at all time for exceptions (on booke exception run with 
MMU on in space 0).

In order to use standard pages, we'd need to reorganise the kernel to 
have it run mostly in space 1 (for data at least) where we would map 
almost everything with standard pages, and keep pinned TLB to map linear 
space on space 0 for TLB miss exceptions. Then we'd do more or less like 
book3s/32 and switch back into space 1 into other exceptions prolog.

That could be good to do it as we could maybe have more code in common 
with non booke 32 bits, but it is not a trivial job.

So I suggest that for now, we just make KFENCE and DEBUG_PAGEALLOC 
unselectable for booke/32 (e500 and 44x).

Yep seems reasonable.

cheers

Le 13/10/2021 à 02:48, Michael Ellerman a écrit :

Christophe Leroy [off-list ref] writes:

quoted

Le 12/10/2021 à 08:24, Michael Ellerman a écrit :

quoted

Liu Shixin [off-list ref] writes:

quoted

kindly ping.

I was under the impression you were trying to debug why it wasn't
working with Christophe.

The investigation was a bit dormant to be honest since Liu confirmed
that neither KFENCE not DEBUG_PAGEALLOC works.

No worries. Sorry it fell to you to do the investigation.

No problem.

quoted

I now looked at the effort to make it work, and it is not trivial.
At the time being, all linear space is mapped with pinned TLBs and
everything is setup for space 0, with space 1 being used temporarily
when doing heavy changes to space 0.

We can't use standard pages for linear space on space 0 because we need
memory mapped at all time for exceptions (on booke exception run with
MMU on in space 0).

In order to use standard pages, we'd need to reorganise the kernel to
have it run mostly in space 1 (for data at least) where we would map
almost everything with standard pages, and keep pinned TLB to map linear
space on space 0 for TLB miss exceptions. Then we'd do more or less like
book3s/32 and switch back into space 1 into other exceptions prolog.

That could be good to do it as we could maybe have more code in common
with non booke 32 bits, but it is not a trivial job.

So I suggest that for now, we just make KFENCE and DEBUG_PAGEALLOC
unselectable for booke/32 (e500 and 44x).

Yep seems reasonable.

We also have a problem with STRICT_KERNEL_RWX as it is based on the same principles until someone implements it by blocks like book3s/32 and 8xx.

So it should also be unselectable on e500 and 44x for now.

Christophe