-----Original Message-----
From: Alexei Starovoitov <redacted>
Sent: Friday, March 26, 2021 10:25 PM
To: Jianlin Lv <redacted>
Cc: bpf <redacted>; David S. Miller <davem@davemloft.net>;
Jakub Kicinski [off-list ref]; Alexei Starovoitov [off-list ref];
Daniel Borkmann [off-list ref]; Andrii Nakryiko
[off-list ref]; Martin KaFai Lau [off-list ref]; Song Liu
[off-list ref]; Yonghong Song [off-list ref]; John Fastabend
[off-list ref]; KP Singh [off-list ref]; Alexander
Viro [off-list ref]; Andrey Ignatov [off-list ref]; Dmitry
Vyukov [off-list ref]; Nicolas Dichtel
[off-list ref]; Kees Cook [off-list ref];
Masahiro Yamada [off-list ref]; Mahesh Bandewar
[off-list ref]; LKML [off-list ref]; Network
Development [off-list ref]; iecedge@gmail.com; nd
[off-list ref]
Subject: Re: [PATCH bpf-next] bpf: trace jit code when enable
BPF_JIT_ALWAYS_ON

On Fri, Mar 26, 2021 at 5:40 AM Jianlin Lv [off-list ref] wrote:

quoted

When CONFIG_BPF_JIT_ALWAYS_ON is enabled, the value of

bpf_jit_enable

quoted

in /proc/sys is limited to SYSCTL_ONE. This is not convenient for debugging.
This patch modifies the value of extra2 (max) to 2 that support
developers to emit traces on kernel log.

Signed-off-by: Jianlin Lv <redacted>
---
 net/core/sysctl_net_core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c
index d84c8a1b280e..aa16883ac445 100644
--- a/net/core/sysctl_net_core.c
+++ b/net/core/sysctl_net_core.c

@@ -386,7 +386,7 @@ static struct ctl_table net_core_table[] = {
                .proc_handler   = proc_dointvec_minmax_bpf_enable,
 # ifdef CONFIG_BPF_JIT_ALWAYS_ON
                .extra1         = SYSCTL_ONE,
-               .extra2         = SYSCTL_ONE,
+               .extra2         = &two,

"bpftool prog dump jited" is much better way to examine JITed dumps.
I'd rather remove bpf_jit_enable=2 altogether.

In my case, I introduced a bug when I made some adjustments to the arm64
jit macro A64_MOV(), which caused the SP register to be replaced by the
XZR register when building prologue, and the wrong value was stored in fp,
which triggered a crash.

Test case:
modprobe test_bpf test_name="SPILL_FILL"

jited code:
   0:   stp     x29, x30, [sp, #-16]!
        fd 7b bf a9
   4:   mov     x29, xzr//Err, should be 'mov  x29, sp'
        fd 03 1f aa
   8:   stp     x19, x20, [sp, #-16]!
        f3 53 bf a9
   c:   stp     x21, x22, [sp, #-16]!
        f5 5b bf a9
  10:   stp     x25, x26, [sp, #-16]!
        f9 6b bf a9
  14:   mov     x25, xzr//Err,  should be 'mov x25, sp'
        f9 03 1f aa
...
  3c:   mov     x10, #0xfffffffffffffff8        // #-8
        ea 00 80 92
  40:   str     w7, [x25, x10]// Crash
        27 6b 2a b8

This bug is likely to cause the instruction to access the BPF stack in
jited prog to trigger a crash.
I tried to use bpftool to debug, but bpftool crashed when I executed the
"bpftool prog show" command.
The syslog shown that bpftool is loading and running some bpf prog.
because of the bug in the JIT compiler, the bpftool execution failed.

bpf_jit_disasm saved me, it helped me dump the jited image:

echo 2> /proc/sys/net/core/bpf_jit_enable
modprobe test_bpf test_name="SPILL_FILL"
./bpf_jit_disasm -o

So keeping bpf_jit_enable=2 is still very meaningful for developers who
try to modify the JIT compiler.

Jianlin



IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.