[PATCH bpf-next v4 1/2] bpf, x86: inline bpf_get_current_task() for x86_64
From: Menglong Dong <hidden>
Date: 2026-01-12 10:45:45
Also in:
lkml
Subsystem:
bpf [core], bpf [general] (safe dynamic programs and tools), the rest · Maintainers:
Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi, Linus Torvalds
Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64 to obtain better performance. In !CONFIG_SMP case, the percpu variable is just a normal variable, and we can read the current_task directly. Signed-off-by: Menglong Dong <redacted> --- v4: - handle the !CONFIG_SMP case v3: - implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in x86_64 JIT. --- kernel/bpf/verifier.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 3d44c5d06623..12e99171afd8 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c@@ -17688,6 +17688,8 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm) switch (imm) { #ifdef CONFIG_X86_64 case BPF_FUNC_get_smp_processor_id: + case BPF_FUNC_get_current_task_btf: + case BPF_FUNC_get_current_task: return env->prog->jit_requested && bpf_jit_supports_percpu_insn(); #endif default:
@@ -23273,6 +23275,33 @@ static int do_misc_fixups(struct bpf_verifier_env *env) insn = new_prog->insnsi + i + delta; goto next_insn; } + + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */ + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) && + verifier_inlines_helper_call(env, insn->imm)) { +#ifdef CONFIG_SMP + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task); + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0); + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); +#else + struct bpf_insn ld_current_addr[2] = { + BPF_LD_IMM64(BPF_REG_0, (unsigned long)¤t_task) + }; + insn_buf[0] = ld_current_addr[0]; + insn_buf[1] = ld_current_addr[1]; + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0); +#endif + cnt = 3; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + goto next_insn; + } #endif /* Implement bpf_get_func_arg inline. */ if (prog_type == BPF_PROG_TYPE_TRACING &&
--
2.52.0