On Sun, Jan 4, 2026 at 5:17 AM Menglong Dong [off-list ref] wrote:
quoted hunk ↗ jump to hunk
Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64
to obtain better performance. The instruction we use here is:
65 48 8B 04 25 [offset] // mov rax, gs:[offset]
Signed-off-by: Menglong Dong <redacted>
---
v2:
- check the variable type in emit_ldx_percpu_r0 with __verify_pcpu_ptr
- remove the usage of const_current_task
---
arch/x86/net/bpf_jit_comp.c | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e3b1c4b1d550..f5ff7c77aad7 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1300,6 +1300,25 @@ static void emit_st_r12(u8 **pprog, u32 size, u32 dst_reg, int off, int imm)
emit_st_index(pprog, size, dst_reg, X86_REG_R12, off, imm);
}
+static void __emit_ldx_percpu_r0(u8 **pprog, __force unsigned long ptr)
+{
+ u8 *prog = *pprog;
+
+ /* mov rax, gs:[ptr] */
+ EMIT2(0x65, 0x48);
+ EMIT2(0x8B, 0x04);
+ EMIT1(0x25);
+ EMIT((u32)ptr, 4);
+
+ *pprog = prog;
+}
Why asm?
Let's use BPF_MOV64_PERCPU_REG() similar to the way
BPF_FUNC_get_smp_processor_id inlining is handled.
pw-bot: cr