On Fri, Jun 25, 2021 at 4:49 PM Qiang Wang
[off-list ref] wrote:
We found that we couldn't get the correct kernel stack from
kretprobe. For example:
bpftrace -e 'kr:submit_bio {print(kstack)}'
Attaching 1 probe...
kretprobe_trampoline+0
kretprobe_trampoline+0
The problem is caused by the wrong instruction register which
points to the address of kretprobe_trampoline in regs.
So we set the real return address in instruction register.
Finally, we tested and successfully fixed it.
bpftrace -e 'kr:submit_bio {print(kstack)}'
Attaching 1 probe...
ext4_mpage_readpages+475
read_pages+139
page_cache_ra_unbounded+417
filemap_get_pages+245
filemap_read+169
__kernel_read+327
bprm_execve+648
do_execveat_common.isra.39+409
__x64_sys_execve+50
do_syscall_64+54
entry_SYSCALL_64_after_hwframe+68
Reported-by: Chengming Zhou <redacted>
Signed-off-by: Qiang Wang <redacted>
Seems like a bug. Maybe we should add a "Fixes" tag here.
quoted hunk
---
kernel/kprobes.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 745f08fdd..1130381ca 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -1899,6 +1899,9 @@ unsigned long __kretprobe_trampoline_handler(struct pt_regs *regs,
current->kretprobe_instances.first = node->next;
node->next = NULL;
+ /* Kretprobe handler expects address is the real return address */
+ instruction_pointer_set(regs, (unsigned long)correct_ret_addr);
+
/* Run them.. */
while (first) {
ri = container_of(first, struct kretprobe_instance, llist);
--2.20.1
On Fri, 25 Jun 2021 18:24:12 +0800
Muchun Song [off-list ref] wrote:
On Fri, Jun 25, 2021 at 4:49 PM Qiang Wang
[off-list ref] wrote:
quoted
We found that we couldn't get the correct kernel stack from
kretprobe. For example:
bpftrace -e 'kr:submit_bio {print(kstack)}'
Attaching 1 probe...
kretprobe_trampoline+0
kretprobe_trampoline+0
The problem is caused by the wrong instruction register which
points to the address of kretprobe_trampoline in regs.
So we set the real return address in instruction register.
Finally, we tested and successfully fixed it.
bpftrace -e 'kr:submit_bio {print(kstack)}'
Attaching 1 probe...
ext4_mpage_readpages+475
read_pages+139
page_cache_ra_unbounded+417
filemap_get_pages+245
filemap_read+169
__kernel_read+327
bprm_execve+648
do_execveat_common.isra.39+409
__x64_sys_execve+50
do_syscall_64+54
entry_SYSCALL_64_after_hwframe+68
Reported-by: Chengming Zhou <redacted>
Signed-off-by: Qiang Wang <redacted>
Seems like a bug. Maybe we should add a "Fixes" tag here.
No, that is not a bug in the kretprobes. If you carefully check
kretprobes provided the rp->addr as ip address. BPF just did not
use it.
Anyway, I already made a same patch in the below series.
https://lore.kernel.org/bpf/162400000592.506599.4695807810528866713.stgit@devnote2/
and you can see that the series is including below 3 patches for that change.
https://lore.kernel.org/bpf/162399997853.506599.13701157683968161733.stgit@devnote2/
https://lore.kernel.org/bpf/162399998747.506599.1115560529431673586.stgit@devnote2/
https://lore.kernel.org/bpf/162399999702.506599.16339931387573094059.stgit@devnote2/
Without these patches, this change will break other arch.
Thanks,
quoted
---
kernel/kprobes.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 745f08fdd..1130381ca 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -1899,6 +1899,9 @@ unsigned long __kretprobe_trampoline_handler(struct pt_regs *regs,
current->kretprobe_instances.first = node->next;
node->next = NULL;
+ /* Kretprobe handler expects address is the real return address */
+ instruction_pointer_set(regs, (unsigned long)correct_ret_addr);
+
/* Run them.. */
while (first) {
ri = container_of(first, struct kretprobe_instance, llist);
--2.20.1
--
Masami Hiramatsu [off-list ref]