Thread (23 messages) 23 messages, 3 authors, 2d ago

Re: [RFC 0/2] arm64: kprobes: Fix single-step fault and reentry handling

From: Pu Hu <hidden>
Date: 2026-07-01 12:30:45
Also in: linux-arm-kernel, lkml

Dear Maintainers,

I would like to provide some additional background for this patchset.

We observed a high-probability crash on an Android device running a
6.1.145-based kernel when recording preemptirq tracepoints for a user
space process with dwarf callchains enabled.

The command used to reproduce the issue is:

   simpleperf record -p <PID> -f 10000 \
     -e preemptirq:preempt_disable \
     -e preemptirq:preempt_enable \
     --duration 9 --call-graph dwarf \
     -o /data/local/tmp/perf.data

Here <PID> is the PID of a user space process, for example a foreground
application UI thread or RenderThread.

One important observation is that the crash does not reproduce if
"--call-graph dwarf" is removed.

The crash log shows a data abort on a user virtual address while the PC
is at a probed kernel instruction:

   [  297.177775] Unable to handle kernel paging request at virtual 
address 0000007ff042e000
   [  297.177792] Mem abort info:
   [  297.177795]   ESR = 0x0000000096000007
   [  297.177799]   EC = 0x25: DABT (current EL), IL = 32 bits
   [  297.177803]   SET = 0, FnV = 0
   [  297.177806]   EA = 0, S1PTW = 0
   [  297.177808]   FSC = 0x07: level 3 translation fault
   [  297.177811] Data abort info:
   [  297.177814]   ISV = 0, ISS = 0x00000007
   [  297.177817]   CM = 0, WnR = 0
   [  297.177820] user pgtable: 4k pages, 39-bit VAs, pgdp=000000098c9f2000
   [  297.177825] [0000007ff042e000] pgd=08000009aaaea003, 
p4d=08000009aaaea003, pud=08000009aaaea003, pmd=08000000abca0003, 
pte=0000000000000000
   [  297.177835] Internal error: Oops: 0000000096000007 [#1] PREEMPT SMP
   [  297.178070] Skip md ftrace buffer dump for: 0x2800d70
   ...
   [  297.178485] CPU: 6 PID: 10214 Comm: id.article.news Tainted: P S 
    W  O       6.1.145-android14-11-maybe-dirty-qki-consolidate #1
   [  297.178489] Hardware name: Qualcomm Technologies, Inc. Volcano 
QRD,x6878 (DT)
   [  297.178491] pstate: 22400005 (nzCv daif +PAN -UAO +TCO -DIT -SSBS 
BTYPE=--)
   [  297.178493] pc : folio_wait_bit_common+0x0/0x408
   [  297.178499] lr : perf_output_sample+0x57c/0xacc
   [  297.178502] sp : ffffffc0366c2f90
   [  297.178503] x29: ffffffc0366c2fb0 x28: 0000000000001000 x27: 
0000007ff042d5f8
   [  297.178507] x26: 00000000000035e7 x25: 0000000000000000 x24: 
ffffff892cec3000
   [  297.178510] x23: 0000000000001000 x22: 0000000000009370 x21: 
ffffffc0366c3140
   [  297.178512] x20: ffffff888aa1a180 x19: ffffffc0366c3020 x18: 
ffffffe01103b340
   [  297.178515] x17: 00000000ad6b63b6 x16: 00000000ad6b63b6 x15: 
0000007ff042d5f8
   [  297.178518] x14: 0000000000000000 x13: 003436737365636f x12: 
72705f7070612f6e
   [  297.178520] x11: 69622f6d65747379 x10: 732f0030333d7972 x9 : 
616d6972705f6c6f
   [  297.178523] x8 : 6f705f706173755f x7 : 54454b434f535f44 x6 : 
ffffff892cec39d8
   [  297.178526] x5 : ffffff892cec4000 x4 : 0000000000000008 x3 : 
6e6f6973736e6172
   [  297.178528] x2 : 00000000000005b8 x1 : 0000007ff042e000 x0 : 
ffffff892cec3000
   [  297.178531] Call trace:
   [  297.178532]  folio_wait_bit_common+0x0/0x408
   [  297.178535]  perf_event_output_forward+0x90/0xdc
   [  297.178537]  __perf_event_overflow+0x128/0x1e8
   [  297.178540]  perf_swevent_event+0x94/0x1a0
   [  297.178543]  perf_tp_event+0x140/0x270
   [  297.178545]  perf_trace_run_bpf_submit+0x84/0xe0
   [  297.178547]  perf_trace_preemptirq_template+0xe8/0x124
   [  297.178553]  trace_preempt_on+0xec/0x150
   [  297.178555]  preempt_count_sub+0xa8/0x12c
   [  297.178562]  do_debug_exception+0xd0/0x148
   [  297.178568]  el1_dbg+0x64/0x80
   [  297.178575]  el1h_64_sync_handler+0x3c/0x90
   [  297.178577]  el1h_64_sync+0x68/0x6c
   [  297.178579]  folio_wait_bit_common+0x0/0x408
   [  297.178582]  __get_node_page+0xdc/0x49c
   [  297.178587]  f2fs_get_dnode_of_data+0x404/0x950
   [  297.178589]  f2fs_map_blocks+0x1e0/0xdf8
   [  297.178591]  f2fs_mpage_readpages+0x1f0/0x8d0
   [  297.178594]  f2fs_readahead+0x84/0x10c
   [  297.178596]  read_pages+0xb8/0x434
   [  297.178603]  page_cache_ra_unbounded+0x9c/0x2f0
   [  297.178605]  page_cache_ra_order+0x2b0/0x348
   [  297.178608]  do_sync_mmap_readahead+0xd0/0x228
   [  297.178612]  filemap_fault+0x158/0x46c
   [  297.178615]  f2fs_filemap_fault+0x28/0x114
   [  297.178617]  handle_mm_fault+0x4f8/0x1468
   [  297.178620]  do_page_fault+0x208/0x4b8
   [  297.178622]  do_translation_fault+0x38/0x54
   [  297.178624]  do_mem_abort+0x58/0x118
   [  297.178626]  el0_da+0x48/0xb8
   [  297.178629]  el0t_64_sync_handler+0x98/0xb4
   [  297.178632]  el0t_64_sync+0x1a4/0x1a8
   [  297.178634] Code: 94000004 a8c17bfd d50323bf d65f03c0 (d4200080)
   [  297.178639] ---[ end trace 0000000000000000 ]---

The instruction d4200080 is the kprobe BRK instruction. The stack also
shows that the fault happens while handling a kprobe debug exception, 
and the perf/trace path is entered from that window.

 From the fulldump analysis, the issue appears to be related to the arm64
kprobe single-step/reentry handling. While a kprobe is preparing or
executing its XOL single-step instruction, perf/trace code may run in 
the same window. With dwarf callchains enabled, this path may also 
access user memory and take a data abort. In addition, another kprobe 
may be hit while the first kprobe is still in KPROBE_HIT_SS state.

This matches the type of issue that was fixed on x86 by the following
commits:

   6381c24cd6d5 ("kprobes/x86: Fix page-fault handling logic")
   6a5022a56ac3 ("kprobes/x86: Allow to handle reentered kprobe on 
single-stepping")

This patchset applies the same idea to arm64:

   - Patch 1 makes the arm64 kprobe fault handler handle a fault in
     KPROBE_HIT_SS/KPROBE_REENTER only when the faulting PC is the current
     kprobe's XOL instruction. Otherwise, the fault is left to the normal
     fault handling path.

   - Patch 2 allows a kprobe hit in KPROBE_HIT_SS to be handled as a
     recoverable one-level reentry. The unrecoverable case remains a hit
     while already in KPROBE_REENTER.

With both patches applied, we have kept the same stress test running for
three days and the crash is no longer reproduced.

I still have the full dmesg and fulldump from the crash device. Please
let me know if any additional information would be useful.

Thanks,
hupu
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help