Re: tprobe event tracing error

On Mon, 26 Feb 2024 23:41:56 +0900
Masami Hiramatsu (Google) [off-list ref] wrote:

Hi,
(Cc: linux-kernel-trace ML for sharing this knowledge)

On Mon, 26 Feb 2024 16:36:29 +0300
Максим Морсков [off-list ref] wrote:

quoted

Hello, dear Masami.
I am researching Linux event tracing subsystem in part of tprobes,
and found interesting behavior in kernel version 6.6:
 
echo  't:my_fchmodat sys_enter_fchmodat' | 

sudo tee ‘/sys/kernel/tracing/dynamic_events’

quoted

bash: line 1: echo: write error: Invalid argument  

Yeah, I understand that you are confused by this behavior, but it is
actually expected behavior. syscalls:* events looks like trace events
based on tracepoint, but those are software generated trace event.

You can find raw_syscalls:* trace events, that is based on the tracepoint,
and other syscalls:* are based on that raw_syscalls:* trace points.
(IOW, those are a kind of pre-compiled dynamic events)

e.g.

/sys/kernel/tracing # echo "t sys_enter \$arg*" >> dynamic_events 
/sys/kernel/tracing # cat dynamic_events 
t:tracepoints/sys_enter sys_enter regs=regs id=id

/sys/kernel/tracing # echo "t sys_enter_open \$arg*" >> dynamic_events 
sh: write error: Invalid argument
/sys/kernel/tracing # cat error_log 
[  227.981347] trace_fprobe: error: Tracepoint is not found
  Command: t sys_enter_open $arg*
             ^

So, tprobe can not find the hard-coded tracepoints for those dynamically
generated syscall trace events. But raw_syscall sys_enter/sys_exit are OK.

Ah, that's because "tprobes" are attached to tracepoints and not
trace events. If you want to attach to trace events, you need to use eprobes
(which I need to add documentation for!).

 # echo 'e:my_fchmodat syscalls/sys_enter_fchmodat' > dynamic_events

Works.

-- Steve