Re: [PATCH v2] tracing: Optimize event type allocation with IDA

On Tue, 22 Nov 2022 22:32:58 -0500
Steven Rostedt [off-list ref] wrote:

On Wed, 23 Nov 2022 11:18:06 +0800
Zheng Yejian [off-list ref] wrote:

quoted

But Yujie Liu [off-list ref] reported a problem that highly
reproducible after applying this patch:
Link: https://lore.kernel.org/lkml/54f23c9c-97ae-e326-5873-bfa5d2c81f52@intel.com/ (local)

So please DO NOT apply this patch before I find what happened about it.

I know what the issue is.

The current way of assigning types is to always increment. And not to
reuse until it fills up. And even then, it looks for the next available
number.

I'm guessing the IDA will reuse a number as soon as it is freed. This

Yes, it is.

may also have uncovered a bug, as in reality, we must actually clear
the tracing buffers every time a number is reused.

Thanks for your explanation!

It seems almost the case, and with current way of assigning types, this
problem maybe also happend when reuse type id, am I right?

But instead of clear tracing buffers, would it be better to just mark
that record invalid if we had a way of knowing that the format had changed?

What happens is that the type number is associated to a print format.
That is, the raw data is tagged with the type. This type maps to how to
parse the raw data. If you have a kprobe, it creates a new type number.
If you free it, and create another one. With the IDA, it is likely to
reassign the previously freed number to a new probe.

To explain this better, let's look at the following scenario:

 echo 'p:foo val=$arg1:u64' > kprobe_events
 echo 1 > events/kprobes/foo/enable
 sleep 1
 echo 0 > events/kprobes/foo/enable

 echo 'p:bar val=+0($arg1):string' > kprobe_events

 # foo kprobe is deleted and bar is created and
 # with IDA, bar has the same number for type as foo

 cat trace

When you read the trace, it will see a binary blob representing an
event and marked with a type. Although the event was foo, it will now
map it to bar. And it will read foo's $arg1:u64 as bar's
+0($arg1):string, and will crash.

-- Steve