Hello Ian,

Thanks for the review. I will fix the typo in the next version.

I agree that shell test coverage similar to the existing converter tests 
would be useful, and I plan to include that in the next revision.

I will also go through the review comments from Sashiko, validate them, 
and address the necessary fixes in the next version.

Thanks,
Tanushree Shah

On 08/06/26 20:48, Ian Rogers wrote:

On Mon, Jun 8, 2026 at 6:00 AM Tanushree Shah [off-list ref] wrote:

quoted

This RFC patch series introduces support for converting perf.data files
containing tracepoint events into trace.dat format, enabling seamless
visualization and analysis using KerneShark.

Thanks for doing this, this is a useful feature!

nit: typo KernelShark

quoted

======================
Background and Motivation
======================

Currently, perf and trace-cmd operate as separate tracing ecosystems with
incompatible data formats. Users who collect tracepoint data with
'perf record' cannot easily visualize it in KernelShark's graphical
timeline view or leverage trace-cmd's analysis capabilities.

This creates workflow friction when users need to:

- Visualize perf tracepoint data in KernelShark's interactive graphical
   timeline
- Share trace data between perf and trace-cmd workflows and toolchains
- Perform architecture-independent conversion and analysis of traces

This conversion bridge eliminates these barriers by enabling seamless
data exchange between perf and trace-cmd ecosystems, allowing users to
choose the best tool for each analysis phase.

======================
Implementation Overview
======================

The series implements the trace.dat file format specification (version 7)
within perf's data conversion framework.

**Patch 1/4: Core trace.dat Export Infrastructure**
Introduces util/trace-dat.c and util/trace-dat.h implementing:
- Per-CPU raw event buffer management (init, collect, free)
- Ftrace ring buffer page construction
- trace.dat section writers (strings, options, flyrecord sections)

**Patch 2/4: Metadata Integration**
Extends util/trace-event-read.c to write trace.dat metadata during
perf.data
parsing:
- Initial format header (magic, version, endian, page size, compression)
- Section 16: HEADER INFO (header_page + header_event)
- Section 17: FTRACE EVENT FORMATS
- Section 18: EVENT FORMATS (per system/event format files)
- Section 19: KALLSYMS
- Section 21: CMDLINES
- Section 15: STRINGS (written last after all sections)

**Patch 3/4: Conversion Backend**
Implements util/data-convert-trace.c with trace_convert__perf2dat()
function:
- Processes PERF_TYPE_TRACEPOINT samples via process_sample_event()
- Collects raw event data per-CPU using trace_dat__collect_cpu_event()
- Writes OPTIONS sections (CPUCOUNT, TRACECLOCK, metadata offsets)
- Writes FLYRECORD section with per-CPU ring buffer pages

**Patch 4/4: User Interface**
Extends tools/perf/builtin-data.c with --to-trace-dat option:
- Adds command-line option for trace.dat output
- Mutually exclusive with --to-ctf and --to-json
- Calls trace_convert__perf2dat() to perform conversion

======================
Current Implementation Details
======================

**trace.dat Format Version:**
The implementation currently targets trace.dat format version 7, which
is the stable version supported by current trace-cmd releases (v3.x).
This version is hardcoded to ensure compatibility with existing
trace-cmd and KernelShark installations. Future enhancements could add
version negotiation or support for newer format versions as they become
standardized.

**Compression Strategy:**
Compression is explicitly disabled (set to NONE) in the generated
trace.dat files.
This design choice:
- Simplifies the initial implementation and testing
- Ensures maximum compatibility across trace-cmd versions
- Avoids external compression library dependencies

Future work could add support for various compression algorithms (zlib,
zstd, lz4) with runtime selection via command-line options, significantly
reducing file sizes for large traces.

======================
Usage Example
======================

*Record tracepoint events with perf*
perf record -e sched:sched_switch -e sched:sched_wakeup -a sleep 10

*Convert to trace.dat format*
perf data convert --to-trace-dat=output.dat

*Verify trace.dat structure*
trace-cmd dump --summary output.dat

*Analyze with trace-cmd*
trace-cmd report output.dat

*Visualize in KernelShark*
kernelshark output.dat

**Conversion Output:**

[ perf data convert: Converted 'perf.data' into trace.dat format
'output.dat' ]
[ perf data convert: Converted 2684 events ]

**trace-cmd dump --summary Output:**

  Tracing meta data in file output.dat:
         [Initial format]
                 7       [Version]
                 0       [Little endian]
                 8       [Bytes in a long]
                 65536   [Page size, bytes]
                 none    [Compression algorithm]
                         [Compression version]
         [buffer "", "local" clock, 65536 page size, 16 cpus, 1048576 bytes
     flyrecord data]
         [10 options]
         [Saved command lines, 0 bytes]
         [Kallsyms, 0 bytes]
         [Ftrace format, 0 events]
         [Header page, 206 bytes]
         [Header event, 205 bytes]
         [Events format, 1 systems]
         [9 sections]

======================
Testing and Verification
======================

The series has been extensively tested with:
- Various tracepoint events (sched, irq, syscalls, block I/O)
- Mixed recordings containing both tracepoint and non-tracepoint events
   only tracepoints converted)
- Verification with trace-cmd report and KernelShark visualization
- Memory leak testing with Valgrind (0 bytes leaked)
- Cross-architecture testing (x86_64, ppc64le)

It seems that some of this could be a test to give coverage of the
feature. We have similar tests for other convertors:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/shell/test_perf_data_converter_ctf.sh?h=perf-tools-next
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/shell/test_perf_data_converter_json.sh?h=perf-tools-next

I think Sashiko has caught some coding issues, so I'll hold off on a
full review until the churn from Sashiko subsides.

Thanks!
Ian

quoted

All generated trace.dat files successfully open in:
- trace-cmd report (v3.1+)
- KernelShark (v2.0+)

======================
Next Steps
======================

We would highly appreciate reviews, comments, and feedback on:
- The overall architectural approach and integration points
- Compatibility considerations with trace-cmd ecosystem
- Performance characteristics for large-scale traces
- Additional use cases or workflow scenarios
- Future enhancement priorities

Tanushree Shah (4):
   perf/trace-dat: Add trace.dat export infrastructure
   perf/trace-event: Write trace.dat metadata sections during parsing
   perf data-convert: Add perf.data to trace.dat conversion backend
   perf data: Add --to-trace-dat option for converting perf.data
     tracepoint events into trace.dat format

  tools/perf/builtin-data.c            |  38 +-
  tools/perf/util/Build                |   2 +
  tools/perf/util/data-convert-trace.c | 152 ++++++
  tools/perf/util/data-convert.h       |   4 +
  tools/perf/util/trace-dat.c          | 705 +++++++++++++++++++++++++++
  tools/perf/util/trace-dat.h          |  79 +++
  tools/perf/util/trace-event-read.c   | 259 +++++++++-
  7 files changed, 1230 insertions(+), 9 deletions(-)
  create mode 100644 tools/perf/util/data-convert-trace.c
  create mode 100644 tools/perf/util/trace-dat.c
  create mode 100644 tools/perf/util/trace-dat.h

--
2.53.0