How to Export Trace Events and Generate Flame Graphs in Linux Using perf script

When profiling high-throughput servers, database engines, or complex user-space runtimes with the Linux perf subsystem, tools such as perf report and perf top provide valuable text-based hierarchical summaries. However, navigating deep call stacks and detecting subtle performance regressions across millions of sampled CPU events can quickly overwhelm terminal interfaces. To facilitate programmatic trace analysis, custom pipeline filtering, and the generation of interactive SVG Flame Graphs, Linux perf provides perf script. This command parses binary perf.data capture files, translates raw hardware counters and tracepoints into human-readable event streams, and allows engineers to pipe stack traces directly into visualization engines or custom Python and Perl post-processing scripts.

Role and Functionality of perf script in Performance Analysis

The perf script tool acts as the primary data extraction layer for the Linux tracing architecture. Rather than aggregating samples into statistical buckets like perf report, perf script reconstructs the chronological timeline of individual sampled events:

  • Chronological Event Streaming: Dumps raw sample records—including precise high-resolution timestamps, process identifiers (PIDs), thread IDs (TIDs), CPU core numbers, instruction pointers, symbol names, and mapped binary paths.
  • Call Graph Normalisation: Unwinds DWARF or frame-pointer call chains, emitting one call stack per sample record formatted for downstream consumers.
  • Custom Field Formatting: Using the -F parameter, developers can selectively emit or suppress specific telemetry fields to streamline trace parsing.
  • Programmable Scripting Engine: Embeds runtime hooks to execute Python or Perl scripts directly against the trace stream, allowing automated detection of latency spikes or lock contention patterns.

Recording High-Resolution Call Graphs for Flame Graph Generation

Before running perf script, record CPU activity using perf record with call graph capture enabled. Frame pointer unwinding (-g) is standard, though DWARF unwinding (--call-graph dwarf) is recommended for stripped binaries or applications compiled with -fomit-frame-pointer:

# Record system-wide CPU samples with frame pointers for 15 seconds
sudo perf record -F 99 -a -g -- sleep 15

# Record CPU samples for an active process using DWARF call chains
sudo perf record -F 99 -p 38410 --call-graph dwarf -g -- sleep 20

Sampling at 99 Hertz (-F 99) avoids lockstep synchronization with standard 100 Hz timer interrupts, ensuring unbiased statistical sampling across all CPU cores.

Exporting Raw Trace Events with Custom Field Filters

Once the binary trace file (perf.data) is captured, execute perf script to inspect raw event data. By default, the output displays extensive metadata for every sample:

# Stream all recorded trace events to the terminal
perf script -i perf.data | head -n 30

# Output specific fields: timestamp, thread ID, CPU core, event name, and symbol
perf script -F time,pid,tid,cpu,event,sym -i perf.data | head -n 20

Review the standard output structure:

# Process Name   PID/TID    [CPU]   Timestamp:        Event:
postgres        18492/18492 [003] 148201.291042: cycles: 
	ffffffffa0123456 do_sys_poll ([kernel.kallsyms])
	ffffffffa0124567 __x64_sys_poll ([kernel.kallsyms])
	00007f9b8c012345 poll (/usr/lib/x86_64-linux-gnu/libc.so.6)
	000055c1e9012345 ServerLoop (/usr/lib/postgresql/bin/postgres)

Generating Interactive SVG Flame Graphs

The most widespread application of perf script is generating Flame Graphs—hierarchical visual representations of code paths where the x-axis represents the population of CPU samples and the y-axis represents call stack depth. To produce an interactive SVG Flame Graph, leverage Brendan Gregg’s open-source FlameGraph toolkit:

# Step 1: Clone the FlameGraph repository if not present
git clone https://github.com/brendangregg/FlameGraph.git

# Step 2: Dump the trace events using perf script
perf script -i perf.data > out.perf

# Step 3: Collapse call stacks into single-line records
./FlameGraph/stackcollapse-perf.pl out.perf > out.folded

# Step 4: Render the collapsed trace into an interactive SVG
./FlameGraph/flamegraph.pl out.folded > cpu_profile.svg

Alternatively, stream the entire pipeline without intermediate disk files:

perf script | ./FlameGraph/stackcollapse-perf.pl | ./FlameGraph/flamegraph.pl > cpu_flamegraph.svg

Open cpu_flamegraph.svg in any modern web browser (such as Safari, Edge, or Chrome). The graphic allows interactive zooming, searching for specific symbol names via regex (highlighting matching frames in magenta), and hovering over boxes to view exact CPU time percentages.

Automating Analysis with Embedded Python and Perl Scripts

The perf script subsystem includes native scripting support. You can list pre-packaged diagnostic scripts or generate template code to process samples programmatically:

# List available built-in perf analysis scripts
perf script -l

# Generate a starter Python script based on events in perf.data
perf script -g python

# Execute a custom Python analysis script against the trace
perf script -s custom_analyzer.py -i perf.data

Using perf script decouples raw kernel event collection from post-processing workflows, allowing developers and DevOps engineers to automate performance regression tracking and build rich visualization pipelines across production Linux environments.

Get the best tech tips delivered straight to your inbox.

Join thousands of readers mastering Apple, Google, Microsoft, and Linux.