Modern high-performance processors execute instructions speculatively across hundreds of pipeline stages, relying on branch prediction units to forecast conditional branches, loops, and polymorphic function dispatch. When a processor mispredicts a branch, the entire speculative pipeline must be flushed and restarted, resulting in severe CPU cycle penalties. While standard statistical profiling with perf record samples instruction pointers at periodic counter intervals, it cannot reconstruct the exact sequence of preceding control-flow decisions that led to a bottleneck or cache miss.
Hardware Last Branch Record (LBR) solves this challenge by leveraging dedicated on-chip ring buffers built into modern Intel and AMD processors. With LBR enabled, the CPU continuously records source and target addresses of recently executed branches in hardware with virtually zero runtime overhead. Using the Linux perf tool with the -b (branch stack) flag, developers can capture continuous control-flow histories, identify hot branch paths, and profile deeply nested function call chains even in binaries compiled without frame pointers. This guide explains how LBR works, how to record branch stacks, analyze branch misprediction penalties, and optimize binary layout with profile-guided optimization.
Understanding Hardware Last Branch Record Architecture
The Last Branch Record mechanism operates independently of the main CPU execution pipeline. Modern processors feature an internal circular ring of Model-Specific Registers (MSRs)—often storing between 16 and 64 recent branch records:
FROM_IPandTO_IPRegisters: For every jump, conditional branch, call, or return instruction, the hardware logs the departure memory address (FROM_IP) and arrival destination address (TO_IP).- Branch Prediction Metadata: Each record includes a
MISPREDbit indicating whether the hardware branch prediction unit correctly anticipated the destination, alongside cycle counters measuring elapsed time between branches. - Hardware Filtering Capabilities: Rather than recording every branch, processor MSRs can be programmed to filter branches at the hardware level—recording only user-space branches (
u), kernel branches (k), function calls (call), or returns (ret). - Frame Pointer Independence: Because LBR logs physical call and return instructions directly,
perfcan reconstruct accurate call graphs even if code was compiled with-fomit-frame-pointerand lacks DWARF debugging sections.
Verifying CPU LBR Support on Linux
Before recording branch stacks, verify that your CPU architecture supports hardware branch recording and that virtualised hypervisors have not disabled MSR passthrough. Check dmesg and CPU capabilities:
# Check kernel log for LBR PMU driver detection
dmesg | grep -i "last branch record\|intel_pmu.*lbr"
To confirm that perf can program branch filters, execute a quick test validation:
# Test basic user-space branch recording on a simple command
perf record -b -e cycles:u -- true
If the command exits with [ perf record: Captured and wrote ... ] without returning an “Operation not supported” error, hardware LBR is fully active.
Recording Execution Paths Using perf record -b
To profile a workload and capture branch histories, invoke perf record with the -b flag (equivalent to --branch-filter any):
# Profile a specific application with user-space branch stack sampling
sudo perf record -b -e cycles:u -- ./high_performance_engine --workload=compute
To refine what the hardware logs and minimize file sizes, pass specific branch filter masks using --branch-filter:
any,u: All branch types occurring in user-space code.any_call,u: Direct and indirect function calls in user space.any_ret,u: Return instructions in user space.ind_call,u: Indirect calls through function pointers or C++ virtual method tables (crucial for optimizing interface dispatch).
For example, to trace only indirect function calls and virtual table lookups:
# Record user-space indirect calls and function pointer jumps
sudo perf record -j ind_call,u -e cycles:u -- ./high_performance_engine
Analyzing Branch Execution and Call Histories
Once profiling is complete, parse the recorded perf.data file using perf report. Standard analysis displays top individual instructions, but adding branch flags exposes sequence flow:
# Display sorted list of branch transitions
perf report --sort from,to,mispred -F overhead,from,to,mispred --stdio
The console displays the most frequently executed branch pairs along with their misprediction percentages:
# Overhead From To Mispredicted
# ........ .......................... .......................... ............
#
14.28% [.] process_event+0x42 [.] handle_data_packet+0x0 N
8.12% [.] validate_token+0x18 [.] reject_connection+0x0 Y
6.45% [.] parse_stream+0x8c [.] buffer_flush+0x0 N
Visualizing Reconstructed Call Stacks via –branch-history
One of the most powerful capabilities of LBR is reconstructing the exact call chain leading into a sample without walking the stack. Use --branch-history:
# Interactively inspect execution call flow using branch history
perf report --branch-history
The resulting call graph shows the preceding 16 to 32 function transitions that preceded the PMU interrupt, allowing you to trace complex polymorphic logic without incurring stack-walking overhead.
Extracting Raw LBR Traces via perf script
For automated pipeline analysis or feeding profile data into compiler optimization tools (such as AutoFDO or LLVM BOLT), export raw branch traces using perf script:
# Dump chronological branch transitions with cycle timestamps
perf script -F ip,brstack
Each output line reveals the circular chain of branch jumps:
0x401254 0x401230/0x401250/P/-/-/1 0x4011f0/0x401210/M/-/-/2 ...
Here, each bracket represents FROM/TO/PRED/ABORT/CYCLES, detailing whether the branch was predicted (P) or mispredicted (M), alongside cycle latencies.
Troubleshooting LBR Profiling Failures
When working with hardware branch recording on Linux, watch out for these platform restrictions:
- Virtual Machine Hypervisor Limitations: Most public cloud virtual machines (such as standard AWS EC2 or Google Cloud compute instances) disable LBR MSR virtualization by default. If
perf record -breturnsnot supported, you must run profiling on bare-metal cloud instances or dedicated physical hardware. - Simultaneous Watchdogs or NMI Conflicts: If another diagnostic profiler or kernel lockup detector (such as the hard lockup watchdog) holds specific performance monitoring MSRs, branch recording may fail. Temporarily disable the kernel watchdog using
sudo sysctl -w kernel.nmi_watchdog=0. - Privileged Access and perf_event_paranoid: Accessing raw branch registers requires elevated permissions. Set the paranoid threshold to allow hardware counter access via
sudo sysctl -w kernel.perf_event_paranoid=1.
Using perf record -b and Last Branch Record provides software engineers with an unprecedented window into low-level CPU control flow, enabling targeted optimizations that eliminate branch misprediction bottlenecks.