How to Trace TCP Socket TCP_TIMESTAMP Precision in Linux Using eBPF

Accurate Round-Trip Time (RTT) measurement and Protection Against Wrapped Sequences (PAWS) in TCP connections depend fundamentally on the TCP Timestamps option (RFC 7323). Historically, the Linux kernel generated TCP timestamps using internal kernel jiffies (typically 1ms resolution at 1000Hz). In modern ultra-low-latency data center networks, sub-millisecond precision is required. Recent Linux kernels introduce microsecond-resolution TCP timestamps. Tracing timestamp option generation and echo reflection dynamics with eBPF gives network engineers critical visibility into TCP clock drift and RTT calculations.

Architecture of TCP Timestamps in the Linux Network Stack

When timestamps are enabled (via sysctl net.ipv4.tcp_timestamps), each outbound TCP segment carries an 8-byte payload inside TCP option kind 8: the sender timestamp (TSval) and the reflected echo timestamp (TSecr). When an incoming segment arrives, the receiver records TSval and echoes it back in the subsequent ACK segment.

The kernel calculates smoothed RTT (tp->srtt_us) by subtracting TSecr from the current timestamp clock. If remote endpoints mangle timestamp values or clock frequencies drift, congestion control algorithms (such as BBR or CUBIC) receive distorted RTT telemetry, leading to premature throughput throttling or unhandled buffer bloat.

Authoring a Custom bpftrace Timestamp Tracing Script

To inspect TCP timestamp options directly inside the kernel transmission and reception paths, create an eBPF script using bpftrace:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_v4_send_ack
{
  $sk = (struct sock *)arg0;
  $tp = (struct tcp_sock *)arg0;
  
  if ($tp->rx_opt.tstamp_ok) {
    @ts_val_sent[$sk] = $tp->tcp_clock_cache;
    @ts_ecr_sent[$sk] = $tp->rx_opt.rcv_tsval;
  }
}

kprobe:tcp_rcv_rtt_update
{
  $tp = (struct tcp_sock *)arg0;
  $rtt_sample_us = arg1;

  @rtt_hist_us = hist($rtt_sample_us);
}

interval:s:10
{
  print(@rtt_hist_us);
  clear(@rtt_hist_us);
}

Save this script as trace_timestamps.bt. It inspects outgoing ACK segments to track active TSval and reflected TSecr values while capturing live RTT sample distributions from tcp_rcv_rtt_update().

Executing and Evaluating Kernel Telemetry

Run the bpftrace instrumentation script with root privileges:

sudo bpftrace trace_timestamps.bt

Observe the RTT sample histogram and verify timestamp behavior:

• RTT distribution: Verify that @rtt_hist_us displays a tight latency cluster corresponding to known physical link latency. Outliers indicate ACK compression or delayed ACK scheduling.
• Clock monotonicity: Ensure that TSval values strictly increment without retrograding, which would trigger PAWS drops in tcp_paws_discard().
• Timestamp reflection integrity: Cross-reference with nstat -z | grep -i paws to confirm zero packet drops due to invalid timestamp sequences.

Configuring TCP Timestamp Sysctls in Linux

To tune TCP timestamp behaviour on modern production Linux servers:

1. Inspect active timestamp configuration via sysctl net.ipv4.tcp_timestamps. A value of 1 enables standard RFC 7323 timestamps, while 2 or higher enables high-resolution randomized offsets.
2. To reduce attack surface against uptime fingerprinting while retaining RTT benefits, modern kernels automatically add random per-connection offsets to TSval.
3. Ensure network hardware timestamp offloading does not conflict with kernel TCP timestamp options on high-throughput interfaces.