How to Trace TCP Socket TCP_REPAIR_OPTIONS in Linux Using eBPF

Stateful container migration and live process checkpointing frameworks—such as CRIU (Checkpoint/Restore in Userspace)—rely on Linux TCP repair mode to pause, extract, and reconstruct active network connections without terminating established TCP sessions. A critical component of this reconstruction process is the TCP_REPAIR_OPTIONS socket option, which allows userspace migration engines to serialize and inject raw TCP header options (such as Window Scale, Timestamps, and SACK Permitted flags). Tracing these low-level socket option manipulations using eBPF provides kernel-level visibility during container migrations.

Mechanics of TCP_REPAIR_OPTIONS Reconstruction

When a socket is switched to repair mode (TCP_REPAIR enabled via setsockopt()), the kernel permits privileged userspace applications to read and write internal protocol state variables that are normally immutable during active connections. Using TCP_REPAIR_OPTIONS, the migration controller injects an array of struct tcp_repair_opt structures:

struct tcp_repair_opt {
    __u32   opt_code;
    __u32   opt_val;
};

The kernel parses these values, updating internal socket flags (such as tp->rx_opt.wscale_ok, tp->rx_opt.tstamp_ok, and tp->rx_opt.sack_ok). If corrupted or invalid option codes are injected, connection restoration fails silently or causes subtle window negotiation corruptions when traffic resumes.

Authoring a Custom bpftrace Socket Inspection Script

To monitor when TCP_REPAIR_OPTIONS is invoked and trace the exact option codes and values being injected, create a bpftrace instrumentation script:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_repair_set_options
{
  $sk = (struct sock *)arg0;
  $opt_code = arg1;
  $opt_val = arg2;

  time("%H:%M:%S: TCP_REPAIR_OPTIONS on sock %p: code=%d, val=%u\n", 
       $sk, $opt_code, $opt_val);
  @option_counts[$opt_code] = count();
}

interval:s:10
{
  print(@option_counts);
  clear(@option_counts);
}

Save this script as trace_repair_options.bt. It targets tcp_repair_set_options() inside the Linux kernel network subsystem, recording timestamped option injections and maintaining aggregate counts per option code.

Executing and Analysing Kernel Repair Telemetry

Execute the bpftrace monitoring script with administrative privileges:

sudo bpftrace trace_repair_options.bt

Trigger a live container checkpoint or CRIU restoration cycle. Evaluate the output telemetry for expected option codes:

• Code 2 (MSS): Injects Maximum Segment Size negotiation limits.
• Code 3 (Window Scale): Restores receive and send window scaling multipliers.
• Code 4 (SACK Permitted): Restores Selective Acknowledgment capability flags.
• Code 8 (Timestamps): Injects TCP timestamp options for RTT calculations and PAWS protection.

Security and Operational Considerations

Because TCP repair options manipulate live kernel socket state directly, strict operational safeguards must be enforced:

1. Executing TCP_REPAIR_OPTIONS requires explicit CAP_NET_ADMIN privileges. Verify that container daemons run with minimal capability bounding sets.
2. Always verify sequence number synchronisation (TCP_REPAIR_QUEUE) in conjunction with option injection to prevent immediate RST packet generation upon connection unfreezing.
3. Ensure firewall state tables (such as Conntrack) are refreshed to reflect restored socket tuples.