How to Trace TCP Socket TCP_RACK Loss Detection in Linux Using eBPF

Traditional TCP loss detection mechanisms (such as standard duplicate ACK thresholds in RFC 5681) perform poorly in modern data centers and cellular networks where packet reordering is common or flight sizes are too small to generate three duplicate ACKs. To resolve this, RFC 8985 defines RACK (Recent Acknowledgment), a time-based loss detection algorithm implemented in the Linux kernel network stack. Rather than counting packet sequences, RACK uses ACK timestamps to detect packet losses dynamically. Tracing RACK loss heuristics and reordering windows using eBPF provides kernel-level visibility into retransmission accuracy.

Mechanics of RACK Time-Based Loss Detection

RACK operates by tracking the transmission and acknowledgment timestamps of in-flight segments. When a later transmitted packet is selectively acknowledged (SACKed), RACK infers that an earlier unacknowledged packet is lost if:

time_since_sent > RTT + reordering_window

The reordering window (rack.rtt_min / 4) dynamically expands if packet reordering is observed on the path. This time-based approach allows Linux to detect losses swiftly even when flight sizes are small, without triggering spurious retransmissions during transient link reordering.

Authoring a Custom bpftrace RACK Telemetry Script

To inspect when RACK detects loss events and monitor reordering window adjustments, create an eBPF script using bpftrace:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_rack_detect_loss
{
  $sk = (struct sock *)arg0;
  $tp = (struct tcp_sock *)arg0;

  @rack_loss_checks = count();
  if ($tp->rack.reord) {
    @rack_reordering_detected = count();
  }
}

kprobe:tcp_rack_reo_timeout
{
  $sk = (struct sock *)arg0;
  time("%H:%M:%S: RACK reordering timer fired on sock %p\n", $sk);
  @rack_reo_timeouts = count();
}

interval:s:10
{
  print(@rack_loss_checks);
  print(@rack_reordering_detected);
  print(@rack_reo_timeouts);
  clear(@rack_loss_checks);
  clear(@rack_reordering_detected);
  clear(@rack_reo_timeouts);
}

Save this script as trace_rack.bt. It targets tcp_rack_detect_loss() and tcp_rack_reo_timeout() to record loss evaluations and reordering timeouts.

Executing and Evaluating Kernel Telemetry

Run the bpftrace monitoring script with administrative privileges:

sudo bpftrace trace_rack.bt

Subject the network interface to bursty traffic or synthetic reordering tests. Review the telemetry metrics:

• Loss check frequency: A high count in @rack_loss_checks during SACK processing confirms active RACK loss recovery.
• Reordering adaptation: If @rack_reordering_detected increments, RACK has dynamically expanded its reordering window to prevent premature retransmission spikes.
• Timeout analysis: Frequent firing of @rack_reo_timeouts indicates that packets are suffering severe delay spread across multi-path fabric links.

Configuring Host RACK Parameters in Linux

To verify and tune RACK operation on Linux servers:

1. Inspect TCP recovery sysctls via sysctl net.ipv4.tcp_recovery. The default bitmask value 1 enables RACK loss detection.
2. Ensure Selective Acknowledgment is active: sysctl net.ipv4.tcp_sack must equal 1, as RACK relies on SACK blocks to establish acknowledgment timelines.
3. Monitor system-wide RACK statistics via nstat -z | grep -i rack.