How to Trace TCP Socket TCP_INFO tcpi_retrans_total Retransmission Bursts in Linux Using eBPF

Diagnosing whether network latency spikes are caused by occasional micro-burst buffer drops or persistent transport congestion requires capturing instantaneous rates of retransmission accumulation. The Linux kernel struct tcp_info records tcpi_retrans_total, counting lifetime retransmitted segments on a socket. Tracing the first derivative (rate of change) of this counter with eBPF exposes sudden packet drop bursts in real time.

Rate of Change vs Cumulative Counters

While static retransmission totals tell engineers that packet loss occurred at some point during a connection, calculating the delta over short time windows (e.g. 100ms) pinpoints the exact millisecond when queue overflow occurred on top-of-rack switches.

eBPF BCC Program for Retransmission Burst Detection

The following BCC Python script attaches to tcp_retransmit_skb to calculate rolling retransmission frequency per socket:

from bcc import BPF
import time

bpf_source = """
#include <net/sock.h>
#include <linux/tcp.h>

struct burst_event_t {
    u32 pid;
    u16 sport;
    u16 dport;
    u32 total_retrans;
};

BPF_PERF_OUTPUT(burst_events);

int trace_burst(struct pt_regs *ctx, struct sock *sk) {
    if (sk->sk_family != AF_INET)
        return 0;

    struct tcp_sock *tp = tcp_sk(sk);
    struct burst_event_t evt = {};

    evt.pid = bpf_get_current_pid_tgid() >> 32;
    evt.sport = sk->sk_num;
    evt.dport = bpf_ntohs(sk->sk_dport);
    evt.total_retrans = tp->total_retrans;

    burst_events.perf_submit(ctx, &evt, sizeof(evt));
    return 0;
}
"""

b = BPF(text=bpf_source)
b.attach_kprobe(event="tcp_retransmit_skb", fn_name="trace_burst")

last_counts = {}

def print_burst(cpu, data, size):
    e = b["burst_events"].event(data)
    key = (e.sport, e.dport)
    prev = last_counts.get(key, 0)
    delta = e.total_retrans - prev
    last_counts[key] = e.total_retrans
    if delta > 3:
        print(f"BURST DETECTED: Port {e.sport} -> {e.dport} dropped {delta} packets in burst!")

b["burst_events"].open_perf_buffer(print_burst)
print("Listening for TCP retransmission bursts... Press Ctrl+C to terminate.")
while True:
    b.perf_buffer_poll()

Operational Diagnosis and Network Tuning

Execute the monitoring script under superuser privileges:

sudo python3 trace_retrans_burst.py

Correlating burst occurrences with traffic volume identifies micro-burst events caused by concurrent multi-flow fan-in on leaf switches.