How to Trace TCP Socket TCP_INFO tcpi_lost Drop Bursts in Linux Using eBPF

In high-speed production networks, distinguishing between transient out-of-order delivery and genuine packet drops is essential for isolating infrastructure bottlenecks. The Linux kernel struct tcp_info records tcpi_lost, representing the number of unacknowledged segments currently judged lost by the transport stack (via SACK scoreboard analysis or duplicate ACKs). Tracing this value with eBPF gives real-time visibility into instantaneous packet drop bursts.

Understanding SACK Scoreboard Loss Markings

Modern Linux TCP stacks do not immediately assume packet loss upon receiving duplicate ACKs; instead, the SACK scoreboard tracks holes in sequence ranges. When the gap exceeds the reordering threshold or when RACK (Recent ACKs) timers expire, the kernel marks segments as lost in tcpi_lost. Tracing this metric exposes packet drops before retransmission queues complete recovery.

eBPF BCC Program for Lost Packet Tracing

The following BCC Python script attaches to tcp_mark_lost to log instantaneous packet loss events:

from bcc import BPF

bpf_source = """
#include <net/sock.h>
#include <linux/tcp.h>

struct lost_event_t {
    u32 pid;
    u16 sport;
    u16 dport;
    u32 lost_segs;
};

BPF_PERF_OUTPUT(lost_events);

int trace_lost(struct pt_regs *ctx, struct sock *sk) {
    if (sk->sk_family != AF_INET)
        return 0;

    struct tcp_sock *tp = tcp_sk(sk);
    struct lost_event_t evt = {};

    evt.pid = bpf_get_current_pid_tgid() >> 32;
    evt.sport = sk->sk_num;
    evt.dport = bpf_ntohs(sk->sk_dport);
    evt.lost_segs = tp->lost_out;

    lost_events.perf_submit(ctx, &evt, sizeof(evt));
    return 0;
}
"""

b = BPF(text=bpf_source)
b.attach_kprobe(event="tcp_mark_lost_retrans", fn_name="trace_lost")

def print_lost(cpu, data, size):
    e = b["lost_events"].event(data)
    print(f"LOSS EVENT: Port {e.sport} -> {e.dport} | Lost Segments: {e.lost_segs}")

b["lost_events"].open_perf_buffer(print_lost)
print("Listening for TCP lost segment events... Press Ctrl+C to terminate.")
while True:
    b.perf_buffer_poll()

Operational Diagnosis and Network Tuning

Run the script under root privileges:

sudo python3 trace_lost.py

Correlate sudden spikes in tcpi_lost with switch port egress drop statistics via SNMP or streaming telemetry to identify micro-congested switch queues.