How to Trace TCP Socket TCP_INFO tcpi_retrans_synack TCP PMTU Black Hole Discovery Drops in Linux Using eBPF

Path MTU Discovery (PMTUD, RFC 1191) allows TCP endpoints to dynamically determine the maximum transmission unit along a network path without IP fragmentation. However, when upstream routers or firewalls drop ICMP “Fragmentation Needed” (Type 3, Code 4) messages, a PMTU black hole occurs: large packets are silently discarded while small handshake packets succeed. When large SYN-ACK payloads containing extensive TCP options or Fast Open data exceed link MTUs, connections stall, escalating tcpi_retrans_synack counters. Extended Berkeley Packet Filters (eBPF) enable non-intrusive kernel tracing to isolate PMTU black hole drops in real time.

Kernel Mechanics of PMTU Probing and Black Hole Detection

Inside the Linux network stack, the kernel manages route MTUs in the fib info structure. When ICMP unreachable messages are blocked, the kernel cannot automatically decrement the route MSS via ipv4_update_pmtu().

To recover from black holes, Linux implements black hole probing (RFC 4821) inside tcp_mtu_probe(). If black hole probing is disabled or encounters repeated losses during initial handshakes, the socket repeatedly retransmits SYN-ACK segments. Using eBPF attached to kernel PMTU discovery and drop tracepoints gives infrastructure engineers exact per-socket visibility into black hole bottlenecks.

Authoring the Diagnostic Probe with bpftrace

Construct a dedicated bpftrace script that intercepts TCP MTU probing and correlates PMTU drop events with SYN-ACK retransmits:

#!/usr/bin/env bpftrace
/* trace_pmtu_blackhole.bt - Trace Linux TCP PMTU black hole drops */

#include <net/sock.h>
#include <linux/tcp.h>

tracepoint:tcp:tcp_retransmit_synack
{
    $sk = (struct sock *)args->skaddr;
    $saddr = ntop(args->saddr);
    $daddr = ntop(args->daddr);
    $sport = args->sport;
    $dport = args->dport;

    printf("SYN-ACK Retransmit: %s:%d -> %s:%d (PMTU Black Hole Candidate)
",
           $saddr, $sport, $daddr, $dport);
    @synack_retransmits[$saddr, $daddr] = count();
}

kprobe:tcp_mtu_probing
{
    $sk = (struct sock *)arg0;
    $saddr = ntop($sk->__sk_common.skc_rcv_saddr);
    $daddr = ntop($sk->__sk_common.skc_daddr);

    printf("MTU Probe Initiated: %s <-> %s
", $saddr, $daddr);
    @mtu_probes[$saddr, $daddr] = count();
}

interval:s:5
{
    time("%H:%M:%S ");
    printf("--- TCP PMTU Activity (5s Window) ---
");
    print(@mtu_probes);
    print(@synack_retransmits);
    clear(@mtu_probes);
    clear(@synack_retransmits);
}

Running the Trace on Production Linux Systems

Deploy the probe with root privileges on the target server:

# Install bpftrace and kernel dependencies
sudo apt-get update && sudo apt-get install -y bpftrace linux-headers-$(uname -r)

# Execute the trace
sudo bpftrace trace_pmtu_blackhole.bt

Kernel Optimization and Parameter Tuning

To enable automated RFC 4821 Path MTU Discovery black hole probing:

# Enable TCP MTU probing (1 = enable when black hole detected, 2 = always enable)
sudo sysctl -w net.ipv4.tcp_mtu_probing=1

# Persist configuration
echo "net.ipv4.tcp_mtu_probing = 1" | sudo tee /etc/sysctl.d/99-tcp-pmtu.conf
sudo sysctl --system

By pairing eBPF kernel instrumentation with calibrated sysctl adjustments, network engineers can eliminate elusive SYN-ACK stalls while preserving optimal MTU utilization.