How to Trace TCP Socket TCP_INFO tcpi_pmtu Path MTU Discovery in Linux Using eBPF

Modern cloud networks and hybrid enterprise infrastructure frequently route traffic through multiple overlay tunnels, including VXLAN, WireGuard, IPsec, and GRE encapsulations. Each encapsulation layer introduces additional protocol header overhead, reducing the maximum transmission unit (MTU) along the network path below the standard 1500-byte Ethernet payload. Path MTU Discovery (PMTUD) dynamically identifies the bottleneck MTU along a connection path by sending packets with the Don’t Fragment (DF) flag set and listening for ICMP “Fragmentation Needed” (Type 3, Code 4) notifications. However, when upstream firewalls or cloud security groups drop ICMP messages, connections suffer “PMTU black-holing”—large packets are silently dropped while small handshakes succeed.

Inside the Linux networking subsystem, the active path MTU for each TCP connection is tracked in struct tcp_info as tcpi_pmtu. Using extended Berkeley Packet Filter (eBPF) and bpftrace, network engineers can trace tcpi_pmtu in real time to diagnose MTU mismatches without running continuous packet captures.

Understanding tcpi_pmtu in the Linux TCP Stack

The Linux kernel defines tcpi_pmtu within struct tcp_info in include/uapi/linux/tcp.h:

struct tcp_info {
    /* ... state fields ... */
    __u32   tcpi_snd_mss;
    __u32   tcpi_rcv_mss;
    __u32   tcpi_pmtu;      /* Current Path MTU (bytes) */
    __u32   tcpi_rcv_ooopack;
    /* ... additional counters ... */
};

When the kernel receives an ICMP Destination Unreachable notification, it invokes tcp_v4_err() or tcp_mtu_probing() to update the cached path MTU. If ICMP is blocked, the connection stalls until Linux PMTU probing (tcp_mtu_probing) kicks in to step down the packet size systematically.

Authoring the bpftrace Path MTU Diagnostic Probe

The following bpftrace script monitors calls to tcp_sync_mss(), triggering alerts whenever a socket’s PMTU adjusts or falls below standard Ethernet thresholds:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_sync_mss
{
    $sk = (struct sock *)arg0;
    $inet = (struct inet_sock *)$sk;
    $tp = (struct tcp_sock *)$sk;

    $daddr = ntop($inet->inet_daddr);
    $saddr = ntop($inet->inet_saddr);
    $dport = bswap($inet->inet_dport);
    $pmtu = arg1; // New PMTU passed to sync function

    if ($pmtu > 0 && $pmtu < 1500) {
        time("%H:%M:%S ");
        printf("WARN: Sub-Ethernet PMTU Detected | %s -> %s:%d | PMTU: %d bytes | MSS: %d\n",
            $saddr, $daddr, $dport, $pmtu, $tp->mss_cache);
    }
}

Running the Trace and Diagnosing Overlay Bottlenecks

Launch the probe on your Linux host:

sudo bpftrace trace_pmtu.bt

When a tunnel constraint or ICMP notification occurs, bpftrace outputs structured telemetry:

20:04:15 WARN: Sub-Ethernet PMTU Detected | 10.0.3.12 -> 192.0.2.88:443 | PMTU: 1420 bytes | MSS: 1380
20:04:18 WARN: Sub-Ethernet PMTU Detected | 10.0.3.12 -> 203.0.113.50:8443 | PMTU: 1280 bytes | MSS: 1240

The first log entry indicates a WireGuard tunnel restriction (1420 bytes), while the second entry reveals a connection traversing an IPv6 minimum MTU constraint (1280 bytes).

Kernel Tuning for MTU Blackhole Resilience

To ensure automated recovery when ICMP notifications are dropped by middleboxes:

1. Enable kernel MTU probing in sysctl:

   sudo sysctl -w net.ipv4.tcp_mtu_probing=1

   Setting this to 1 activates probing when black-holing is suspected, while 2 enforces MTU probing continuously.
2. Configure minimum MSS probing bounds:

   sudo sysctl -w net.ipv4.tcp_base_mss=1024

3. On border routers and edge firewalls, configure TCP MSS clamping to negotiate appropriate segment sizes automatically during TCP handshakes.