How to Trace TCP Socket TCP_INFO tcpi_snd_wnd in Linux Using eBPF

High-throughput data pipelines and low-latency API clusters frequently encounter throughput throttles that are not caused by packet drops or network bandwidth saturation, but by flow control constraints at the remote endpoint. In the TCP protocol, the receiver advertises an available buffer size via the TCP window header field, which the sender kernel tracks as the send window (snd_wnd). If a receiver application processes data too slowly, its receive buffer fills, causing it to advertise a shrinking or zero window. When this occurs, the sender’s transmission queue stalls, creating latency spikes and connection queuing.

Within the Linux networking subsystem, the current advertised peer receive window is maintained in struct tcp_sock and exposed through struct tcp_info as tcpi_snd_wnd. Tracing this value via extended Berkeley Packet Filter (eBPF) provides immediate visibility into receiver-side flow control bottlenecks in real time.

Understanding tcpi_snd_wnd in the Linux TCP Stack

Defined in include/uapi/linux/tcp.h, the struct tcp_info structure provides real-time window metrics:

struct tcp_info {
    /* ... state and RTT metrics ... */
    __u32   tcpi_snd_mss;
    __u32   tcpi_rcv_mss;
    __u32   tcpi_unacked;
    __u32   tcpi_sacked;
    __u32   tcpi_lost;
    __u32   tcpi_retrans;
    __u32   tcpi_fackets;
    /* Window telemetry */
    __u32   tcpi_snd_wnd;   /* Current peer advertised send window (bytes) */
    __u32   tcpi_rcv_wnd;   /* Current local receive window (bytes) */
};

The tcpi_snd_wnd field reflects the unscaled byte capacity the remote receiver can accept before its buffer overflows. When tcpi_snd_wnd drops below the maximum segment size (MSS), the Linux TCP stack enters a zero-window probing state (sending persist timers via tcp_send_probe0()), halting normal application throughput.

Authoring the bpftrace Send Window Probe

We can use bpftrace to attach to tcp_write_xmit() or tcp_ack() to monitor outgoing traffic and capture occurrences where snd_wnd collapses:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_ack
{
    $sk = (struct sock *)arg0;
    $inet = (struct inet_sock *)$sk;
    $tp = (struct tcp_sock *)$sk;

    $daddr = ntop($inet->inet_daddr);
    $saddr = ntop($inet->inet_saddr);
    $dport = bswap($inet->inet_dport);
    $snd_wnd = $tp->snd_wnd;

    // Trigger alert if remote window drops below 4KB (receiver buffer constraint)
    if ($snd_wnd < 4096 && $snd_wnd > 0) {
        time("%H:%M:%S ");
        printf("WARN: SndWnd Throttled | %s -> %s:%d | SndWnd: %d bytes | CWND: %d\n",
            $saddr, $daddr, $dport, $snd_wnd, $tp->snd_cwnd);
    }

    if ($snd_wnd == 0) {
        time("%H:%M:%S ");
        printf("CRITICAL: ZERO WINDOW | %s -> %s:%d | Transmission Stalled\n",
            $saddr, $daddr, $dport);
    }
}

Running the Trace and Diagnosing Bottlenecks

Run the script on the sending Linux host:

sudo bpftrace trace_snd_wnd.bt

During an active workload, bpftrace flags receiver buffer constraints directly:

17:05:12 WARN: SndWnd Throttled | 10.0.2.14 -> 10.0.5.88:9000 | SndWnd: 1460 bytes | CWND: 102
17:05:13 CRITICAL: ZERO WINDOW | 10.0.2.14 -> 10.0.5.88:9000 | Transmission Stalled
17:05:15 WARN: SndWnd Throttled | 10.0.2.14 -> 10.0.5.88:9000 | SndWnd: 2920 bytes | CWND: 102

This output demonstrates that the destination process on 10.0.5.88:9000 is unable to drain its TCP socket buffer quickly enough, forcing its kernel to advertise a zero window and freezing data flow.

Remediation Strategies

When eBPF telemetry identifies persistent send window bottlenecks:

1. Audit the receiving application’s processing architecture for blocking operations, garbage collection pauses, or thread starvation.
2. Tune the receiver’s socket receive buffer sizes in Linux via net.ipv4.tcp_rmem and ensure net.ipv4.tcp_window_scaling = 1 is enabled across all hosts.
3. Increase client-side backpressure handling to regulate transmission velocity before kernel socket buffers are exhausted.