How to Trace TCP Socket TCP_SAVE_SYN and Handshake Diagnostics in Linux Using eBPF

When troubleshooting client connectivity anomalies, subtle TCP option incompatibilities (such as broken Window Scaling or malformed SACK Permitted options), or TCP fingerprinting behavior, network engineers traditionally run packet capture tools like tcpdump. However, capturing full packet streams on high-throughput multi-gigabit interfaces consumes significant CPU and disk resources. The Linux kernel provides the TCP_SAVE_SYN socket option, allowing applications to preserve the client’s original SYN packet in socket memory. This tutorial covers tracing and inspecting saved SYN headers using eBPF.

The Mechanics of TCP_SAVE_SYN in the Linux Kernel

When an application configures setsockopt(listen_fd, IPPROTO_TCP, TCP_SAVE_SYN, &val, sizeof(val)) on a listening socket, the kernel instructs tcp_v4_syn_recv_sock() to clone the incoming SYN packet’s IP and TCP header into tp->saved_syn. Once accept() returns the child connection, the application can extract the original raw byte buffer via getsockopt(client_fd, IPPROTO_TCP, TCP_SAVED_SYN, buffer, &len). Tracing this allocation via eBPF kprobes reveals connection parameters before state transformations take place.

Prerequisites and Environment Setup

Install the required tracing tools on your Linux server:

sudo apt-get update
sudo apt-get install -y bpftrace linux-headers-$(uname -r)

Confirm that the system supports eBPF socket tracing:

sudo bpftrace -e 'kprobe:tcp_v4_syn_recv_sock { printf("SYN received\n"); exit(); }'

Authoring the bpftrace SYN Diagnostic Monitor

Save the following script as trace_savesyn.bt to inspect initial TCP window sizes, MSS advertisements, and TCP options directly from the SYN stage:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_v4_syn_recv_sock
{
    $sk = (struct sock *)arg0;
    $skb = (struct sk_buff *)arg1;

    $inet = (struct inet_sock *)arg0;
    $daddr = ntop($inet->sk.__sk_common.skc_daddr);
    $dport = bswap($inet->sk.__sk_common.skc_dport);
    $saddr = ntop($inet->sk.__sk_common.skc_rcv_saddr);
    $sport = bswap($inet->inet_sport);

    @syn_arrivals[$saddr, $sport] = count();
    printf("[SYN ARRIVAL] %s:%d -> %s:%d (Packet Length: %d bytes)\n",
        $saddr, $sport, $daddr, $dport, $skb->len);
}

kprobe:tcp_get_info
{
    $sk = (struct sock *)arg0;
    $tp = (struct tcp_sock *)arg0;

    if ($tp->saved_syn) {
        @saved_syn_read_count = count();
    }
}

END
{
    printf("\n--- Saved SYN Diagnostics Summary ---\n");
    print(@syn_arrivals);
    print(@saved_syn_read_count);
}

Running the Trace and Observing Header Telemetry

Execute the bpftrace script with root permissions:

sudo bpftrace trace_savesyn.bt

Simulate incoming client requests or test against edge load balancers. The output records raw handshake transitions:

[SYN ARRIVAL] 192.168.1.50:58102 -> 10.0.0.10:443 (Packet Length: 60 bytes)
[SYN ARRIVAL] 10.20.1.100:44120 -> 10.0.0.10:443 (Packet Length: 60 bytes)

Comparing packet lengths isolates whether clients are supplying full TCP options (60 bytes with timestamps, SACK, and window scale) or stripped headers (40–44 bytes) indicative of legacy stacks or middlebox proxy manipulation.

Application Architecture Recommendations

• Use TCP_SAVE_SYN in custom reverse proxies and security gateways to log client TCP fingerprints (such as JA3-like passive OS detection via TCP window size and option orders) without running background pcap processes.
• Ensure buffer memory is promptly freed by reading or closing sockets to prevent memory accumulation under high connection volumes.