How to Trace TCP Socket Fast Open (TFO) Cookie Handshakes in Linux Using eBPF

TCP Fast Open (TFO, RFC 7413) eliminates an entire round-trip time (RTT) during connection establishment by allowing data payloads to be transmitted directly inside the initial SYN packet. To protect servers against SYN-flood reflection attacks, the server issues an encrypted cryptographic cookie during the first handshake, which the client presents on subsequent connections. When firewalls or middleboxes mangle SYN payloads or drop TFO packets, connections fall back to standard three-way handshakes. Tracing TFO with eBPF illuminates real-time cookie validation and handshake acceleration. This tutorial covers tracing TFO on Linux.

Linux Kernel TFO Architecture and Flags

The Linux TCP stack controls TCP Fast Open via the sysctl setting net.ipv4.tcp_fastopen:

• 0x1 (1): Enable client TFO.
• 0x2 (2): Enable server TFO.
• 0x3 (3): Enable both client and server TFO.
• 0x200 (512): Enable server TFO without requiring client cookies (for trusted internal networks).

During a TFO exchange, the server processes tcp_fastopen_cookie_gen() to issue cookies and tcp_fastopen_check() to authenticate client cookies. By hooking these kernel routines with eBPF kprobes, engineers can audit connection acceleration in real time.

Prerequisites and Environment Verification

Confirm that your Linux kernel has TFO enabled:

sysctl net.ipv4.tcp_fastopen
# Enable full client and server support if not active:
sudo sysctl -w net.ipv4.tcp_fastopen=3

Ensure bpftrace and kernel development headers are installed:

sudo apt-get install -y bpftrace linux-headers-$(uname -r)

Authoring the bpftrace TFO Monitor

Create a script named trace_tfo.bt to capture cookie operations and payload validation:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_fastopen_cookie_gen
{
    $sk = (struct sock *)arg0;
    $inet = (struct inet_sock *)arg0;
    $daddr = ntop($inet->sk.__sk_common.skc_daddr);
    $dport = bswap($inet->sk.__sk_common.skc_dport);

    printf("[TFO COOKIE GEN] Server generated new cookie for client %s:%d\n", $daddr, $dport);
    @cookie_gens = count();
}

kprobe:tcp_fastopen_check
{
    $sk = (struct sock *)arg0;
    $inet = (struct inet_sock *)arg0;
    $saddr = ntop($inet->sk.__sk_common.skc_rcv_saddr);
    $sport = bswap($inet->inet_sport);

    @tfo_checks = count();
}

kretprobe:tcp_fastopen_check
{
    $ret = retval;
    if ($ret == 0) {
        printf("[TFO SUCCESS] Valid cookie accepted! SYN data acknowledged immediately.\n");
        @tfo_accepted = count();
    } else {
        printf("[TFO REJECTED/MISS] Cookie invalid or missing (code %d). Falling back to 3WHS.\n", $ret);
        @tfo_fallback = count();
    }
}

END
{
    printf("\n--- TCP Fast Open Telemetry Summary ---\n");
    print(@cookie_gens);
    print(@tfo_checks);
    print(@tfo_accepted);
    print(@tfo_fallback);
}

Running the Trace and Observing Fast Handshakes

Execute the bpftrace monitor:

sudo bpftrace trace_tfo.bt

When clients equipped with TFO connect (such as curl with --tcp-fastopen or modern browsers), output captures handshake telemetry:

[TFO COOKIE GEN] Server generated new cookie for client 192.168.1.55:51240
[TFO SUCCESS] Valid cookie accepted! SYN data acknowledged immediately.
[TFO REJECTED/MISS] Cookie invalid or missing (code -1). Falling back to 3WHS.

Auditing Network Firewalls and Middlebox Blackholes

If @tfo_fallback heavily outpaces @tfo_accepted on networks where clients support TFO, middlebox firewalls may be stripping unknown TCP options. Inspecting dropped SYN-data transitions with eBPF isolates whether middleboxes are corrupting packets or dropping connections, allowing network teams to calibrate edge security appliances accordingly.