How to Trace TCP SYN Flood Protection Stalls in Linux Using eBPF

During volumetric network attacks or severe ingress traffic surges, malicious or misconfigured clients overwhelm server listening sockets with incomplete connection requests. When the kernel’s SYN backlog queue (tcp_max_syn_backlog) fills, the system either drops new connection attempts or activates TCP SYN cookies (tcp_syncookies). While SYN cookies maintain service availability without consuming state memory, fallback processing disables TCP options like window scaling and timestamps unless explicitly preserved. Tracing SYN flood events with eBPF exposes backlog saturation and validates mitigation thresholds in production.

Understanding Linux SYN Queue and SYN Cookie Architecture

Connection establishment traverses two distinct kernel queuing structures before application delivery:

• SYN Backlog (Incomplete Queue): Holds embryonic connections that have sent a SYN packet and are awaiting final ACK completion. Size is bounded by net.ipv4.tcp_max_syn_backlog.
• Accept Queue (Complete Queue): Holds fully established connections awaiting user-space retrieval via accept(). Bounded by the application’s listen backlog parameter.
• SYN Cookies (RFC 4987): When the SYN backlog overflows, the kernel calculates a cryptographic hash over source IP, destination IP, ports, and time, encoding the result directly into the initial sequence number (ISN). If a valid ACK returns, the kernel reconstructs the socket without state storage.

Auditing Kernel SYN Protection Settings

Inspect existing kernel sysctl parameters governing flood protection:

# Check SYN cookie activation status (1 = enabled when queue full)
sysctl net.ipv4.tcp_syncookies

# Check current SYN backlog capacity
sysctl net.ipv4.tcp_max_syn_backlog

# Inspect system-wide SYN cookie counters
netstat -s | grep -i "syn cookies"

Review the counter TCPSynCookiesSent and TCPSynCookiesFailed to evaluate past flood occurrences.

Tracing SYN Cookie Generation with bpftrace

When the kernel generates a SYN cookie, it invokes cookie_v4_init_sequence (or cookie_v6_init_sequence). Hook this function to log flood events in real time:

sudo bpftrace -e '
#include <net/sock.h>
#include <linux/tcp.h>
#include <linux/ip.h>

kprobe:cookie_v4_init_sequence
{
    $iph = (struct iphdr *)((struct sk_buff *)arg0)->head + ((struct sk_buff *)arg0)->network_header;
    $saddr = ntop($iph->saddr);
    $daddr = ntop($iph->daddr);

    printf("SYN Cookie Triggered: Attacking/Client IP: %-15s -> Target: %-15s\n",
           $saddr, $daddr);
    @syn_flood_events = count();
}'

Running this probe confirms whether incoming bursts are genuinely activating cryptographic fallback mechanisms.

Tracing SYN Drop and Listen Queue Overflow Rates

If SYN cookies are disabled or accept queues are exhausted, incoming SYN packets are dropped immediately. Trace packet drops inside tcp_v4_syn_recv_sock:

sudo bpftrace -e '
#include <net/sock.h>
#include <linux/tcp.h>

kprobe:tcp_v4_syn_recv_sock
{
    $sk = (struct sock *)arg0;
    if ($sk->sk_ack_backlog > $sk->sk_max_ack_backlog) {
        printf("Listen Queue Overflow Drop: Port %d (Current: %d, Max: %d)\n",
               ntohs($sk->sk_num), $sk->sk_ack_backlog, $sk->sk_max_ack_backlog);
        @listen_drops[ntohs($sk->sk_num)] = count();
    }
}

END
{
    printf("\nTotal Drops per Listening Port:\n");
    print(@listen_drops);
}'

Tuning Network Parameters to Mitigate Stalls

Apply kernel adjustments to harden network stacks against flood exhaustion:

• Expand SYN Backlog: Increase capacity for high-concurrency frontends:

  sudo sysctl -w net.ipv4.tcp_max_syn_backlog=65536

• Preserve TCP Options with Timestamps: Ensure net.ipv4.tcp_timestamps=1 is enabled. SYN cookies use timestamp bits to encode window scaling and SACK support.
• Enable TCP SYN Retries Limit: Reduce SYN-ACK retry counts to release abandoned half-open connections faster:

  sudo sysctl -w net.ipv4.tcp_synack_retries=2