How to Trace TCP Socket TCP_AO (Authentication Option) in Linux Using eBPF

Enterprise routing backbones and cloud edge border routers historically secured BGP sessions using TCP MD5 signatures (RFC 2385). However, MD5 is vulnerable to collision attacks and lacks automated key rotation mechanisms. The Linux kernel introduced native support for the TCP Authentication Option (TCP-AO / RFC 5925), providing cryptographically secure SHA-1 and HMAC-SHA-256 message authentication codes with dynamic Master Key Tuple (MKT) rotation. Tracing TCP-AO handshake validation and signature mismatches with eBPF gives security engineers deep visibility into kernel authentication paths.

Architecture of TCP-AO in the Linux Network Stack

TCP-AO integrates cryptographic verification directly into the TCP header parsing loop. For every incoming segment carrying option kind 29 (TCP-AO), the kernel inspects the KeyID and Current Mac Key (CMK) fields. It looks up the matching MKT stored in the socket data structure and computes the Message Authentication Code (MAC) over the pseudo-header and TCP payload.

If key rotation fails, timestamps drift, or keys mismatch between BGP peers, the kernel silently drops packets. Traditional userland packet captures only reveal dropped connections, leaving administrators uncertain whether the root cause is a cryptographic mismatch, sequence rollover, or invalid key ID.

Authoring a Custom bpftrace TCP-AO Telemetry Script

To inspect TCP-AO authentication events and capture signature verification failures, write a bpftrace instrumentation script targeting kernel TCP-AO functions:

#!/usr/bin/env bpftrace

#include <net/sock.h>
#include <net/tcp_ao.h>
#include <linux/tcp.h>

kprobe:tcp_ao_verify_hash
{
  $sk = (struct sock *)arg0;
  $skb = (struct sk_buff *)arg1;
  @ao_verifications = count();
}

kretprobe:tcp_ao_verify_hash
{
  if (retval != 0) {
    @ao_auth_failures[retval] = count();
    time("%H:%M:%S: TCP-AO verification failed with code %d\n", retval);
  } else {
    @ao_auth_successes = count();
  }
}

interval:s:10
{
  print(@ao_verifications);
  print(@ao_auth_successes);
  print(@ao_auth_failures);
  clear(@ao_verifications);
  clear(@ao_auth_successes);
  clear(@ao_auth_failures);
}

Save this script as trace_tcp_ao.bt. It instruments tcp_ao_verify_hash(), capturing success ratios and logging cryptographic verification failures with non-zero return codes.

Executing and Evaluating Kernel Authentication Traces

Execute the bpftrace monitor with root privileges:

sudo bpftrace trace_tcp_ao.bt

When BGP or routing peering sessions establish connections, observe the output telemetry:

• Authentication successes: A high and continuous count in @ao_auth_successes confirms that incoming segments match established MKT keys and valid algorithm hashes.
• Authentication failure codes: Non-zero return values (such as -1 or -2) pinpoint invalid KeyIDs, mismatched shared secrets, or misconfigured salt values between peer routers.
• Segment drop rates: Cross-reference verification drops against nstat -z | grep -i ao to confirm that cryptographic discards correlate directly with SNMP MIB counters.

Configuring TCP-AO Sockets via iproute2

To establish TCP-AO keys on Linux routing endpoints, manage Master Key Tuples using the modern iproute2 suite:

1. Verify kernel support: Ensure your Linux kernel is version 6.5 or later with CONFIG_TCP_AO=y enabled.
2. Add a Master Key Tuple for a peer endpoint:

   ip tcp_ao add 192.0.2.1 alg hmac-sha-256-128 key "SecretPresharedKey987!" sndid 1 rcvid 1

3. Verify active keys using ip tcp_ao show to inspect SendID, ReceiveID, and cryptographic algorithm parameters before initiating routing daemon sessions.