How to Use ss in Linux to Inspect Socket States, TCP Connection Queues, and Network Memory Buffers

Architectural Foundation of the Linux Socket Layer and the sock_diag Subsystem

Diagnosing network connectivity failures, thread starvation, and connection backlog drops on high-throughput Linux servers requires real-time socket observability. Historically, system administrators relied on the legacy netstat utility. However, netstat operates by parsing flat text files inside /proc/net/tcp, an obsolete mechanism that acquires global kernel spinlocks and introduces severe CPU stall latency on servers maintaining tens of thousands of concurrent connections.

The modern Linux replacement is ss (Socket Statistics), maintained within the authoritative iproute2 toolchain. Rather than scanning pseudo-files in /proc, ss interfaces directly with the kernel’s sock_diag Netlink subsystem (NETLINK_INET_DIAG). This Netlink socket interface streams binary socket metadata structures directly out of kernel space, allowing ss to evaluate socket states, inspect internal TCP connection buffers, and extract microscopic TCP_INFO telemetry in microseconds without degrading active network traffic.

Inspection Tool	Kernel Communication Interface	Performance Under 50,000+ Sockets	TCP Internal Telemetry (RTT / cwnd)	Kernel Locking Behaviour
Legacy netstat	Sequential /proc/net/tcp reads	Severe CPU spikes; process blocks	Unavailable	Acquires global kernel spinlocks
Modern ss	Binary sock_diag Netlink socket	Instantaneous binary streaming	Full access to tcp_info kernel struct	Lockless atomic memory reads

Decoding Recv-Q and Send-Q Across Socket States

One of the most frequently misunderstood aspects of Linux socket diagnostics is the operational difference between the Recv-Q and Send-Q columns, whose technical definitions invert depending on whether a socket is in the LISTEN state or the ESTABLISHED state.

Socket State	Recv-Q Column Definition	Send-Q Column Definition	Diagnostic Significance
LISTEN	Current unaccepted connection backlog	Maximum listen queue capacity	If Recv-Q > Send-Q, listen queue has overflowed; incoming TCP connections are dropped
ESTABLISHED	Bytes received in kernel buffer, unread by user app	Bytes sent by app, unacknowledged by remote peer	Elevated Recv-Q indicates application thread stall; elevated Send-Q indicates network congestion

Essential Execution Syntax and State Filtering

To inspect active sockets efficiently, combine protocol toggles with state filters and process resolvers:

# 1. Inspect all listening TCP sockets with process names and numerical ports
sudo ss -tlpn

# 2. Inspect all established HTTPS and HTTP connections
ss -tan '( sport = :443 or sport = :80 )'

# 3. Filter sockets by specific TCP lifecycle states
ss -tan state time-wait
ss -tan state close-wait
ss -tan state syn-recv

Example listening socket telemetry:

State      Recv-Q Send-Q Local Address:Port  Peer Address:Port Process
LISTEN     0      511    0.0.0.0:80          0.0.0.0:*     users:(("nginx",pid=1240,fd=6))
LISTEN     128    128    0.0.0.0:3306        0.0.0.0:*     users:(("mysqld",pid=1890,fd=21))

Notice the MySQL line on port 3306: Recv-Q has reached 128, matching the Send-Q limit of 128. The MySQL accept queue is completely saturated; incoming database connection handshakes are actively dropping.

Extracting Granular TCP_INFO Micro-Metrics

Adding the -i (internal) flag prompts ss to dump the kernel’s internal tcp_info diagnostic structure for individual TCP connections:

# Inspect deep TCP internal metrics for outbound connections to a database
ss -ti dst 192.168.1.50

Example telemetry output breakdown:

ESTAB      0      0      192.168.1.10:48210   192.168.1.50:5432
     cubic wscale:7,7 rto:204 rtt:0.32/0.04 ato:40 mss:1448 rcvspace:14600 ssthresh:10 cwnd:10 pacing_rate 72.4Mbps delivery_rate 180Mbps retrans:0/1

Key metrics decoded:

• rtt:0.32/0.04: Smoothed round-trip time is 0.32 milliseconds with 0.04 ms mean deviation.
• cwnd:10: The congestion window size in packets (MSS).
• retrans:0/1: Reports retransmitted segments, identifying packet loss on the link.
• cubic: Active TCP congestion control algorithm.

Inspecting Socket Memory Buffers with ss -m

To diagnose memory pressure and socket buffer exhaustion, query memory allocations via -m:

# Audit socket memory allocations across active connections
ss -tem dst :443

Telemetry fields explained:

• rmem_alloc: Bytes allocated for the receive buffer.
• wmem_alloc: Bytes committed to transmit queues.
• fwd_alloc: Pre-allocated memory slice held by the socket.
• skmem:(r0,rb131072,t0,tb16384,f4096,w0,o0,bl0,d0): Direct memory limits enforcing TCP receive/transmit buffer ceilings.

Troubleshooting Critical Linux Socket Faults

Observed Socket Anomaly	Underlying Root Cause	Technical Verification Method	Remediation Protocol
LISTEN Queue Full (Recv-Q equals Send-Q)	Application worker threads cannot call accept() fast enough, or backlog limit too low	Check ss -tlpn; Recv-Q matches Send-Q; netstat -s | grep overflowed increments	Increase application listen backlog and tune kernel limits: sysctl -w net.core.somaxconn=4096
Massive Accumulation of CLOSE_WAIT Sockets	Remote peer closed TCP connection (sent FIN), but local application failed to call close()	Run ss -tan state close-wait; counts continue climbing indefinitely	Application bug; inspect code for unhandled exception or file descriptor leak in socket handler
Port Exhaustion in TIME_WAIT Sockets	High rate of short-lived TCP connections consuming ephemeral port range	ss -tan state time-wait | wc -l exceeds 30,000; outbound connections fail	Enable HTTP/2 keep-alive connection pooling; enable net.ipv4.tcp_tw_reuse = 1 in sysctl
High Send-Q on Established Web Sockets	TCP window stall; receiving client has advertised a zero-window, or network path congested	Check ss -ti; inspect rcv_space and cwnd on stalled socket	Check client network latency; ensure kernel TCP buffer auto-tuning (net.ipv4.tcp_wmem) is enabled