Modern Linux servers, container runtimes, and local microservice architectures rely heavily on Unix domain sockets (AF_UNIX) for high-performance inter-process communication (IPC). Unlike TCP/IP network sockets that route packets through virtual network interfaces and loopback network layers, Unix domain sockets communicate directly through Linux kernel memory buffers and virtual filesystem paths. System daemons such as systemd, Docker, PostgreSQL, Redis, and desktop display servers like Wayland communicate with client processes through these endpoints.
When IPC bottlenecks, authorization failures, or microservice connection timeouts occur, standard network packet analysers like tcpdump cannot inspect Unix domain socket traffic. The BPF Compiler Collection (BCC) provides uconnsnoop, an eBPF tracing utility designed specifically to track Unix domain socket connections, connecting processes, and connection results in real time. This guide shows how to trace Unix domain socket connections in Linux using uconnsnoop.
Prerequisites and BCC Installation
The uconnsnoop utility uses extended Berkeley Packet Filters (eBPF) to hook into the kernel’s unix_stream_connect function. It requires Linux kernel 4.9 or later compiled with eBPF support and elevated administrative permissions.
Installing BCC Tools on Ubuntu and Debian
- Open your terminal and refresh your package repositories:
sudo apt update
- Install the core BCC tracing package and Linux kernel headers:
sudo apt install bpfcc-tools linux-headers-$(uname -r)
On Debian and Ubuntu systems, BCC utilities are installed into /usr/sbin/ with a -bpfcc suffix. The executable is accessible as uconnsnoop-bpfcc or symlinked to uconnsnoop.
Installing on RHEL, Fedora, and Rocky Linux
Install the BCC package using dnf:
sudo dnf install bcc-tools kernel-devel-$(uname -r)
Utilities on RPM distributions are stored in /usr/share/bcc/tools/uconnsnoop.
Basic Tracing with uconnsnoop
To begin tracing Unix domain socket connections system-wide, launch uconnsnoop with root privileges:
sudo uconnsnoop-bpfcc
When applications initiate IPC connections, uconnsnoop prints an event line with formatted columns:
TIME(s) PID COMM RET PATH
0.000000 1428 docker 0 /var/run/docker.sock
0.012480 2045 systemd-journal 0 /run/systemd/journal/stdout
0.104210 3812 psql 0 /var/run/postgresql/.s.PGSQL.5432
1.205411 4102 node -2 /var/run/app/backend.sock
Interpreting the Output Columns
- TIME(s): Elapsed seconds since tracing commenced, or a wall-clock timestamp depending on active flags.
- PID: Process ID of the client application requesting the socket connection.
- COMM: Executable command or process name initiating the request.
- RET: The return status code of the connection syscall. A value of
0indicates success. Negative integers reflect standard Linux errno codes (for example,-2indicatesENOENT: No such file or directory;-111indicatesECONNREFUSED: Connection refused). - PATH: The target filesystem pathname or abstract namespace identifier where the listening socket resides.
Advanced Filtering and Troubleshooting Options
System-wide IPC monitoring on production servers can generate extensive output. Use command-line arguments to isolate specific applications and connection issues.
Tracing Failures and Rejected Connections Only
When debugging application timeouts or permissions errors, filter output to show only failed connections using -F or --failed:
sudo uconnsnoop-bpfcc -F
This filters out normal traffic and highlights missing socket paths (ENOENT) or uninitialised listening daemons (ECONNREFUSED) instantly.
Filtering by Process ID
To inspect a single suspicious service or daemon, restrict tracing to its specific Process ID using the -p parameter:
sudo uconnsnoop-bpfcc -p 3812
Adding Real-Time Wall-Clock Timestamps
To correlate IPC socket requests with entries in application logs, append the -T flag to output ISO 8601 wall-clock timestamps:
sudo uconnsnoop-bpfcc -T
TIME(s) PID COMM RET PATH
14:02:15.102341 5120 curl 0 /run/containerd/containerd.sock
Tracing Abstract Namespace Sockets
Linux supports abstract namespace sockets which do not appear as physical files on disk and start with a null byte (\0). In uconnsnoop, abstract sockets are displayed with an @ prefix, such as @/tmp/dbus-X11-session.
If an application attempts to bind or connect to an abstract socket that no longer has an active server process listening, uconnsnoop reports an immediate -111 (ECONNREFUSED), allowing you to pinpoint socket mismatches without inspecting virtual filesystem directories manually.
Diagnostic Considerations and Overhead
- Minimal Kernel Overhead:
uconnsnoopattaches to kernel tracepoints and kprobes using eBPF, running lightweight bytecode in-kernel. It introduces negligible overhead compared tostrace, making it safe for production diagnostics. - Stream Sockets vs Datagram Sockets:
uconnsnooptraces stream-oriented connections (SOCK_STREAM). Unix domain datagram communications (SOCK_DGRAM) do not invoke connection handshakes and will not appear in stream connection traces. - Container Namespaces: When tracing across container runtimes, processes running inside containers display their host PID unless translated via container namespace tools.