How to Trace Directory Cache Lookups and Latency in Linux Using dcsnoop

The Linux Directory Entry Cache (dcache) accelerates file path lookups by keeping resolved directory entries in memory. While aggregate tools like dccount report overall cache hit and miss tallies, they do not reveal individual file paths, requesting processes, or lookup latencies. When diagnosing storage stalls, rogue crawling scripts, or compilation bottlenecks, system engineers need per-operation path visibility. The dcsnoop utility, part of the BPF Compiler Collection (BCC), dynamically traces directory cache lookup queries in real time using extended Berkeley Packet Filters (eBPF).

How dcsnoop Observes Kernel Path Lookups

Whenever a process attempts to access a file (e.g. via open, stat, or execve), the Linux kernel evaluates each directory component along the path. The dcsnoop tool attaches to internal kernel dcache lookup routines (such as lookup_fast) to record:

  • Timestamp and process identification (PID and command name).
  • The exact filename or directory component being evaluated.
  • Lookup latency in microseconds.
  • Whether the lookup resulted in a cache reference (hit) or failed to find the entry.

Installing the BCC Tools Suite

To run dcsnoop, install the BCC package and kernel development headers corresponding to your active kernel version.

On Debian, Ubuntu, and Linux Mint:

sudo apt update
sudo apt install bpfcc-tools linux-headers-$(uname -r)

On RHEL, Rocky Linux, AlmaLinux, or Fedora:

sudo dnf install bcc-tools kernel-devel-$(uname -r)

The executable is typically installed at /usr/sbin/dcsnoop-bpfcc or /usr/share/bcc/tools/dcsnoop.

Tracing Directory Lookups in Real Time

To stream directory cache lookups system-wide, execute dcsnoop with root privileges:

sudo dcsnoop-bpfcc

Sample output from an active server:

TIME(s)      PID    COMM             T FILE
0.000000     1042   nginx            R index.html
0.000120     1042   nginx            R static
0.002410     3841   git              R HEAD
0.002590     3841   git              M nonexistent.lock

The output columns detail:

  • TIME(s): Elapsed time since tool invocation.
  • PID: Process ID executing the lookup.
  • COMM: Process command name.
  • T (Type): R indicates a successful reference (cache hit); M indicates a cache miss requiring deeper VFS storage inspection.
  • FILE: Name of the dentry component queried.

Filtering by Process ID

On production servers handling high concurrent traffic, tracing every lookup can quickly flood your terminal. To focus specifically on a single daemon (e.g., an Nginx worker or build process):

sudo dcsnoop-bpfcc -p 1042

Tracing Failed Lookups Only

Repeated failed lookups (cache misses) often highlight configuration bugs, such as web servers probing for non-existent .htaccess or fallback configuration files on every HTTP request. Use the -f flag to isolate failed queries:

sudo dcsnoop-bpfcc -f

Identifying these failed lookups allows developers to eliminate unnecessary file existence checks in application middleware, drastically reducing kernel overhead.

Measuring Lookup Latencies

To inspect latency durations for individual lookup queries:

sudo dcsnoop-bpfcc -d

This appends a latency column (in microseconds), making it straightforward to detect slow path evaluations caused by hash collisions, severe lock contention, or underlying disk thrashing.

Correlating Lookups with Dentry Cache Size

If dcsnoop displays high miss ratios across established applications, check available memory allocated to directory caches using slabtop:

sudo slabtop -s c | grep dentry

By pairing dcsnoop with slab metrics, Linux system administrators can identify wasteful file access patterns, fine-tune kernel cache pressure, and optimise file serving performance.

Get the best tech tips delivered straight to your inbox.

Join thousands of readers mastering Apple, Google, Microsoft, and Linux.