How to Trace Directory Cache Lookups and Hit Rates in Linux Using dcachetop

Every Linux filesystem path lookup—whether opening configuration files, loading shared libraries, or reading database shards—begins with the Virtual File System (VFS) directory cache (dcache). The dcache stores directory entries (dentry structures) in high-speed RAM, resolving pathnames without executing costly physical disk seeks. When dcache hit rates plummet under heavy multi-tenant container churn or recursive file walks, system call latency skyrockets. The eBPF-powered dcachetop utility provides real-time top-style visibility into dcache lookup efficiency.

Understanding Linux VFS dcache Lookup Mechanics

Pathname lookup traverses directory hierarchies using fast kernel hash tables:

• dentry Resolution: Translates path strings (e.g., /var/log/nginx/access.log) component-by-component into inode numbers.
• Positive vs Negative Dentries: Positive dentries point to existing files; negative dentries record non-existent paths, preventing repeated disk lookups for missing files.
• Cache Miss Bottlenecks: On a dcache miss, the kernel must invoke underlying filesystem inode lookup routines (e.g., ext4_lookup() or xfs_lookup()), blocking the calling thread on physical storage I/O.

Installing the BCC Tracing Toolkit

Ensure the BPF Compiler Collection (BCC) is installed on your Linux distribution:

# Ubuntu / Debian
sudo apt update
sudo apt install -y bpfcc-tools linux-headers-$(uname -r)

# RHEL / Rocky Linux / Fedora
sudo dnf install -y bcc-tools kernel-devel-$(uname -r)

The executable is typically installed at /usr/sbin/dcachetop-bpfcc or /usr/share/bcc/tools/dcachetop.

Running dcachetop for Real-Time Path Tracing

Launch dcachetop with superuser privileges:

sudo dcachetop-bpfcc

The utility clears the terminal screen at regular intervals and outputs real-time dcache lookup metrics:

15:10:02 loadavg: 0.72 0.81 0.65 3/420 22019
PID    COMM             READS  HITS   MISSES  %HIT
14201  postgres         84200  84180      20  99.98%
18902  find             45100  12040   33060  26.70%
22014  nginx            28400  28390      10  99.96%
30105  git              15200   8400    6800  55.26%

The output columns indicate:

• PID & COMM: Process ID and executable name executing path resolution.
• READS: Total path component lookup attempts during the sample window.
• HITS: Successful lookups serviced entirely from RAM.
• MISSES: Uncached lookups that required underlying disk or inode table scans.
• %HIT: Percentage hit ratio (optimal production workloads should exceed 95%).

Diagnosing and Mitigating Low dcache Hit Rates

If utilities like find, build systems, or custom crawlers cause severe miss rates:

1. Check memory pressure and vfs cache pressure sysctl tuning:

   sysctl vm.vfs_cache_pressure

2. Lower vfs_cache_pressure from the default 100 to 50 to prioritize retaining directory and inode caches in RAM over page cache:

   sudo sysctl -w vm.vfs_cache_pressure=50

3. Persist the parameter in /etc/sysctl.d/99-vfs-tuning.conf to preserve high dcache performance across reboots.