How to Use slabtop in Linux to Diagnose Kernel Slab Memory Allocation and Dentry Cache Leaks

When diagnosing memory exhaustion or unexpected Out-Of-Memory (OOM) killer invocations on Linux servers, conventional monitoring tools such as free -m or standard top often obscure the underlying culprit. While user-space processes might appear to consume modest memory, system RAM can dwindle rapidly if kernel-space allocations expand uncontrollably. The Linux kernel relies on specialised caching algorithms—the SLAB and SLUB allocators—to manage memory for frequently allocated internal objects such as directory entries (dentries), file descriptors, and inode caches.

The slabtop utility provides real-time, interactive visibility into kernel slab caches, parsing data directly from /proc/slabinfo. For systems engineers and site reliability specialists, mastering slabtop is critical for differentiating healthy reclaimable metadata caches from insidious kernel memory leaks.

Architectural Overview: The Linux Kernel Slab Allocator

In the Linux virtual memory architecture, allocating physical memory frames via the Buddy Allocator occurs in multiples of 4 KB pages. However, the kernel frequently requires small, fixed-size data structures that measure only tens or hundreds of bytes. Allocating an entire 4 KB page for a 128-byte object would induce catastrophic internal fragmentation.

Memory Layer	Primary Allocator	Management Granularity
Physical Page Frames	Buddy System Allocator	Orders of 4 KB pages (4 KB, 8 KB, 16 KB … up to 4 MB).
Kernel Object Pools	SLUB / SLAB Allocator	Pre-initialised object caches tailored to specific kernel structs (dentry, inode, sk_buff).
Generic Kernel Memory	kmalloc-* Pools	Power-of-two generic buffers (e.g. kmalloc-64, kmalloc-512, kmalloc-2048).
Reclaimable Slabs (SReclaimable)	Kernel VFS Pruner (kswapd)	Caches that can be freed under memory pressure to satisfy user-space allocations.
Unreclaimable Slabs (SUnreclaim)	Active Kernel Subsystems	In-flight kernel objects that cannot be dropped until the owning subsystem releases them.

A kernel memory leak occurs when an unreclaimable slab cache continuously grows without bound, or when the Virtual Filesystem (VFS) cache pressure is configured so low that reclaimable dentries crowd out working application sets.

Running slabtop and Interactive Navigation

The slabtop command displays a top-like curses interface summarising active caches, total object counts, cache sizing, and allocation utilisation percentages. Because it accesses restricted kernel interfaces, running slabtop requires elevated root privileges.

Step 1: Launch Interactive Mode

# Launch slabtop with the default refresh interval (3 seconds)
sudo slabtop

Step 2: Understand the Interface Header

The top four lines of the slabtop display present high-level aggregate metrics:

 Active / Total Objects (% used)    : 3421500 / 3450200 (99.2%)
 Active / Total Slabs (% used)      : 86255 / 86255 (100.0%)
 Active / Total Caches (% used)     : 142 / 215 (66.0%)
 Active / Total Size (% used)       : 1245.82M / 1256.40M (99.2%)
 Minimum / Average / Maximum Object : 0.02K / 0.36K / 12.00K

• Active Objects: Number of allocated objects currently in use by the kernel.
• Total Objects: Number of allocated objects that could fit into the allocated slabs.
• % used: Sizing efficiency. A low utilisation percentage (e.g. < 50%) indicates significant fragmentation within slab pages.

Step 3: Interactive Sorting Commands

While slabtop is running, press single key commands to alter the sort criteria instantly:

Key	Sort Metric	Diagnostic Purpose
a	Active Objects (ACTIVE)	Identifies caches holding the highest count of concurrently active instances.
c	Cache Size (CACHE SIZE)	Highlights the largest consumer of physical RAM across all slabs (default sort).
l	Slab Count (SLABS)	Reveals caches allocating excessive kernel page frames.
o	Object Size (OBJ SIZE)	Locates heavy, complex data structures in the kernel.
u	Utilisation Percentage (%USE)	Pinpoints fragmented slabs with high overhead but low object density.

Automated and Scripted Reporting with slabtop

In non-interactive environments, continuous integration systems, or telemetry collection daemons, slabtop can emit deterministic one-shot snapshots:

# Output a single snapshot sorted by cache size (-s c) in once-only mode (-o)
sudo slabtop -o -s c | head -n 25

# Output sorted by active object count (-s a)
sudo slabtop -o -s a | head -n 25

Diagnosing High Dentry and Inode Cache Consumption

The two most common consumers of gigabytes of slab memory in production Linux environments are dentry (directory entry cache) and filesystem-specific inode structures (such as ext4_inode_cache, xfs_inode, or btrfs_inode).

Investigating Dentry Bloat

Dentries are created whenever a process traverses paths in the filesystem. If find jobs, backup agents, or container file scanners traverse millions of unique paths (such as deeply nested Git repositories or Node.js node_modules trees), millions of dentries are pinned in RAM.

# Inspect memory distribution between reclaimable and unreclaimable slabs
grep -E '(Slab|SReclaimable|SUnreclaim)' /proc/meminfo

# Check active dentry cache metrics specifically
cat /proc/sys/fs/dentry-state

The third number in /proc/sys/fs/dentry-state represents negative dentries (paths that were queried but do not exist). A very high negative dentry count indicates malicious path scanning or poorly written path-probing application loops.

Tuning Kernel VFS Cache Pressure and Reclaiming Memory

When slab caches grow excessively large and impede application performance, kernel sysctl parameters can be adjusted to tune pruning behaviour.

Step 1: Inspect vfs_cache_pressure

The vm.vfs_cache_pressure parameter controls the kernel’s tendency to reclaim directory and inode caches relative to pagecache and swap memory:

# View current value (default is typically 100)
sysctl vm.vfs_cache_pressure

• Value = 100: The kernel maintains a fair balance between reclaiming pagecache and slab caches.
• Value < 100 (e.g. 50): The kernel strongly prefers retaining directory and inode caches in RAM, which accelerates filesystem lookups but risks memory exhaustion.
• Value > 100 (e.g. 150–200): The kernel aggressively evicts dentries and inodes, keeping slab footprint lean.

Step 2: Adjust vfs_cache_pressure Dynamically

# Temporarily raise cache pressure to aggressively purge metadata
sudo sysctl -w vm.vfs_cache_pressure=150

# Make persistent across reboots
echo "vm.vfs_cache_pressure = 150" | sudo tee -a /etc/sysctl.d/99-vfs-cache.conf

Step 3: Forcing Manual Slab Cache Drop

If memory is critically low and you must test whether the slab memory is reclaimable without restarting the system, issue a kernel drop caches sync:

# Flush unwritten filesystem buffers to disk first
sync

# Echo '2' to free slab objects (dentries and inodes)
# (Echo '3' frees pagecache AND slab objects)
echo 2 | sudo tee /proc/sys/vm/drop_caches

# Observe slabtop changes immediately
sudo slabtop -o -s c | head -n 15

Troubleshooting Kernel Slab Anomalies

When slab consumption exhibits abnormal behaviour, consult this diagnostic matrix:

Symptom	Underlying Root Cause	Remediation Procedure
SUnreclaim in /proc/meminfo continues expanding despite drop_caches.	Kernel driver or subsystem memory leak (e.g. network socket buffers sk_buff, eBPF maps, or third-party DKMS modules).	Identify the specific growing cache in slabtop. Use perf or kernel kmemleak (if compiled with CONFIG_DEBUG_KMEMLEAK) to trace unfreeable allocations.
kmalloc-1024 or kmalloc-2048 consumes majority of RAM.	Unbounded allocation of generic kernel buffers by network drivers or cryptographic pipes.	Audit network interface ring buffers (ethtool -g) and inspect open sockets with ss -s to check socket buffer quotas.
OOM killer terminates application, but free -m shows hundreds of megabytes in “buff/cache”.	Memory was locked in SUnreclaim or fragmented slabs that the page allocator could not coalesce into contiguous physical blocks.	Inspect /proc/buddyinfo for external fragmentation. Tune vm.min_free_kbytes to maintain adequate headroom for page allocation.
slabtop fails with “Cannot read /proc/slabinfo”.	Command executed without sudo or system kernel restricts slabinfo access via kernel.dmesg_restrict / hardened security profiles.	Execute with elevated permissions (sudo slabtop) and verify kernel permissions on /proc/slabinfo.

By regularly monitoring slab allocations with slabtop, configuring vfs_cache_pressure, and isolating unmanaged kernel expansions, systems administrators can prevent silent OOM panics and maintain pristine memory efficiency across dense production workloads.