How to Trace VFS Read and Write Call Rates in Linux Using vfsstat

The Linux Virtual File System (VFS) handles all system-level file and storage interactions, acting as a high-performance bridge between user applications and concrete file system implementations (such as ext4, XFS, Btrfs, and NFS). While tools like iostat or iotop measure block device transfers and disk throughput, they provide no visibility into file operations served entirely by the operating system page cache. The vfsstat utility, part of the BPF Compiler Collection (BCC), dynamically traces high-level VFS function calls in real time, streaming per-second activity rates across all active mounts.

Understanding VFS Operations in Linux

When an application reads or modifies files, the kernel routes execution through core VFS functions:

• vfs_read: Handles read requests across all file descriptors.
• vfs_write: Processes write operations and page cache allocations.
• vfs_fsync: Flushes dirty buffers to persistent physical storage.
• vfs_open: Resolves pathnames, locates inodes, and creates open file descriptions.
• vfs_create: Handles the creation of new inodes and directory entries.

Measuring the frequency of these operations helps system administrators evaluate application I/O patterns, detect excessive synchronous flushing, and identify runaway metadata loops.

Installing the BCC Tools Suite

To run vfsstat, install the BCC package and kernel development headers for your running kernel version.

On Debian, Ubuntu, and Linux Mint:

sudo apt update
sudo apt install bpfcc-tools linux-headers-$(uname -r)

On RHEL, Rocky Linux, AlmaLinux, or Fedora:

sudo dnf install bcc-tools kernel-devel-$(uname -r)

The executable is typically installed at /usr/sbin/vfsstat-bpfcc or /usr/share/bcc/tools/vfsstat.

Streaming Real-Time VFS Operation Rates

Running vfsstat with a specified interval (in seconds) prints periodic summaries of VFS function calls per second:

sudo vfsstat-bpfcc 1

Sample output from a live server:

TIME         READ/s  WRITE/s CREATE/s   OPEN/s  FSYNC/s
18:24:01      18420     1250        4      120        2
18:24:02      24190     1890       12      240        5
18:24:03      19510     1140        2      110        1

Each column represents the rate of kernel events executed during that sample window:

• READ/s: Number of vfs_read invocations per second.
• WRITE/s: Number of vfs_write invocations per second.
• CREATE/s: Rate of new file or inode creation.
• OPEN/s: Frequency of file open calls.
• FSYNC/s: Count of synchronous disk flushes requested by processes.

Diagnosing Application Performance Bottlenecks

Evaluating the balance between VFS metrics and disk throughput pinpoints common architecture issues:

• High READ/s with zero disk I/O: Indicates excellent page cache efficiency, where datasets are read directly from RAM without hitting physical disks.
• Elevated FSYNC/s: Suggests an application or database is frequently forcing synchronous disk commits (such as unbuffered SQLite transactions or overly aggressive WAL writes), severely bottlenecking concurrency.
• High OPEN/s relative to READ/s: Highlights inefficient software that repeatedly opens, reads a few bytes, and closes files without using persistent handles or connection pooling.

Limiting Sampling Count and Output Logging

To sample file system rates for a finite window (e.g., 5-second intervals repeated 12 times for a one-minute benchmark):

sudo vfsstat-bpfcc 5 12

To archive performance telemetry to a log file for capacity planning:

sudo vfsstat-bpfcc 1 60 | tee vfs_benchmark.log

Complementary BCC Tracing Tools

When vfsstat detects abnormal call rates, pair it with deeper BCC diagnostic tools:

# Trace slow file system calls taking longer than 10 ms
sudo fileslower-bpfcc 10

# Count total VFS function calls system-wide
sudo vfscount-bpfcc

Using vfsstat provides Linux system engineers with rapid, low-overhead insight into operating system I/O pressure, allowing teams to tune storage caches and streamline database operations.