When troubleshooting a slow Linux server, administrators immediately check the CPU and memory usage using tools like top or htop. If those metrics look normal, but the server is still lagging or databases are responding slowly, the bottleneck is almost certainly located in the storage drives. This is known as a Disk I/O (Input/Output) bottleneck.
To definitively diagnose storage performance issues, you must monitor how fast data is being read from and written to the hard drives. In Linux, the standard tool for observing real-time storage metrics is the iostat command.
Installing the sysstat Package
While some server distributions include iostat by default, it is often missing from minimal or desktop installations. The command is bundled within a package called sysstat.
If you are using Ubuntu, Debian, or Linux Mint, open your terminal and install it by running:
sudo apt update && sudo apt install sysstat
For CentOS, RHEL, or Fedora, use:
sudo dnf install sysstat
Running a Basic iostat Report
The simplest way to use the tool is to run the command without any arguments.
iostat
This outputs a brief summary containing two main sections: a CPU utilisation report and a Device utilisation report. While useful for a quick glance, this basic command displays the average statistics since the system was last rebooted, not what is happening exactly at this moment.
How to Monitor Disk Usage in Real-Time
To find a bottleneck, you need to watch the disk activity change continuously over time. You can force iostat to update itself automatically at specific intervals.
Run the following command:
iostat -d 2
- The -d flag tells the command to only display the Device (disk) statistics, hiding the CPU data to make the output easier to read.
- The 2 tells the command to refresh the data every 2 seconds.
The terminal will now continuously print a new block of data every two seconds until you stop it by pressing Ctrl + C. Remember, the very first block of data printed is always the average since boot. Only the second block and beyond represent the actual real-time, 2-second snapshot.
Understanding the iostat Metrics
When you run the continuous report, you will see several columns of data for every storage drive attached to the system (e.g., sda, sdb, nvme0n1). Here is what the most important columns mean:
- tps (Transfers Per Second): This represents the total number of I/O requests (both read and write) issued to the physical disk per second. A consistently high number here indicates the disk is working very hard.
- kB_read/s: The amount of data being read from the drive, measured in kilobytes per second.
- kB_wrtn/s: The amount of data being written to the drive, measured in kilobytes per second.
How to View Extended Statistics (-x)
If you suspect a disk is failing or completely overwhelmed, the basic read/write speeds won’t give you the full picture. You need to see how long requests are waiting in a queue before the disk can processing them.
To view the extended statistics, add the -x flag:
iostat -dx 2
This produces a much wider output with several critical columns for diagnosing bottlenecks:
- await: The average time (in milliseconds) that an I/O request had to wait in the queue before the disk processed it. If this number is consistently high (e.g., over 20-30ms for an SSD, or over 100ms for a spinning HDD), the disk is a severe bottleneck.
- %util (Utilisation Percentage): This shows how much time the disk spent actively processing requests. If a drive is consistently sitting at 95% to 100% utilisation, it is completely saturated. No more data can be written or read until the current queue is cleared, resulting in massive system slowdowns.