How to Deploy the Linux PSI (Pressure Stall Information) Interface to Mathematically Quantify Resource Starvation

When a Linux server experiences performance degradation, administrators instinctively rely on legacy tools like top, htop, or vmstat to diagnose the bottleneck. They examine the CPU utilization percentage or the load average (e.g., 15.4, 12.2, 8.9). However, these legacy metrics are mathematically flawed in modern, highly concurrent environments (especially Kubernetes nodes running hundreds of cgroups). A CPU utilization of 100% does not inherently mean the system is failing; it simply means the hardware is being fully utilized. Conversely, a system can grind to a halt with only 40% CPU utilization if threads are stalled waiting for disk I/O or memory swapping. To provide a mathematically precise, deterministic measurement of exactly how long a system is stalled waiting for hardware resources, the Linux kernel community introduced the Pressure Stall Information (PSI) interface.

The Mathematical Superiority of PSI

PSI fundamentally alters the telemetry paradigm. Instead of measuring utilization (what the hardware is doing), it measures stall time (what the software is prevented from doing).

Developed primarily by Facebook engineers to optimize their massive fleet, PSI tracks the exact amount of time that a task (a process or thread) is runnable but unable to execute because it is waiting for one of three core resources: CPU, Memory, or I/O.

PSI calculates two distinct metrics:

1. Some: The percentage of time in which at least one task was stalled waiting for a resource. This indicates minor contention (latency).
2. Full: The percentage of time in which all non-idle tasks were simultaneously stalled waiting for a resource. This indicates catastrophic contention (total system lockup). If memory PSI “full” hits 100%, the entire server is completely frozen, thrashing the swap file, and unable to process a single instruction.

Enabling the PSI Subsystem

Because tracking every single thread context switch introduces a microscopic (less than 1%) CPU overhead, PSI is sometimes disabled by default depending on your Linux distribution (though it is enabled by default on Ubuntu 22.04+ and Fedora).

To enable it, you must append a kernel boot parameter. Open your GRUB configuration (e.g., /etc/default/grub) and add psi=1 to the GRUB_CMDLINE_LINUX string:

GRUB_CMDLINE_LINUX="... psi=1"

Update GRUB and reboot the server:

sudo update-grub
sudo reboot

Reading and Interpreting the Telemetry

Once enabled, the kernel exposes the telemetry directly via the /proc filesystem. You do not need any specialized software to read it. The metrics are aggregated over three distinct time windows: 10 seconds, 60 seconds, and 300 seconds.

To view the current CPU pressure stall information, execute:

cat /proc/pressure/cpu

The output will resemble this:

some avg10=2.34 avg60=1.12 avg300=0.45 total=543120

This output mathematically proves that over the last 10 seconds, 2.34% of the time, at least one thread was ready to execute but had to wait for a CPU core. (Note: CPU only possesses the some metric, because a CPU cannot stall “all” processes simultaneously; if a process is running on the CPU, the CPU is not stalled).

To view the Memory pressure stall information, execute:

cat /proc/pressure/memory

The output will resemble this:

some avg10=0.00 avg60=0.00 avg300=0.00 total=2340
full avg10=0.00 avg60=0.00 avg300=0.00 total=1200

If you begin seeing the full metric for Memory spike above 0.00, it mathematically confirms that the kernel is heavily paging memory to disk (swapping), and the entire application stack is completely blocked during those microsecond intervals.

Integrating PSI with cgroups v2

The true power of PSI is not just global server monitoring, but container-level isolation. PSI integrates natively with the cgroups v2 architecture.

If you navigate to a specific cgroup (e.g., a specific Docker container or systemd service), you can interrogate its localized PSI telemetry:

cat /sys/fs/cgroup/system.slice/docker-12345.scope/memory.pressure

This allows a Kubernetes orchestrator or a custom daemon (like Facebook’s oomd or systemd’s systemd-oomd) to autonomously detect when a specific container is experiencing severe memory pressure. Rather than waiting for the kernel’s aggressive Out-Of-Memory (OOM) killer to randomly annihilate processes, the daemon can mathematically observe the memory.pressure spike and gracefully terminate the exact container causing the contention before the host server crashes.