When you are managing a massive, enterprise-grade Linux server (such as a dual-socket database machine with 64 CPU cores and 1 Terabyte of RAM), the physical architecture of the motherboard drastically affects performance. Modern multi-socket servers utilize NUMA (Non-Uniform Memory Access) architecture. This means each physical CPU processor is physically wired to its own dedicated bank of RAM. If CPU 1 needs data, accessing its local RAM is incredibly fast, but if it is forced to cross the motherboard to access CPU 2’s RAM, the performance plummets. To monitor and diagnose these critical hardware-level memory bottlenecks, you must use the numastat command.
Understanding NUMA Nodes
In a standard, cheap desktop computer, all memory is pooled together globally. In a NUMA server, the memory is strictly segregated into “Nodes” (e.g., Node 0 for CPU 1, and Node 1 for CPU 2). The Linux kernel aggressively attempts to keep a running application (like a database query) entirely constrained within a single Node to maximize speed.
How to Read numastat Output
To view exactly how the Linux kernel is managing these physical memory zones, simply type the command into the terminal and press Enter:
numastatThe command will instantly output a highly technical grid of data, separated into columns (Node 0, Node 1, etc.).
You must pay strict attention to two highly specific metrics:
- numa_hit: This number represents how many times the CPU successfully found its data in its own, local, high-speed RAM bank. You want this number to be as massively high as possible.
- numa_miss: This is the most dangerous metric on the board. It represents how many times the CPU failed to find the data locally and was forced to painfully cross the motherboard to access the slower, remote RAM bank located on the other CPU. If this number is rapidly climbing, your database is actively suffering from a catastrophic architectural bottleneck.
Monitoring Specific Processes
If your server is experiencing massive numa_miss spikes, you need to identify exactly which application is causing the problem.
You can force numastat to track the exact memory allocation of a specific running process using the -p (process) flag, followed by the target PID (Process ID) or the exact name of the binary.
numastat -p mysqlThe terminal will instantly clear and draw a highly detailed table showing exactly how many megabytes of RAM the MySQL database is currently consuming on Node 0 versus Node 1. If the database is completely fragmented across both physical nodes, you will need to utilize advanced tools like taskset to forcefully pin the process to a single CPU socket.