How to Control NUMA CPU and Memory Policy Using numactl in Linux

Modern multi-socket enterprise servers and high-core-count processors utilise Non-Uniform Memory Access (NUMA) architecture to scale compute performance. In a NUMA architecture, physical system memory is segregated into dedicated nodes directly attached to specific CPU sockets or core complexes. While any CPU core can access memory attached to any node, accessing local node memory delivers substantially higher bandwidth and lower latency than traversing interconnect fabrics to access remote nodes.

The Linux kernel automatically handles NUMA scheduling and memory placement, but performance-critical workloads—such as high-throughput relational databases, hypervisors, and scientific computing nodes—frequently require manual affinity enforcement to prevent remote memory access penalties. The numactl command-line utility provides granular control over hardware inspection, process placement, and memory allocation policies across Linux environments.

Installing numactl and Inspecting NUMA Topology

Before pinning workloads, administrators must survey the server’s physical layout and identify node assignments. On enterprise distributions, the numactl package can be installed via default package managers:

# Debian, Ubuntu, and Linux Mint
sudo apt update && sudo apt install numactl -y

# RHEL, Rocky Linux, and Fedora
sudo dnf install numactl -y

To inspect the available NUMA hardware configuration, run:

numactl --hardware

The output reports the total number of operational NUMA nodes, the specific CPU core IDs associated with each node, the total physical memory installed per node, and a matrix indicating the relative interconnect distances between nodes. A distance value of 10 represents local node memory, while higher numbers (such as 21 or 30) reflect increasing latency across inter-socket buses.

Understanding NUMA Memory Allocation Policies

The numactl tool supports four primary memory allocation policies depending on the nature of the application workload:

• Local Allocation (--localalloc): Default kernel behaviour where memory is allocated on the node executing the thread. If local memory fills up, the kernel allocates memory on neighbouring nodes.
• Preferred Node (--preferred=<node>): Directs memory allocations to a specific node, but allows fallback to other nodes if memory exhaustion occurs.
• Memory Binding (--membind=<nodes>): Strictly restricts memory allocations to the specified nodes. If the allocated nodes run out of memory, allocation requests fail or trigger the Linux OOM (Out Of Memory) killer.
• Interleaving (--interleave=<nodes>): Distributes memory allocations in a round-robin fashion evenly across all designated nodes, smoothing bandwidth consumption for workloads that exceed single-node memory capacities.

Binding Processes and Workloads to Specific Nodes

To launch a latency-sensitive application bound to local CPU cores and local physical RAM, combine the CPU core affinity (--cpunodebind or --physcpubind) and memory binding switches:

1. To run a high-performance database instance exclusively on NUMA node 0 (using both node 0’s CPU cores and node 0’s RAM):

   numactl --cpunodebind=0 --membind=0 /usr/bin/postgres -D /var/lib/postgresql/data

2. To bind a process to specific physical CPU cores (e.g., cores 0 through 7) while preferring node 0’s memory pool:

   numactl --physcpubind=0-7 --preferred=0 ./compute_engine

3. To execute a memory-bandwidth-bound analytics task across all nodes using interleaved allocation:

   numactl --interleave=all python3 data_aggregation.py

Auditing Active NUMA Statistics and Process Mapping

To confirm that memory policies are functioning as intended and verify whether processes are suffering from remote memory access bottlenecks:

1. Review system-wide NUMA hit and miss counters:

   numastat

   Examine the numa_miss and numa_foreign rows. High numbers in these fields indicate that threads are frequently forced to access remote node RAM due to node memory exhaustion or misconfigured affinity rules.

2. Inspect real-time memory usage per node for an individual active process ID:

   numastat -p <PID>

3. To inspect the NUMA policy applied to the current shell or an active terminal session:

   numactl --show

Carefully aligning multi-threaded enterprise services with physical NUMA topologies via numactl eliminates bus congestion, reduces memory access latency, and unlocks maximum hardware throughput.