How to Use the lsmem and chmem Commands in Linux to Inspect and Manage Hotplug Memory Blocks

In modern Linux enterprise computing, virtualisation, and high-performance server administration, dynamically adjusting system memory without restarting the operating system is a fundamental capability. Cloud hypervisors—such as KVM/QEMU, VMware ESXi, Microsoft Hyper-V, and AWS EC2 Nitro instances—allow administrators to dynamically attach or detach RAM to running virtual machines via memory hotplug mechanisms. At the kernel level, Linux organises physical memory into modular, discrete sections known as memory blocks, exposed through the sysfs filesystem under /sys/devices/system/memory/. To inspect and manipulate these memory blocks from the command line, the util-linux suite provides two purpose-built administrative utilities: lsmem and chmem.

While standard diagnostic tools like free or vmstat report aggregated byte counters and memory utilisation metrics, they lack visibility into the underlying hardware address ranges, NUMA node assignments, or whether individual memory chunks are removable. The lsmem command provides a comprehensive structural view of physical memory blocks, while chmem gives administrators low-level control to set blocks online or offline dynamically. This in-depth guide covers the Linux memory block architecture, demonstrates how to inspect address ranges with lsmem, explains hotplug memory management with chmem, and addresses common challenges when offlining memory in production environments.

Understanding Linux Memory Blocks and Hotplug Architecture

The Linux kernel abstracts physical RAM into uniform memory blocks. The size of an individual memory block depends on the system architecture and kernel configuration; on standard x86_64 systems, a block is typically 128 MB, whereas on s390x or large ARM64 enterprise platforms, block sizes may scale to 1 GB or 2 GB.

Command / Tool	Primary Diagnostic Focus	Granularity / Scope	Hardware & Hotplug Control
lsmem	Physical memory block ranges, online/offline status, NUMA affinity.	Per-block memory hardware descriptors.	Read-only inspection and state reporting.
chmem	Setting memory blocks online or offline dynamically.	Block ranges or specified memory capacities.	Active runtime modification (hotplug).
free	High-level RAM allocation (used, free, buffers, cached, available).	System-wide aggregate counters.	None (metric reporting only).
numactl	NUMA node topology, CPU affinity, and memory allocation policy.	Per-node memory nodes and distance.	Process memory binding and allocation policy.

Each memory block in the Linux kernel exists in one of several states:

• online: The memory block is actively mapped into the kernel page tables and available for general memory allocation (typically in ZONE_NORMAL).
• offline: The memory block is physically present (or allocated by the hypervisor) but unmapped by the kernel, consuming zero operational overhead.
• online_movable: The block is placed into ZONE_MOVABLE, ensuring it only holds page-cache data or anonymous user-space memory that can be migrated when offlining.

Inspecting Physical Memory with the lsmem Command

The lsmem utility reads entries from /sys/devices/system/memory/ and displays a structured, human-readable summary of all memory ranges. To view the default memory block layout, open your terminal and execute:

lsmem

A typical output on a modern Linux server looks similar to the following:

RANGE                                 SIZE  STATE REMOVABLE  BLOCK
0x0000000000000000-0x000000007fffffff   2G online       no   0-15
0x0000000100000000-0x000000027fffffff   6G online      yes  32-79

Memory block size:       128M
Total online memory:       8G
Total offline memory:      0B

The output provides several critical pieces of architectural information:

• RANGE: The physical start and end hexadecimal memory addresses mapped to the block group.
• SIZE: The cumulative capacity of the memory range.
• STATE: Indicates whether the range is currently online or offline.
• REMOVABLE: Highlights whether the kernel can safely migrate contents to take the block offline (yes) or whether it contains unmovable kernel structures (no).
• BLOCK: The sequential sysfs block identifier index (e.g., 0-15 corresponds to /sys/devices/system/memory/memory0 through memory15).

Customising lsmem Output for Scripting and Auditing

To inspect individual memory blocks without automatic range consolidation, pass the --all (or -a) option:

lsmem --all

To display NUMA node assignments alongside memory blocks on multi-socket enterprise servers, use the --split (or -s) flag:

lsmem --split=NODE

For automated system audits, CI/CD provisioning scripts, or monitoring agents, lsmem natively supports JSON formatting and raw delimiter-free output:

lsmem --json

Managing Hotplug Memory Blocks with chmem

The chmem command allows system administrators to change the operational state of memory blocks dynamically. Because altering kernel memory mappings directly affects system stability, executing chmem requires elevated superuser privileges via sudo.

Enabling Offline Memory by Capacity

When you allocate additional RAM to a running virtual machine in a cloud hypervisor, the new memory blocks often register in an offline state initially. To bring a specific volume of memory online—for instance, 2 gigabytes—run:

sudo chmem --enable 2G

Terminal reports the status of the operation:

chmem: Memory 0x0000000280000000-0x00000002ffffffff enabled

You can verify that the new memory is immediately available to the operating system using free -h:

free -h

Enabling Memory into ZONE_MOVABLE for Safe Future Removal

If you intend to remove or reclaim memory later (for dynamic cloud auto-scaling), you should bring memory blocks online specifically into the kernel’s movable zone. This prevents unmovable kernel allocations from being placed in those blocks:

sudo chmem --enable --movable 2G

Blocks initialised with --movable will consistently register as REMOVABLE: yes in subsequent lsmem audits.

Disabling or Offlining Memory Blocks

To reduce RAM allocation before dynamically downsizing a virtual guest, use the --disable (or -d) parameter:

sudo chmem --disable 1G

The kernel will scan active pages, migrate movable user-space data and file caches to alternative online memory zones, and unmap the specified memory blocks.

Targeting Specific Memory Ranges and Block IDs

Rather than specifying a generic memory size, you can target exact hexadecimal address ranges or block indices identified via lsmem:

sudo chmem --disable 0x0000000280000000-0x00000002ffffffff

Alternatively, target specific block numbers directly:

sudo chmem -b 64-79 -d

Troubleshooting: Resolving “Device or resource busy” Errors

When attempting to offline memory using chmem --disable, you may occasionally encounter the following error:

chmem: Could not disable memory
chmem: Device or resource busy

This failure occurs when the target memory blocks contain “unmovable” memory pages. Examples include:

• Direct kernel data structures, slab caches, or page tables.
• Pages pinned by device drivers performing Direct Memory Access (DMA).
• Locked memory segments created via the mlock() system call.

To overcome this limitation, ensure that hotplugged memory intended for temporary or elastic scaling is always initialised using the --movable flag upon activation. Additionally, inspecting lsmem beforehand allows you to verify that the REMOVABLE column displays yes before attempting an offline transition.

By integrating lsmem and chmem into your administration toolkit, you gain precise, low-level oversight over Linux physical memory management, ensuring seamless hotplug scalability across modern hypervisors and enterprise infrastructure.