How to Configure systemd-pstore to Capture Kernel Panic Dumps on Ubuntu Server

Diagnosing spontaneous server reboots, unhandled hardware exceptions, and kernel panic crashes on headless Ubuntu servers presents significant diagnostic obstacles. Under catastrophic failure conditions, the Linux kernel crashes so abruptly that storage subsystems cannot write crash telemetry to persistent log files in /var/log/dmesg or the systemd journal before the hardware undergoes a hard reboot.

The Linux kernel features a specialized persistent storage mechanism known as pstore (Persistent Storage). Paired with the native systemd-pstore service in modern Ubuntu Server releases, pstore utilizes non-volatile platform memory—such as ACPI ERST (Error Record Serialization Table), EFI variable NVRAM, or battery-backed RAM—to capture the kernel’s dying ring buffer messages. Upon system restart, systemd-pstore automatically archives these persistent panic logs into standard files for forensic analysis.

How pstore and systemd-pstore Operate

The persistent storage architecture operates across two distinct operational stages:

• Panic-Time Capture: When a fatal kernel panic or oops occurs, the kernel invokes the pstore subsystem. Bypassing disk I/O schedulers and filesystem layers entirely, pstore writes the final kernel log messages (dmesg) and panic trace directly into platform NVRAM or ACPI memory structures before hardware reset.
• Post-Reboot Archival: During the subsequent system boot, systemd mounts the virtual /sys/fs/pstore/ filesystem. The systemd-pstore.service daemon reads any recorded panic fragments, saves them permanently into /var/lib/systemd/pstore/, and flushes the NVRAM storage to ensure capacity remains available for future events.

Verifying pstore Kernel Support on Ubuntu

To check whether your Ubuntu Server kernel has pstore mounted and supported by hardware firmware:

# Check mounted pstore virtual filesystem
mount | grep pstore

You should see confirmation that pstore is mounted at /sys/fs/pstore using filesystem type pstore.

You can also inspect which backend storage driver your hardware platform initialized:

dmesg | grep -i pstore

Modern enterprise servers typically initialize erst: Attached to ACPI ERST or efi_pstore: Mounted platform backend.

Enabling and Configuring the systemd-pstore Service

The archival service is configured via /etc/systemd/pstore.conf:

1. Open the configuration file in a text editor:

sudo nano /etc/systemd/pstore.conf

2. Ensure the following directives are configured:

[PStore]
# Automatically copy pstore dumps to /var/lib/systemd/pstore/
Storage=external

# Remove logs from NVRAM once archived to prevent filling firmware memory
Unlink=yes

3. Enable and start the systemd-pstore service:

sudo systemctl enable --now systemd-pstore.service

Inspecting Archived Panic Logs After an Outage

When an unhandled kernel panic triggers an unexpected reboot, inspect the archived panic dumps:

ls -la /var/lib/systemd/pstore/

Captured log files follow predictable naming conventions:

• dmesg-<backend>-<timestamp>: Contains the complete kernel printk ring buffer leading up to the crash.
• dmesg-oops-<id>: Highlights the specific stack trace, CPU registers, and calling function that failed.

You can examine the crash stack trace using standard text tools:

cat /var/lib/systemd/pstore/dmesg-* | head -n 50

This reveals the exact kernel function, faulty device driver, or hardware memory parity error responsible for the panic.

Configuring systemd-pstore ensures system administrators never lose critical diagnostic telemetry following catastrophic hardware crashes on Ubuntu Server.