How to Deploy Configuration Overlays in Ubuntu Using systemd-confext

Modern Linux server deployments, immutable appliance distributions, and edge gateways increasingly deploy the operating system as a read-only, signed base image. While this architecture guarantees resilience and tamper resistance, injecting site-specific configurations—such as corporate CA certificates, custom systemd unit drop-ins, network configurations, or security audit rules—into /etc/ traditionally required baking distinct base images for every environment or managing brittle post-boot bash scripts. Introduced in recent systemd versions, systemd-confext provides a standardized, modular mechanism to merge Discoverable Disk Images (DDIs) into /etc/ using Linux overlayfs.

In this guide, you will learn how to create configuration extension images, merge them into /etc/ at boot, audit active overlays, and refresh configurations in Ubuntu using systemd-confext.

Understanding Configuration Extensions (confext) vs System Extensions (sysext)

The systemd image extension ecosystem provides two distinct mechanisms:

• System Extensions (systemd-sysext): Merges read-only binary packages, libraries, and binaries into /usr/ and /opt/. Used to add debugging tools, container runtimes, or hardware drivers to immutable OS installations.
• Configuration Extensions (systemd-confext): Merges read-only configuration trees into /etc/. Designed exclusively for declarative configuration management, ensuring the underlying base OS image remains completely untouched and pristine.

Each configuration extension image contains an etc/ directory hierarchy along with an extension metadata definition (usr/lib/extension-release.d/extension-release.<NAME>) matching the host’s OS ID and version.

Verifying systemd-confext Availability

On Ubuntu systems running systemd 254 or later, verify that systemd-confext is installed:

systemd-confext --version

Ensure that the overlayfs kernel driver is active:

sudo modprobe overlay
lsmod | grep overlay

Creating a Custom Configuration Extension Image

To demonstrate configuration extensions, create an extension image named site-hardening that delivers a custom systemd service configuration and an enterprise security banner:

1. Create a temporary staging directory hierarchy:

   mkdir -p /tmp/site-hardening/etc/issue.d
   mkdir -p /tmp/site-hardening/etc/systemd/system
   mkdir -p /tmp/site-hardening/usr/lib/extension-release.d

2. Create the extension release metadata file. This file ensures the extension only binds to matching OS environments:

   cat << 'EOF' > /tmp/site-hardening/usr/lib/extension-release.d/extension-release.site-hardening
   ID=ubuntu
   VERSION_ID=24.04
   EOF

   Note: Match the ID and VERSION_ID values exactly to your host’s /etc/os-release.
3. Populate your configuration files inside etc/:

   # Add a custom security login banner
   echo "Authorized Enterprise Access Only. All activities are monitored." > /tmp/site-hardening/etc/issue.d/01-banner.issue
   
   # Add a custom environment configuration
   echo "MANAGED_BY=systemd-confext" > /tmp/site-hardening/etc/environment.d/99-enterprise.conf

4. Pack the staging directory into a squashfs or raw disk image:

   mksquashfs /tmp/site-hardening /var/lib/confexts/site-hardening.raw -noappend

By default, systemd-confext discovers extension images located in /var/lib/confexts/, /etc/confexts/, and /usr/lib/confexts/.

Inspecting and Auditing Extension Images

Before applying the overlay to your live system, query systemd-confext to audit available extension images:

systemd-confext list

The command lists all discovered images, displaying the extension name, image path, version match status, and whether the extension is currently active or unmerged.

To verify that the extension’s OS metadata matches the current host kernel and release:

systemd-confext status

Merging Configuration Extensions into /etc

To mount and overlay the configuration images on top of /etc/:

sudo systemd-confext merge

The kernel establishes an overlayfs mount on /etc/. Your extension’s files are immediately merged with existing configuration files.

Verify that your new configuration files appear in /etc/:

cat /etc/issue.d/01-banner.issue
cat /etc/environment.d/99-enterprise.conf

Notice that existing system files in /etc/ remain completely functional and intact alongside the merged files.

Unmerging and Refreshing Configurations Dynamically

To revert /etc/ back to its factory base state without rebooting:

sudo systemd-confext unmerge

The overlayfs mount is cleanly dismantled, and the custom files disappear from /etc/ immediately.

If you update an extension image on disk (for example, after pushing new configurations via your configuration management orchestrator), apply the updates atomically using refresh:

sudo systemd-confext refresh

This command unmerges existing overlays and re-mounts the updated disk images in a single atomic operation.

Enabling Automatic Boot Merging

To ensure configuration extensions are automatically merged during the earliest boot phases before network and security daemons initialize, enable the built-in systemd service:

sudo systemctl enable systemd-confext.service

During the system boot sequence, systemd mounts all valid extensions found in /var/lib/confexts/ before executing secondary target services, ensuring consistent, declarative system configurations across all cloud and edge nodes.