How to Configure Late Shutdown Hooks in Ubuntu Using systemd-shutdown

In Linux server environments, automating graceful service shutdown is typically handled through standard systemd unit files using directives such as ExecStop= and TimeoutStopSec=. However, during the final moments of system shutdown, reboot, or poweroff, standard systemd services, daemon processes, and user sessions are already completely terminated. At this final stage, the root filesystem is remounted read-only, swap partitions are deactivated, and process execution transitions from PID 1 to the specialised binary systemd-shutdown. For edge computing hardware, physical appliances, and enterprise infrastructure that require late hardware operations—such as final storage cache flushing, hardware watchdog petting, remote out-of-band power notifications, or GPIO relay triggers—Ubuntu provides late shutdown hooks via /usr/lib/systemd/system-shutdown/.

Execution Lifecycle of systemd-shutdown in Ubuntu

When an administrator or automated system triggers a reboot or poweroff (e.g. via reboot, poweroff, or systemctl halt), systemd executes a highly choreographed shutdown sequence:

1. Service Deactivation: Targets such as shutdown.target, reboot.target, and final.target are reached. All active systemd services receive SIGTERM followed by SIGKILL.
2. Filesystem Teardown: Non-root filesystems, network mounts (NFS/CIFS), and loop devices are unmounted. Encrypted dm-crypt/LUKS devices and LVM volumes are detached.
3. The systemd-shutdown Pivot: PID 1 replaces itself via execve() with the binary located at /usr/lib/systemd/systemd-shutdown.
4. Late Hook Execution: Immediately before invoking the kernel reboot() system call, systemd-shutdown iterates through executable files located inside /usr/lib/systemd/system-shutdown/ and /etc/systemd/system-shutdown/.
5. Kernel Halt: Once the late hook scripts exit, the binary issues the final syscall to power off the motherboard or pulse the hardware reset pin.

Hook Arguments and Runtime Constraints

Because late shutdown hooks run when the root filesystem is remounted read-only and all background daemons (including systemd-journald and networking stacks) are completely dead, scripts must operate under strict constraints:

• Positional Action Argument: systemd-shutdown executes each hook by passing a single command-line argument indicating the intended operation: reboot, poweroff, halt, or kexec. Your script can evaluate this argument ($1) to take conditional action.
• No Writable Filesystem: You cannot write logs to /var/log/ or create runtime locks in /tmp/ unless you explicitly mount a transient RAM disk (tmpfs) or write directly to standard error (which forwards to the active console terminal).
• Static or Standalone Binaries: Complex shell scripts relying on external dependencies or interpreted environments (such as Python or Perl) may fail if shared libraries are inaccessible. Standard POSIX shell scripts utilizing statically linked binaries or kernel sysfs/procfs interfaces are recommended.

Creating a Custom Late Shutdown Hook Script

To implement a custom late hook, create an executable script inside the administrative directory /etc/systemd/system-shutdown/ (or /usr/lib/systemd/system-shutdown/ for distribution packages):

# Create the directory if it does not already exist
sudo mkdir -p /etc/systemd/system-shutdown/

# Create a late shutdown script
sudo nano /etc/systemd/system-shutdown/hardware-notify.sh

Insert the script logic, checking the positional argument passed by systemd-shutdown:

#!/bin/sh
# Late shutdown hook for Ubuntu
ACTION="$1"

# Echo message directly to the hardware console
echo "Executing late shutdown hook for action: $ACTION" > /dev/console

case "$ACTION" in
    reboot)
        echo "Preparing hardware reset registers for reboot..." > /dev/console
        # Example: Triggering hardware watchdog or hardware register
        ;;
    poweroff)
        echo "Preparing out-of-band power-down sequence..." > /dev/console
        # Example: GPIO pin toggling or hardware bus sync
        ;;
    halt)
        echo "System halting without power cutoff..." > /dev/console
        ;;
    kexec)
        echo "Direct kexec kernel handover requested..." > /dev/console
        ;;
    *)
        echo "Unknown shutdown state encountered." > /dev/console
        ;;
esac

exit 0

Applying Proper File Permissions and Testing Execution

Late shutdown hooks must have executable permissions, otherwise systemd-shutdown skips them silently during teardown:

# Make the hook script executable
sudo chmod +x /etc/systemd/system-shutdown/hardware-notify.sh

# Verify ownership belongs to root
sudo chown root:root /etc/systemd/system-shutdown/hardware-notify.sh

Testing and Debugging Late Shutdown Hooks

Because late shutdown output cannot be captured by journalctl (as the logging daemon is already stopped), output must be verified on the physical or virtual console:

1. Connect to the system via a physical monitor, virtual machine console (e.g. Hyper-V VMConnect or Proxmox VNC), or serial console (/dev/ttyS0).
2. Initiate a system reboot by executing:

   sudo systemctl reboot

3. Observe the terminal scroll during the final shutdown sequence. Immediately before the machine resets, observe your hook output echoing directly to /dev/console.

Using systemd-shutdown hooks guarantees deterministic execution at the absolute lowest layer of the Ubuntu operating system lifecycle, providing a robust solution for embedded controllers, hardware watchdogs, and high-reliability server platforms.