How to Configure Declarative Kernel Parameters in Ubuntu Using systemd-sysctl

Modern Linux distributions manage runtime kernel parameters through the kernel sysctl interface, exposing low-level networking, virtual memory, IPC, and filesystem toggles through /proc/sys/. Historically, administrators maintained these tunings in a monolithic /etc/sysctl.conf file executed by system init scripts. In contemporary Ubuntu releases governed by systemd, kernel parameter management is entirely declarative, automated, and prioritised across drop-in configuration hierarchies through systemd-sysctl.service and its underlying utility /usr/lib/systemd/systemd-sysctl.

Understanding the declarative loading precedence of systemd drop-in directories allows system administrators and DevOps engineers to deploy granular, version-controlled kernel configurations without risking package update collisions or uncoordinated parameter overwrites. This guide covers how systemd-sysctl evaluates configuration files, how to create modular configuration snippets, how to test and apply changes dynamically, and how to troubleshoot failed runtime assignments.

Understanding systemd-sysctl Configuration Precedence

Rather than reading a single global file, systemd-sysctl scans several system directories in a strictly defined order. Configuration files ending with the .conf extension are sorted and evaluated alphabetically across these directories. When duplicate file names appear in multiple directories, the file located in the directory with higher precedence completely overrides the file of the same name in lower-precedence directories.

The system evaluates directories in descending priority:

• /etc/sysctl.d/*.conf: Local administrator and custom server configurations. This location holds highest priority and takes precedence over vendor defaults.
• /run/sysctl.d/*.conf: Runtime configurations generated dynamically by services or orchestration tooling during boot. These files do not persist across reboots.
• /usr/local/lib/sysctl.d/*.conf: Locally compiled software packages and third-party vendor installations.
• /usr/lib/sysctl.d/*.conf: Upstream distribution packages and OS vendor defaults provided by Ubuntu packages (such as procps or systemd).

Legacy configuration located at /etc/sysctl.conf is still supported for backward compatibility, but systemd treats it as an alias evaluated alongside /etc/sysctl.d/99-sysctl.conf. For production deployments, dedicated drop-in files inside /etc/sysctl.d/ are recommended.

Inspecting Active Sysctl Directives and Current Settings

Before applying new kernel configurations, inspect the current parameters loaded into the kernel memory. You can list all active values or query a specific namespace directly using the sysctl CLI.

To inspect active values for virtual memory management, execute:

sysctl vm.swappiness vm.dirty_ratio vm.dirty_background_ratio

To query specific TCP networking parameters, such as BBR congestion control and maximum receive buffer sizes, run:

sysctl net.ipv4.tcp_congestion_control net.core.rmem_max net.core.wmem_max

To determine exactly which configuration files are currently being discovered and evaluated by systemd, execute the following command:

systemd-sysctl --cat-config

The --cat-config flag prints the aggregated contents of all active .conf files discovered across /etc/sysctl.d/, /run/sysctl.d/, and /usr/lib/sysctl.d/, displaying the full file path above each section so you can audit which packages establish existing baseline values.

Creating Declarative Sysctl Drop-In Configurations

To implement modular, maintainable configurations, group related kernel parameters into isolated files inside /etc/sysctl.d/ with explicit priority prefixes (such as 60- or 80-).

For example, to configure hardened network security parameters such as SYN flood protection and reverse path filtering, create a dedicated drop-in file:

sudo nano /etc/sysctl.d/60-network-hardening.conf

Populate the configuration with the desired key-value pairs:

# Enable SYN cookies against SYN flood attacks
net.ipv4.tcp_syncookies = 1

# Enable strict reverse path filtering on all interfaces
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Ignore ICMP echo broadcasts to prevent Smurf amplification
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Log martian packets with impossible addresses
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1

# Disable ICMP redirect acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0

Similarly, for database workloads or high-performance servers requiring optimised virtual memory and file descriptor scaling, create a separate configuration file:

sudo nano /etc/sysctl.d/70-vm-performance.conf

Define the targeted memory behaviour:

# Reduce swapping aggression for latency-sensitive applications
vm.swappiness = 10

# Increase maximum queued connection backlog
net.core.somaxconn = 4096

# Increase maximum file handles system-wide
fs.file-max = 2097152

# Allocate broader ephemeral port range
net.ipv4.ip_local_port_range = 10240 65535

Ensure the newly created files possess standard root read permissions (644):

sudo chmod 0644 /etc/sysctl.d/60-network-hardening.conf /etc/sysctl.d/70-vm-performance.conf

Applying and Validating Drop-In Changes at Runtime

Rebooting the server is unnecessary when modifying sysctl parameters. You can trigger systemd-sysctl directly to reload and parse all configuration files into the live kernel.

To reload all declarative sysctl configurations across all directories, execute:

sudo /usr/lib/systemd/systemd-sysctl

Alternatively, you can target an individual configuration file directly to test new settings without reloading the entire system inventory:

sudo /usr/lib/systemd/systemd-sysctl /etc/sysctl.d/60-network-hardening.conf

Verify that the parameters have been accepted by querying the kernel directly:

sysctl net.ipv4.tcp_syncookies net.ipv4.conf.all.rp_filter vm.swappiness

You can also check the raw kernel pseudofiles directly within the /proc/sys/ virtual filesystem:

cat /proc/sys/net/ipv4/tcp_syncookies
cat /proc/sys/vm/swappiness

Masking and Overriding Upstream Package Defaults

When third-party packages or system defaults in /usr/lib/sysctl.d/ establish settings that conflict with your infrastructure policies, you can mask or override them using systemd file resolution rules.

For example, if an upstream package installs /usr/lib/sysctl.d/50-default.conf with parameters you wish to nullify, you can create a symlink pointing to /dev/null inside /etc/sysctl.d/ with the identical filename:

sudo ln -s /dev/null /etc/sysctl.d/50-default.conf

When systemd-sysctl processes the directories, the existence of 50-default.conf in /etc/sysctl.d/ takes precedence over the vendor file in /usr/lib/sysctl.d/. Because the file links to /dev/null, systemd completely masks and skips the configuration.

If you wish to modify individual parameters from a vendor file rather than masking the entire file, copy the file to /etc/sysctl.d/ and adjust the values, or place a higher-numbered file (such as 99-custom-overrides.conf) in /etc/sysctl.d/ so that your parameters are evaluated last.

Troubleshooting systemd-sysctl Boot Service Errors

During initial operating system boot, systemd-sysctl.service runs early in the startup sequence. If a configuration file references a module parameter before the corresponding kernel module has loaded, the service logs a failure.

To inspect the status and logs of the systemd sysctl unit, execute:

systemctl status systemd-sysctl.service

To view detailed diagnostic output and error messages, inspect the system journal:

journalctl -u systemd-sysctl.service --no-pager

Common failure modes include:

• Missing Kernel Modules: Parameters under namespaces such as net.netfilter.* or net.bridge.* will fail to apply if the br_netfilter or nf_conntrack modules have not yet loaded. Ensure the required modules are declared in /etc/modules-load.d/ so that systemd loads them before systemd-sysctl.service runs.
• Read-Only Parameters: Some kernel keys (such as certain hardware topology or architecture values) are read-only at runtime. Attempting to assign values to these keys will cause systemd-sysctl to return a Permission denied or Invalid argument error.
• Prefix Syntax Errors: Using invalid key names or malformed syntax halts parsing of the affected line. Test custom configurations using sudo /usr/lib/systemd/systemd-sysctl /path/to/file.conf to catch syntax discrepancies immediately.