The Danger of Unchecked Logs
In any Linux environment, system services, web servers, and custom applications constantly generate log data. On a high-traffic Ubuntu server running Nginx and MySQL, log files can easily grow by several gigabytes per day.
If these log files are left unchecked, they will eventually consume 100% of the available disk space on the /var partition. When a Linux partition reaches 100% capacity, catastrophic failures occur: databases crash because they cannot write transactions, web servers drop connections, and administrators often cannot even log in via SSH to fix the problem.
To prevent this, Ubuntu relies on a core utility called logrotate. This lightweight, cron-driven utility automatically rotates, compresses, archives, and deletes old log files on a predefined schedule, ensuring that logs remain manageable and disk space is never fully exhausted.
Step 1: Understanding the logrotate Architecture
logrotate is not a daemon (background service). It is a standard binary executed daily by the system’s cron daemon (specifically via the script at /etc/cron.daily/logrotate).
The global configuration file for the utility is located at /etc/logrotate.conf. This file sets the baseline defaults (e.g., rotate logs weekly, keep four weeks of backlogs, compress old logs).
However, administrators should rarely edit the global configuration. Instead, specific rules for individual applications are placed in the /etc/logrotate.d/ directory. When you install a package like nginx via apt, it automatically places a custom rotation script in this directory.
Step 2: Creating a Custom Rotation Configuration
Suppose you have deployed a custom Node.js application that writes aggressive debug logs to /var/log/myapp/debug.log. You need to rotate this log file daily and compress the old versions.
Create a new configuration file for your application:
sudo nano /etc/logrotate.d/myappAdd the following configuration block:
/var/log/myapp/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
postrotate
systemctl reload myapp > /dev/null 2>/dev/null || true
endscript
}Step 3: Decoding the Configuration Directives
Every line in the configuration block dictates precisely how logrotate will handle the file:
/var/log/myapp/*.log: The target files. The wildcard ensures it hits bothdebug.loganderror.log.daily: The frequency of rotation. Other options includeweekly,monthly, orsize 100M(which forces a rotation only when the file hits 100MB).missingok: If the log file doesn’t exist (e.g., the app hasn’t written anything yet), do not throw an error.rotate 14: Keep exactly 14 archived copies. When the 15th rotation occurs, the oldest archive is permanently deleted to free up disk space.compress: Archive the rotated logs usinggzipto save disk space.delaycompress: Crucial for active applications. It delays the compression of the most recently rotated log by one cycle, ensuring the application has completely finished writing to it before it is gzipped.notifempty: Do not rotate the file if it is entirely empty.create 0640 www-data adm: Immediately after moving the old log, create a brand new, empty log file with these specific octal permissions and ownership, so the application can continue writing immediately.
Step 4: Handling Open File Descriptors (postrotate)
The most complicated aspect of log rotation is dealing with active file descriptors. If an application (like Nginx) holds a log file open, and logrotate suddenly renames that file to error.log.1, Nginx will continue blindly writing to error.log.1, ignoring the newly created error.log.
To solve this, you must use the postrotate script block.
postrotate
systemctl reload myapp > /dev/null 2>/dev/null || true
endscriptThis tells logrotate to execute a specific bash command immediately after renaming the files. In this case, reloading the myapp systemd service sends a SIGHUP signal to the application, instructing it to gracefully close its old file descriptors and open the newly created log files.
The sharedscripts directive ensures that if multiple files (like debug.log and error.log) are rotated simultaneously, the postrotate script is only executed once, preventing the application from restarting multiple times.
Step 5: Testing the Configuration
Before relying on your new configuration, you must test it to ensure there are no syntax errors.
You can run logrotate manually in debug mode using the -d flag. This simulates the rotation process and prints exactly what it would do, without actually modifying any files.
sudo logrotate -d /etc/logrotate.d/myappIf you want to force an immediate rotation right now (ignoring the daily schedule), use the -f (force) flag:
sudo logrotate -f /etc/logrotate.d/myappConclusion
Proper log management is a fundamental pillar of Linux system administration. By mastering the logrotate utility and writing precise configuration directives, administrators ensure that critical debug data is retained and compressed, while completely eliminating the risk of unmonitored logs crashing production Ubuntu servers.