How to Configure logrotate in Linux for Automated Log Rotation, Compression, and Retention Policies

Modern Linux servers and containerised application clusters generate gigabytes of unstructured operational logs daily. Without systematic rotation and archiving policies, active services continuously append to open file descriptors until root partition storage exhausts, triggering kernel panics, failed database transactions, and unrecoverable service crashes. The logrotate daemon is the standard Linux administration subsystem designed to automate log file rotation, compression, truncation, and deletion based on temporal intervals or explicit file size thresholds.

Architectural Overview: The logrotate Execution Model, Inodes, and File Descriptors

Understanding how Linux processes handle open file descriptors versus filesystem directory pointers explains why improper rotation strategies break active logging daemons.

Rotation Directive	Kernel Mechanism	File Descriptor Behaviour	Recommended Application Scope
Default (Rename & Reopen)	Atomically renames the active log file to an archival suffix (e.g., app.log.1) and creates a new empty inode.	Processes retaining an open file descriptor continue writing to the renamed file until sent a signal (e.g., SIGHUP via postrotate).	Daemons with native reload capabilities (Nginx, Apache, Rsyslog, BIND).
copytruncate	Copies the live content to a backup file and truncates the original file in place to zero bytes (truncate -s 0).	Maintains the original inode and open file descriptor intact without signaling the process.	Legacy proprietary software, Node.js or Java daemons incapable of catching SIGHUP or reopening log paths.
delaycompress	Defers gzip/xz compression of the rotated file until the next subsequent rotation cycle.	Prevents compression corruption while slow daemons flush remaining buffered write buffers.	High-throughput web servers and multi-threaded logging relays.
sharedscripts	Aggregates wildcard file matches into a single execution of the postrotate script block.	Prevents sending multiple duplicate reload signals to a single parent daemon process.	Directories hosting multiple virtual host logs (e.g., /var/log/nginx/*.log).

Global vs. Modular Configuration Hierarchy

The logrotate subsystem reads configurations hierarchically:

• Global Defaults (/etc/logrotate.conf): Sets system-wide fallbacks (rotation cadence, retention counts, default compression algorithms).
• Modular Drop-In Directory (/etc/logrotate.d/): Contains standalone service-specific configuration files (e.g., /etc/logrotate.d/nginx, /etc/logrotate.d/rsyslog) that override global directives for targeted log paths.
• Invocation Framework: Managed automatically by a daily systemd timer (logrotate.timer) or legacy cron job (/etc/cron.daily/logrotate).

Configuring an Enterprise Custom logrotate Profile

To establish deterministic log governance for a custom production service (e.g., /var/log/custom-app/*.log), create a dedicated modular profile.

1. Creating the Configuration File

Create a new configuration file in the modular drop-in directory:

sudo nano /etc/logrotate.d/custom-app

2. Defining Directives and Script Hooks

Insert the following production-hardened configuration block:

/var/log/custom-app/*.log {
    daily
    rotate 14
    missingok
    notifempty
    compress
    delaycompress
    compresscmd /usr/bin/gzip
    compressext .gz
    dateext
    dateformat -%Y%m%d
    maxsize 500M
    create 0640 customapp adm
    sharedscripts
    postrotate
        if [ -f /var/run/custom-app/app.pid ]; then
            kill -USR1 $(cat /var/run/custom-app/app.pid) > /dev/null 2>&1 || true
        fi
    endscript
}

3. Configuration Directives Breakdown

• daily / maxsize 500M: Rotates daily, but triggers immediately if any log exceeds 500 MB before the daily cron runs.
• rotate 14: Retains exactly 14 rotated archives before permanently deleting the oldest.
• missingok: Suppresses error exceptions if the specified log file does not exist.
• notifempty: Skips rotation if the log file contains zero bytes, preventing empty archive proliferation.
• dateext / dateformat: Appends ISO timestamp suffixes (e.g., app.log-20261023.gz) instead of sequential integers (.1, .2) for automated ingestion into SIEM pipelines.
• create 0640: Automatically applies strict POSIX permissions and user/group ownership to newly spawned log files.

Dry-Run Validation and Manual Execution

Never wait for the daily scheduled job to discover syntax errors. Validate configurations proactively using command-line diagnostic flags.

1. Executing a Dry-Run Simulation

Use the -d (debug/dry-run) flag to simulate rotation without modifying files on disk or signaling services:

sudo logrotate -d /etc/logrotate.d/custom-app

The console prints parsing decisions, pattern matches, and whether size or temporal triggers have been met:

reading config file /etc/logrotate.d/custom-app
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries

Handling 1 logs
rotating pattern: /var/log/custom-app/*.log  after 1 days (14 rotations)
empty log files are not rotated, log files >= 524288000 are rotated earlier
considering log /var/log/custom-app/production.log
  log does not need rotating (log has been rotated at 2026-10-23 06:25:01)

2. Forcing Immediate Execution

To force rotation immediately regardless of file age or state records (useful for verifying postrotate scripts and testing disk space recovery):

sudo logrotate -f -v /etc/logrotate.d/custom-app

Root-Cause Troubleshooting Matrix

When logs stop rotating, disk space leaks, or permissions fail, consult the diagnostic guide below.

Observed Issue	Underlying Cause	Remediation Action
Logs rotate but daemon continues writing to old file	Daemon requires a reload signal (e.g., HUP or USR1) to close the old file descriptor.	Add a postrotate script sending the appropriate reload signal, or switch to the copytruncate directive if the application cannot catch signals.
“Insecure permissions on config file”	Configuration file in /etc/logrotate.d/ is world-writable or owned by a non-root user.	Enforce root ownership and read-only permissions: sudo chown root:root /etc/logrotate.d/* && sudo chmod 0644 /etc/logrotate.d/*.
copytruncate causes brief log line loss	High-concurrency writes occurring in the microsecond window between file copy and truncation.	Refactor the application daemon to catch SIGUSR1 and reopen files natively; avoid copytruncate for high-volume database or API endpoints.
Rotated logs uncompressed on day one	delaycompress intentionally defers compression until the subsequent rotation cycle.	Expected architectural behaviour; prevents corruption from delayed daemon writes. Remove delaycompress only if daemon reload is strictly synchronous.
Daily rotation skipped entirely	The systemd logrotate.timer is masked, inactive, or failed.	Audit timer state: systemctl status logrotate.timer; enable and start the timer: sudo systemctl enable --now logrotate.timer.

Operational Best Practices for High-Volume Storage

To maintain storage reliability and security compliance across Linux server clusters:

• Offload Compressed Archives to Object Storage: Implement a secondary postrotate script or cron job that syncs *.gz archives older than 7 days to an Amazon S3 or Ceph bucket before deleting local files.
• Utilise Parallel Compression (pigz): For log files exceeding several gigabytes, replace standard single-threaded gzip with parallel implementation: specify compresscmd /usr/bin/pigz in your profile.
• Isolate Log Mountpoints: Mount /var/log on a dedicated logical volume (LVM) partition. This guarantees that unexpected log surges will never saturate the root filesystem (/).