Log Rotation for Self-Hosted Apps

The Disk Full Disaster

It’s been six months. Your app is quietly logging every request, every error, every debug message. That log file started at 1MB. Now it’s 50GB. Your disk has 2GB free. Everything’s about to grind to a halt.

Your database can’t write. Your app crashes because it can’t write logs (too funny to be sad). Everything goes down because nobody rotated the logs.

This is stupid to let happen. And also stupid-easy to fix.

What Log Rotation Does

Log rotation is simple: it renames the current log file, starts a fresh one, and archives old logs. Your app doesn’t need to restart. You just tell the OS “hey, this file is getting big, handle it.”

Standard Linux tools handle this. You don’t need anything fancy.

Using logrotate (The Standard Way)

Most systems have logrotate. It runs from cron and handles rotation automatically.

First, find where your app is logging. Let’s say it’s /var/log/myapp.log.

Create a rotation rule:

/var/log/myapp.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root root
}

Breaking this down:

• daily — rotate every day
• rotate 14 — keep 14 old logs, delete older ones
• compress — gzip old logs (saves space)
• delaycompress — don’t compress yesterday’s log yet (app might still be reading it)
• missingok — don’t freak out if the log doesn’t exist
• notifempty — don’t rotate if the log is empty
• create 0640 root root — when you create a new log, this is the permissions

That’s it. Logrotate handles the rest.

Test it:

logrotate -d /etc/logrotate.d/myapp

The -d flag is “dry run” — it shows you what it would do without actually doing it.

Force a rotation:

logrotate -f /etc/logrotate.d/myapp

Now check your log directory:

ls -lh /var/log/myapp.log*

You should see:

-rw-r----- 1 root root  145K Nov 29 14:22 myapp.log
-rw-r----- 1 root root  2.1M Nov 28 00:00 myapp.log-20251128.gz
-rw-r----- 1 root root  1.9M Nov 27 00:00 myapp.log-20251127.gz

Size-Based Rotation

Maybe daily is overkill. You want rotation when the log hits 100MB:

/var/log/myapp.log {
    size 100M
    rotate 10
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root root
}

Same format, just swap daily for size 100M.

Docker Containers

If your app is containerized, you might not have /etc/logrotate.d/ on the container. You’ve got two options:

Option 1: Let Docker handle it

Docker has log rotation built in. Use the json-file driver with limits:

docker run -d \
  --log-driver json-file \
  --log-opt max-size=100m \
  --log-opt max-file=5 \
  myapp:latest

Docker automatically rotates when the log hits 100MB, keeping 5 old logs. Old ones get compressed and archived.

Option 2: Rotation inside the container

If you’re using Docker Compose:

version: '3.8'
services:
  myapp:
    image: myapp:latest
    logging:
      driver: json-file
      options:
        max-size: "100m"
        max-file: "5"

Same effect.

Direct App Logging Control

Some apps (Node.js, Python, etc.) can handle rotation themselves. Don’t reinvent the wheel — use a library.

Node.js with winston:

const winston = require('winston');
require('winston-daily-rotate-file');

const logger = winston.createLogger({
  transports: [
    new winston.transports.DailyRotateFile({
      filename: '/var/log/myapp-%DATE%.log',
      datePattern: 'YYYY-MM-DD',
      maxSize: '100m',
      maxDays: '14d',
    }),
  ],
});

logger.info('App started');

Python with RotatingFileHandler:

import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    '/var/log/myapp.log',
    maxBytes=100 * 1024 * 1024,  # 100MB
    backupCount=10
)
logger = logging.getLogger()
logger.addHandler(handler)

logger.info('App started')

The app handles rotation on its own, without needing logrotate.

Monitor Your Disk

Don’t wait until your disk is full. Check it regularly:

df -h /var/log

If you’re over 80%, start rotating more aggressively.

Add a cron job to alert you:

0 * * * * root df /var/log | tail -1 | awk '{print $5}' | sed 's/%//' | awk '$1>80 {print "ALERT: /var/log is " $1 "%"}' | mail -s "Disk Alert" root

Or use lsof to find what’s filling disk:

sudo lsof | grep -i deleted | sort -k 7 -rn | head -10

This shows files that are deleted but still open (common with log files).

The Nuclear Option: Clean Old Logs

If you’re already in trouble, you can manually remove old compressed logs:

# Remove logs older than 30 days
find /var/log -name "myapp.log-*.gz" -mtime +30 -delete

Be careful — you might want those logs.

Prevention Checklist

Before your disk fills:

• Set up logrotate or container log rotation
• Test it once (don’t just assume it works)
• Set rotation size/frequency based on your disk space
• Compress old logs
• Monitor disk usage monthly
• Archive logs somewhere if you need long-term storage

Your 3 AM self will appreciate the 10 minutes you spend on this now.