fhxd.de

Your logrotate config is probably doing nothing

2026-07-19

The default logrotate strategy is to rename(2) the log file and then create a fresh one in its place. This works because most well-behaved daemons either accept a SIGHUP and reopen their log, or use a library that checks the inode. The rotation config carries a postrotate block to send that signal.

Plenty of software does neither. It opens the file once at startup, keeps the file descriptor forever, and writes to it with O_APPEND. Rename the file out from under it and nothing breaks — the descriptor still points at the same inode, now living at a new path. The daemon keeps writing to app.log.1. The newly created app.log stays at zero bytes forever.

The failure is completely silent. logrotate exits 0. The directory listing looks healthy — you get your seven rotated files on schedule. It is only when you go looking for last Tuesday's error that you notice six of the seven are empty and the seventh is 4 GB.

How to tell

Ask the kernel which file the process actually has open. If the target has (deleted) after it, or points at a rotated name, you have the bug:

ls -l /proc/$(pgrep -f myapp)/fd | grep '\.log'

The other tell is size distribution. A healthy rotation set has roughly comparable file sizes. A broken one has one enormous file and a pile of zero-byte ones.

The fix

Use copytruncate. Instead of renaming, logrotate copies the contents to the rotated name and then truncates the original file in place. The inode never changes, so the daemon's file descriptor stays valid and it keeps writing to the same, now-empty, file.

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
    su myapp myapp
}

Two notes on that block. delaycompress matters more than it looks: with copytruncate there is a brief window between the copy and the truncate, and compressing immediately can catch a partially-written line. Delaying compression by one cycle avoids it.

And su is not optional once the daemon runs as an unprivileged user. logrotate refuses to rotate a directory it does not own unless you tell it which identity to assume — another failure that is quiet, because it shows up in the logrotate log rather than anywhere you look.

The tradeoff

copytruncate is not free. Any line written between the copy and the truncate is lost. For an access log at a few hundred lines per second, that is a handful of records per day, which is usually acceptable. For an audit log where every record matters, it is not — there you need to fix the daemon or put a real log shipper in front of it.

Check rotation the same way you check backups: not by confirming the job ran, but by opening the output and finding data in it.