search by tags

for the user

adventures into the land of the command line

logrotate - boring but useful

for a while i worked on a project where everyone forgot/didn’t care about application logs. some applications had single 13GB log files that had never been rotated. some applications did rotate the logs but never removed them. these application would routinely crash because the filesystem was 100% used.
i set myself the task of cleaning up this absurdity, and in the process i learned a bit about the logrotate utility
logrotate is very easy to use and understand. its job is to manage what happens to a log file after it meets some rule that you define

so how is logrotate run on your system?

The system crontab runs the daily cron “cron.daily” every day at X:XXAM/PM (depending on your system), from this file

/etc/crontab -> /etc/cron.daily

cron.daily executes the commands in files inside

/etc/logrotate.conf

logrotate.conf runs (among other things) configs in

/etc/logrotate.d/someapplicationspecificfile

by way of the command

include /etc/logrotate.d

the logrotate version i like to use is 3.8.7, as it includes 2 directives:

datetime
Specify the extension for dateext using the notation similar to strftime(3) function. Only %Y %m %d and %s specifiers are allowed. The default value is -%Y%m%d.

maxsize
Log files are rotated when they grow bigger than size bytes even before the additionally specified time interval (daily, weekly, monthly, or yearly).

then its just a matter of creating a logrotate config inside

/etc/logrotate.d/example

this is an example

/var/log/example/error_log.log {
    daily
    maxsize 10M
    compress
    copytruncate
    dateext
    dateformat -%Y-%m-%d-%s
    notifempty
    missingok
    rotate 11
    prerotate
        if [ "`find /path/to/log.log-* -type f | wc -l`" -ge "10" ]; then
            mv /path/to/log.log-* /path/to/archived/;
        fi
        if [ "`find /path/to/archived/ -type f -mtime +90`" ]; then
            find /path/to/archived/ -type f -mtime +90 -delete;
        fi
    endscript
}

the config needs to point to the logfile which you want to manage, and include a list of rules you want logrotate to apply to the logfile

you can test a logrotate config with the -d command

$ /usr/sbin/logrotate -d /etc/logrotate.d/example

this will show you what logrotate is going to do, without actually doing it, so you can make sure it works before it is run by cron. this example will do this

rotating pattern: /var/log/example/error_log.log forced from command line (11 rotations)
empty log files are not rotated, log files >= 1024 are rotated earlier, old logs are removed
considering log /var/log/example/error_log.log
log needs rotating
rotating log /var/log/example/error_log.log, log->rotateCount is 6
Converted ' -%Y-%m-%d-%s' -> '-%Y-%m-%d-%s'
dateext suffix '-2014-08-20-1408521725'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
running prerotate script
running script with arg /var/log/test/error_log.log: "
if [ "`find /var/log/example/error_log.log-* -type f | wc -l`" -ge "10" ]; then
mv /var/log/test/error_log.log-* /var/log/archived/;
fi
if [ "`find /var/log/archived/ -type f -mtime +90`" ]; then
find /var/log/archived/ -type f -mtime +90 -delete;
fi
"
copying /var/log/example/error_log.log to /var/log/example/error_log.log-2014-08-20-1408521725
truncating /var/log/test/error_log.log
compressing log with: /bin/gzip

the prerotate commands are optional, and are for moving logfiles to shared storage mounts for archiving, and also for removing old archived logs on the storage mount. its possible for logrotate to move files to shared storage only if you reference a symlink to shared storage, otherwise it may not work. so in this example, a symlink must exist to a location on shared storage

$ ls -l /var/log/example/

lrwxrwxrwx. 1 root root     22 Jul 15 17:56 archived -> /mnt/log/
-rw-r--r--. 1 root root    324 Aug 27 14:41 error_log.log
-rw-r--r--. 1 root root    253 Aug 24 16:39 error_log.log-2015-08-24-1440399661.gz

we can change logrotate to be run hourly as well as daily. this is useful in scenarios where svt runs create massive log files in a few hours and logrotate’s maxsize directive won’t know until the daily cron runs. to get logrotate to run hourly

create a cron.hourly file inside

/etc/cron.hourly/logrotate

containing

#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.d/example
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi

cron.hourly will execute the specific logrotate config you want, every hour, making the maxsize directive more useful. for multiple configs, just specify them in the same file like this

#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.d/example
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi

/usr/sbin/logrotate /etc/logrotate.d/anotherexample
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi

/usr/sbin/logrotate /etc/logrotate.d/exampleagain
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi

one last important thing to note is that logrotate will only rotate logfiles that match the rotated format. if other files exist in the directory but their name’s are in a different format, logrotate’s glob pattern won’t return a match and skip over them. for instance, if the date format is

%Y-%m-%d-%s

of these two logfiles below, only the one that has a matching date format will be acted upon.

error_log.log-2015-08-24-1440399661.gz # will be rotated
error_log.log-2015-08-24.gz            # will be skipped

and thats all you really need to know about logrotate to use it effectively