Maintaining log files is an important part of keeping computer systems running smoothly. Log files provide a history of what has happened on your computer, and provide information on what errors occurred. Linux systems logs are normally stored in the /var/log directory. But the log files must be maintained, both to organize the data, and to prevent them from filling up all of the file space.
This example deals with data logs generated by Xastir, a ham radio packet data program. In Xastir, the log files provide a history of what data has passed through the system, and through which interfaces.
Linux systems provide tools to organize the log files, and automatically delete them after a specified period. However, different Linux distributions provide different tools to handle the job. The first section of this page provides information on handling log files on a SUSE distribution, while the second section describe how to setup log file maintenance on Debian based systems, including Ubuntu and Raspian distributions.
For SUSE distributions:
NOTE: Most if not all of the log files for Xastir now auto-rollover at a certain filesize, keeping several older versions of each before they get deleted. The information on this Wiki page should still be applicable to other log files on your Linux system however.
Crond daemon must be running on your system.
Here are detailed instructions for SuSE systems for auto-rollover of log files:
Edit /etc/logfiles to add the four logfiles for each user:
#xastir /home/archer/.xastir/logs/net.log +1024k 644 archer.users /home/archer/.xastir/logs/tnc.log +1024k 644 archer.users /home/archer/.xastir/logs/wx.log +1024k 644 archer.users /home/archer/.xastir/logs/igate.log +1024k 644 archer.users /home/hacker/.xastir/logs/net.log +1024k 644 hacker.users /home/hacker/.xastir/logs/tnc.log +1024k 644 hacker.users /home/hacker/.xastir/logs/wx.log +1024k 644 hacker.users /home/hacker/.xastir/logs/igate.log +1024k 644 hacker.users
Edit /etc/rc.config, change MAX_DAYS_FOR_LOG_FILES to something reasonable and uncomment the line. I set mine for 5 days. Note that this limit will affect ALL logfiles listed in /etc/logfiles:
#MAX_DAYS_FOR_LOG_FILES=365 MAX_DAYS_FOR_LOG_FILES=5
For Debian derived distributions, including Ubuntu and the Raspberry Pi:
This documentation was developed on a Raspberry PI 2, running Raspian Jessie. It should be accurate for Debian and Ubuntu systems, as well. You can find more information on the web, or by running the /man logrotate/ command.
Cron is a daemon program that is included on most Unix/Linux operating systems that runs jobs on a time based schedule. However under Raspian, cron is not set to start automatically when the Pi powers up. We'll change that in the next few steps.
First we need to check to see if cron is running. At the command line type:
$ sudo service --status-all | grep cron
This will list all services with cron in the name, and display their current status. On my Pi, it produced this result:
pi@deimosraspi:~ $ sudo service --status-all | grep cron [ - ] anacron [ ? ] aprsd [ + ] cron
Since cron appears with a +, we know that it is set to start automatically when the system is rebooted. If cron appears with a -, we need to start it. At the command line, type:
$ sudo service cron start
It's probably a good idea to reboot your Pi now, and recheck that cron starts when the system is rebooted.
Raspian Linux, and other Debian derived Linux distributions, run a program called logrotate. Logrotate can use a default schedule for all log files, or it can use custom settings for individual and groups of log files. In addition, log files can be set to automatically rotate based on time, (days, weeks, months), based on a minimum and/or maximum file size. Rotated log files can be compressed, retained for a certain time period and then be deleted by the system. More details can be found with the /man logrotate /command.
The default setup for logrotate is in /etc/logrotate.conf. Alternatively, you can create a separate logrotate configuration for individual applications in the /etc/logrotate.d directory.
Log files are renamed in the form /filename.x /and /filename.x.gz /where x is the version number, and .gz indicatesa compressed file. The oldest file is deleted when the maximum number of copies is reached, and the x value is incremented for each of the archived copies. If the /dateext/option is used, the file hasthe date the log file is rotated added to the file name eg. /filenameYYYYMMDD/. These files are not renamed, but simply allowed to age, and the system deletes the old files after the specified retention period.
The logrotate file I created for Xastir is /etc/logrotate.d/xastir, and contains the following rules:
/home/pi/.xastir/logs/igate.log
/home/pi/.xastir/logs/message.log
/home/pi/.xastir/logs/net.log
/home/pi/.xastir/logs/tnc.log
/home/pi/.xastir/logs/wx.log
/home/pi/.xastir/logs/wx_alert.log
{
rotate 7
daily
minsize 500k
maxsize 2G
compress
dateext
delaycompress
copytruncate
missingok
notifempty
}
The first section lists the log files to be rotated. The section below the file names and defined by the { and } are the rules that apply to all of these files. The rules break down as follows:
rotate 7 - Retain 7 copies log files
weekly - Rotate the log files weekly. Other options are daily and monthly
minsize 100k - Don't rotate the log file unless it is at least 100kB in size, regardless of how many days it has been attached
maxsize 2G - Rotate the log file if it exceedes2 GB in size, even if the log file hasn't been attached for the period specified
compress - Compress the log file when it is rotated, with gzip.
Dateext - Append the date to the logfile in the format YYMMDD
delaycompress - Don't compress the log file until the next time it is rotated
copytruncate - This option copies the log file to a backup, and truncates the original file. This is useful when processes are still writing information to an open log file.
missingok - Don't issue an error if the log file is missing, continue processing the command
notifempty - Don't rotate the log file if it is empty