...A place where sharing IT monitoring knowledges

Friday, 2 December 2011

Managing Nagios logs

Nagios natively supports log rotation, a functionality managed using log_rotation main configuration option. This is the configuration option description taken from official nagios documentation

Format: log_rotation_method=[n/h/d/w/m]
Example: log_rotation_method=d


This is the rotation method that you would like Nagios to use for your log file. Values are as follows:

n = None (don't rotate the log - this is the default)
h = Hourly (rotate the log at the top of each hour)
d = Daily (rotate the log at midnight each day)
w = Weekly (rotate the log at midnight on Saturday)
m = Monthly (rotate the log at midnight on the last day of the month)

Many times people become confused by the Nagios log management capabilities and believe that, besides rotating, Nagios will erase older files too... or well, some angel in our system will do for us. Saddly this is not real neither in Nagios nor in Centreon systems and older logs remain in our disk for months or even years.

This simple script can be very helpful in order to address the previous fact. It manages log files in two combinable ways: Compressing and or deleting files older than x days. It takes three arguments: 

  • Directory where nagios logs are stored
  • Age, in days, for files that will be compressed
  • Age, in days, for files that will be deleted
For instance, and given that it is named as manage_naglogs, this example would delete files older than 30 days and would compress files older than 7 days:

manage_naglogs /var/log/nagios 7 30

And here comes the script:


#!/bin/bash

if [ $3 -gt 0 ]
then
        find $1/nagios-*.gz -mtime +$3 -exec rm {} \;
        find $1/nagios-*.log -mtime +$3 -exec rm {} \;
fi
if [ $2 -gt 0 ]
then
        find $1/nagios-*.log -mtime +$2 -exec gzip {} \;
fi

In order to run it periodically, I recommend adding the needed commands to cron. In systems like Debian where /etc/cron.daily stores scripts run every day, and assuming you have saved the previous script in /usr/local/nagios/bin, create an script like this, save it in /etc/cron.daily and set proper file permissions for being run for cron daemon (chmod 755 will do the job):

#!/bin/bash
/usr/local/nagios/bin/manage_naglogs /var/nagios/logs 7 30

In systems where only crontab is available, next entry will do the job. It will run our script once every day at 3:00am:

00 3 * * * root /usr/local/nagios/bin/manage_naglogs /var/nagios/log 7 30

Finally one advice for those using Centreon: Keep, at least, one nagios rotated log file untouched (ie, neither compressed nor deleted). Have in mind that centreon run every day (usually at 1:00am) an script for parsing Nagios log files in order to create availability reports. To achieve it, use values higher than 1 for the second and third script arguments.

Last but not least...

If you found this article useful, please leave your comments and support the site by clicking in some (or even in all!) of the interesting advertisements of our sponsors. Thanks in advance!


7 comments:

  1. Thanks for posting this. I tweaked the script slightly to make tarballs instead, and remove the source logs:

    #!/bin/bash

    if [ $3 -gt 0 ]
    then
    find $1/nagios-*.tar.gz -mtime +$3 -exec rm {} \;
    find $1/nagios-*.log -mtime +$3 -exec rm {} \;
    fi
    if [ $2 -gt 0 ]
    then
    find $1/nagios-*.log -mtime +$2 | while read file; do
    tar czvf $(basename $file).tar.gz $file
    rm -f $file
    done
    fi

    ReplyDelete
    Replies
    1. Thanks Steve, your solution is very useful when rotating logs hourly, since you can tar all the logs from a day in a single file (instead, my solution would zip each hourly file).

      Again, very useful. Thanks.

      Delete
  2. I'm having the opposite problem: log are being overwritten daily. The log_rotation was set to daily and the logs were not being saved to /archives directory. Is there anyway I can verify the files are being archived?

    ReplyDelete
    Replies
    1. Hi Jose:

      Maybe your log_archive_path Nagios configuration directive is not well defined and your rotated logs are going to an unknown place. It might be defined as, for instance:

      log_archive_path=/var/log/nagios/archives

      You must check too that the directory was created and that Nagios user (usually 'nagios') had write permission on it.

      Delete
  3. I have a new issue with nagios log. This issue has appeared without any change in nagios configuration. Nagios is archiving the same log file in /usr/local/nagios/var/archives last four days. Nagios user write a log file but always are the same... What can I do??

    ReplyDelete
    Replies
    1. I think that my issues are related with USA time change of last saturday. Nagios are archiving the log of the current day, nagios is archiving a log with only one hour records...

      Delete

 
Design by Free WordPress Themes | Bloggerized by Lasantha - Premium Blogger Themes