Send nagios logs from one server to another

[snapon_admin]

Is there a way to automatically send Nagios logs from our prod server to our test server and vice versa? If not, is there documentation on how best to write Nagios XI log files to an exeternal syslog server? Currently Nagios log files are saved only on the Nagios server itself, which makes troubleshooting certain things somewhat difficult so we'd prefer to send important log files to a different server.

[scottwilkerson]

Is there a way to automatically send Nagios logs from our prod server to our test server and vice versa?

there is no automatic way to do this built-in

If not, is there documentation on how best to write Nagios XI log files to an exeternal syslog server?

You can setup a file in /etc/rsyslog.d like the example below to send to a syslog server

/etc/rsyslog.d/90-nagios_logs.conf

Code: Select all

$ModLoad imfile
$InputFilePollInterval 10
$PrivDropToGroup adm
$WorkDirectory /var/lib/rsyslog

# Input for nagios_logs
$InputFileName /usr/local/nagios/var/nagios.log
$InputFileTag nagios_logs:
$InputFileStateFile nls-state-nagios_logs
# Uncomment the folowing line to override the default severity for messages
# from this file.
#$InputFileSeverity info
$InputFilePersistStateInterval 20000
$InputRunFileMonitor

if $programname == 'nagios_logs' then @@YOUR_SYSLOG_SERVER:514
if $programname == 'nagios_logs' then ~

Then restart rsyslog

Code: Select all

service rsyslog restart

[snapon_admin]

What kind of file? And what logs will that send to our syslog server/how often are new log files generated and sent?

[scottwilkerson]

What kind of file? And what logs will that send to our syslog server/how often are new log files generated and sent?

Just a .conf file named /etc/rsyslog.d/90-nagios_logs.conf with the contents above (changing YOUR_SYSLOG_SERVER to the host you want to send the logs to)

It will send contents of /usr/local/nagios/var/nagios.log to the syslog server in near realtime

[snapon_admin]

Ok sounds good. Are there any other log files I should set this up for or does Nagios.log pretty much cover anything I'd need for troubleshooting most issues?

[scottwilkerson]

Ok sounds good. Are there any other log files I should set this up for or does Nagios.log pretty much cover anything I'd need for troubleshooting most issues?

That's kind of a loaded question, it defiantly doesn't cover everything needed for troubleshooting, but it is the most prevelant.

other logs to investigate would be those found under these 2 directories

Code: Select all

/usr/local/nagiosxi/var/
/usr/local/nagios/var/

[snapon_admin]

Yeah, good point. I know it's going to be a tough one to answer but, just generally speaking, what yould be the most helpful logs that you guys would need in order to accurately troubleshoot most issues? I know there will be some one offs here and there, but there's got to be a few that are just nearly always useful to have. Also, how would I add those to that file so that they're sent along with nagios.log to the syslog server?

[scottwilkerson]

All the .log files in /usr/local/nagiosxi/var can be important for us in troubleshooting. You would need to make a new file for each of them with a different filename changing the $InputFileName to the log path and the $InputFileStateFile to something unique per log

Are you looking for logs to send us if there is a problem? If that is the case, sending then to a syslog server is going to make them difficult to retrieve, it would be better in this case to make periodic profiles by running the following:

Code: Select all

/usr/local/nagiosxi/html/includes/components/profile/getprofile.sh

then copy the profile.zip from /usr/local/nagiosxi/html/includes/components

[snapon_admin]

Are you looking for logs to send us if there is a problem? If that is the case, sending then to a syslog server is going to make them difficult to retrieve, it would be better in this case to make periodic profiles

Yeah, that's exactly what I'm trying to do. The main reason I need this is because your support hours are the same as our business hours, which has caused issues with troubleshooting issues in the past. For example, we've been having many issues upgrading XI to anything beyond version 5.5. The upgrade itself takes 2 hours, and then shortly after the upgrade completes the server just stops running checks. It's like it gets overloaded and just can't keep up so it stops doing anything. Problem is that typically, by the time the upgrade finally completes and I can start gathering log data on the upgraded server we've already been down an unacceptable length of time so there's no time to trouble shoot anything. If I can log externally, I can at least hopefully get those log files to you without sitting on a production monitoring server that isn't monitoring anything. Usually after the upgrade is complete I cannot get a profile because when I try to generate one it just hangs. I can usually access individual logs, but again, I can't sit on that server not functioning for very long and by that poit it's already not been working for a minimum of 2 hours.

[scottwilkerson]

got it