Page 1 of 2
Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:20 pm
by nseltzer
Hey all,
There is a strange quirk where /var/log/messages is doubling all Nagios-related entries.
Code: Select all
Feb 6 12:14:15 papmonmp10 nagios: SERVICE ALERT: hqsadbp0;SNMP - CPU Usage;OK;HARD;5;8 CPU, average load 28.1% < 95% : OK
Feb 6 12:14:15 papmonmp10 nagios: SERVICE ALERT: hqsadbp0;SNMP - CPU Usage;OK;HARD;5;8 CPU, average load 28.1% < 95% : OK
Feb 6 12:15:51 papmonmp10 nagios: SERVICE ALERT: HQSADBT0;SNMP - Process - NTPD;OK;HARD;5;1 process matching ntpd (> 0)
Feb 6 12:15:51 papmonmp10 nagios: SERVICE ALERT: HQSADBT0;SNMP - Process - NTPD;OK;HARD;5;1 process matching ntpd (> 0)
I've exhausted my Google-fu and have perused the forums to no avail.
Thanks,
Nic Seltzer
System Info:
Code: Select all
$ uname -a
Linux papmonmp10.cabelas.corp 2.6.18-308.24.1.el5 #1 SMP Wed Nov 21 11:42:14 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
$cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.8 (Tikanga)
Manual or VM install? Manual XI install
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:38 pm
by lmiltchev
Can you run the following command and post the output?
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:41 pm
by nseltzer
Code: Select all
$ sudo ps -ef | grep /bin/nagios | grep -v grep
nagios 5406 1 3 10:25 ? 00:04:55 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
Nic
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:41 pm
by slansing
I believe this is something that Core does by default I will check into it though.
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:45 pm
by lmiltchev
Open the nagios.cfg file in a text editor and change this line:
to this
Then restart nagios service:
See if this is going to remove the double entries.
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 2:53 pm
by nseltzer
It was successful in one respect. No new Nagios-related entries were being populated in to /var/log/messages. Any ideas?
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 3:14 pm
by lmiltchev
I'm sorry about that - I guess this is not what you really want (to disable syslog facility altogether). You can change it back:
Code: Select all
use_syslog=1
service nagios restart
Do you see the same messages in the nagios log?
Code: Select all
tail /usr/local/nagios/var/nagios.log
Are these active checks are snmp traps?
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 3:19 pm
by nseltzer
We're only seeing errors in nagios.log. Everything nagios-related is being duplicated in /var/log/messages, for example:
Code: Select all
Feb 6 12:54:18 papmonmp10 nagios: Warning: Host 'shqdbadm02' has no default contacts or contactgroups defined!
Feb 6 12:54:18 papmonmp10 nagios: Warning: Host 'shqdbadm02' has no default contacts or contactgroups defined!
Feb 6 12:54:18 papmonmp10 nagios: Finished daemonizing... (New PID=6605)
Feb 6 12:54:18 papmonmp10 nagios: Finished daemonizing... (New PID=6605)
At this time, I don't believe we have any snmptraps going to this host, so all check statuses that we're seeing would be from active hosts.
Nic
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 3:34 pm
by lmiltchev
Are the errors in the nagios.log also duplicated (like the ones in the /var/log/messages)?
Re: Doubled Entries in /var/log/messages
Posted: Wed Feb 06, 2013 3:36 pm
by nseltzer
Nope. They appear to be groovy.
Code: Select all
[1321647824] Warning: Return code of 127 for check of service 'Current Load' on host 'localhost' was out of bounds. Make sure the plugin you're trying to run actually exists.
[1321647884] Warning: Return code of 127 for check of service 'Current Users' on host 'localhost' was out of bounds. Make sure the plugin you're trying to run actually exists.
[1321647944] Warning: Return code of 127 for check of service 'PING' on host 'localhost' was out of bounds. Make sure the plugin you're trying to run actually exists.
[1321648002] Caught SIGTERM, shutting down...
[1321648002] Successfully shutdown... (PID=3136)
Nic