Page 1 of 1
Delayed alert notifications??
Posted: Mon Aug 05, 2013 2:07 pm
by kane3030
Hello,
I am having a problem where, intermittently, alerts are being detected but sending out hours later. For example:
Notification Type: PROBLEM
Service: ****
Host: ****
Address: ****
State: CRITICAL
Date/Time: Fri Aug 2 18:21:20 EDT 2013
Additional Info:
Aug 2 13:24:15
The issue happened at 13:24 but was first being sent as a notification at 18:21.
Re: Delayed alert notifications??
Posted: Mon Aug 05, 2013 3:34 pm
by abrist
This sounds like a timezone or date/time issue. What is the output of:
Code: Select all
grep date.timezone /etc/php.ini
date
Re: Delayed alert notifications??
Posted: Mon Aug 05, 2013 3:36 pm
by kane3030
I will paste the results of that shortly but the issue is very intermittent. It works most of the time and will randomly send out alerts late. If you look at what I sent, its a five hour difference but its not always five hours, its some random time.
Re: Delayed alert notifications??
Posted: Mon Aug 05, 2013 3:42 pm
by abrist
Are these problems happening for all checks, or a specific subset? Checks like snmp traps, if there are dns issues, may stack up and come in late.
Re: Delayed alert notifications??
Posted: Tue Aug 06, 2013 2:43 pm
by kane3030
This is happening for all checks, regardless of host. Just happened again for an alert:
Date/Time: Tue Aug 6 15:41:16 EDT 2013
Additional Info:
Aug 5 13:17:14
Re: Delayed alert notifications??
Posted: Tue Aug 06, 2013 2:48 pm
by sreinhardt
Let's check the snmptt spool.
Re: Delayed alert notifications??
Posted: Mon Aug 12, 2013 9:35 am
by kane3030
I actually don't have that under /var/spool. I didn't originally set up this Nagios installation but now I am the admin of it. I restarted snmpd this morning in hopes that it may be the cause but this has been causing a lot of stress for my co workers.
Re: Delayed alert notifications??
Posted: Mon Aug 12, 2013 10:13 am
by abrist
Lets us know. We still have not heard from you concerning the server date, nor the current request for your snmptt spool directory listing.
Re: Delayed alert notifications??
Posted: Tue Aug 13, 2013 1:22 pm
by kane3030
After trying many different things, it appears restarting the snmpd process worked. Thanks for all of your suggestions!
Re: Delayed alert notifications??
Posted: Tue Aug 13, 2013 1:33 pm
by abrist
The snmptt spooler may have been backed up. Causing some results to not be checked until well after they were received, effectively sending out late alerts. I have seen it before. Let us know if this recurs.