Page 1 of 1

Alert triggered during downtime

Posted: Tue Apr 07, 2020 1:15 pm
by manimurugesan
Hello Team,

We had scheduled down time on 5th April 2020 Sunday from 05-Apr-20 08:00:00 to 05-Apr-20 12:00:00..

when the server was under scheduled downtime i can see the alerts in state history and we enabled snmp trap sender as well it will send the traps to ticketing tool called service now .

I can see downtime started 7:59 in notification window but in state history alerts are started triggering 8:01 but alert was not suppressed and it sent to ticketing tool as well we got all the connection refused alerts .but downtime started at 7:59

i have attached nagiosxi screen shot.

Could you please clarify the same and why it's not suppressed

Re: Alert triggered during downtime

Posted: Tue Apr 07, 2020 4:50 pm
by benjaminsmith
Hi,

Thanks for the screenshots. When you scheduled the downtime for the server, did you put the services in downtime as well? There's an option for the services.
scheduled-downtime.png
If that's not the issue, let's make sure the time settings are sync correctly on the server. You can run the following commands to verify the server, php timezone settings are correct.

Code: Select all

date
ls -l /etc/localtime
php -r 'echo date("D M j G:i:s T Y")."\n";'
grep "date.timezone =" /etc/php.ini
See: Changing The System Time

Lastly, if the time settings are correct, you may have multiple Nagios processes running. Let's do a re-start of the server to eliminate that possibility. Please test this again by scheduling downtime and forcing one of the services into a Critical State by sending passive check results. Let me know if you need assistance with that.

Code: Select all

systemctl stop crond
systemctl stop npcd
systemctl stop nagios
systemctl stop ndo2db
pkill -9 -u nagios
for i in $(ipcs -q | grep nagios |awk '{print $2}'); do ipcrm -q $i; done
rm -rf /usr/local/nagiosxi/var/dbmaint.lock
rm -rf /usr/local/nagiosxi/var/event_handler.lock
rm -rf /usr/local/nagiosxi/scripts/reconfigure_nagios.lock
systemctl restart mariadb
systemctl start ndo2db
systemctl start nagios
systemctl start npcd
systemctl start crond