Page 2 of 2
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:19 am
by WillemDH
Steve,
The notifications are not stopping. As said in previous post we even receive notfications of the services which are already ok. We are 100 %s sure the checks are no longer running. It's the default alert interval of 30 minutes on the services we can no longer find that is the problem.
Thanks for the help.
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:22 am
by WillemDH
I stopped the postgresql and crond. My colleague Michiel had a nice word for our problem. "Orphaned notifications".
Didn't see any new mails coming in since I stopped crond and postgresql. Gonna wait for 2 more minutes to be sure.
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:24 am
by tmcdonald
Emails in XI use at least postgres and I believe cron at least once somewhere along the line, so there is probably something not getting updated or cleared to say the email has been sent. Let's see if they come back after the restart.
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:29 am
by DigNetwerk
You are correct tmdonald. We didn't get any emails while crond and postgres were shutdown.
EDIT: I'm am Willem's colleague
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:34 am
by DigNetwerk
We restarted crond and postgres and now we are getting emails again (from the non-existent services).
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:39 am
by WillemDH
/etc/cron.d/nagiosxi contains:
Code: Select all
# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi
# Backup MySQL & PostgreSQL Databases
0 7 * * * root /root/scripts/automysqlbackup
0 8 * * * root /root/scripts/autopostgresqlbackup
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01 * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php > /usr/local/nagiosxi/var/deadpool.log 2>&1
How can we remove these orphaned notifications? Should I make a support ticket for this?
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 10:58 am
by DigNetwerk
Hello,
I've look around some more and found out that there is nothing regarding notifications in the Postgres db, but there is in the Mysql db (logged on as ndoutils in the nagios database).
There is a notifications table, where there are exactly as many notifications regarding "traceroute" as we have hosts in that hostgroup (x2 for 2 services):
Code: Select all
SELECT * FROM nagios_notifications WHERE lower(output) LIKE '%traceroute%';
Can we just do
Code: Select all
DELETE FROM nagios_notifications WHERE lower(output) LIKE '%traceroute%';
Or are there other dependencies that we should remove as well?
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 11:34 am
by WillemDH
Sent an email to
[email protected] to create a support ticket for this. Did not receive any confirmation though. Did you guys get my email? I(m asking as I had some issues with my changed work email address in the past)
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 11:37 am
by tmcdonald
We got it, however you won't receive any automated response until one of our techs has assigned it to themselves (or I assign it to someone). Since it's in email, mind if I lock this up?
Re: URGENT - Weird issues after error in config
Posted: Tue Oct 27, 2015 11:38 am
by WillemDH
Since it's in email, mind if I lock this up?
Sure