URGENT - Weird issues after error in config

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: URGENT - Weird issues after error in config

Post by WillemDH »

Steve,

The notifications are not stopping. As said in previous post we even receive notfications of the services which are already ok. We are 100 %s sure the checks are no longer running. It's the default alert interval of 30 minutes on the services we can no longer find that is the problem.

Thanks for the help.
Nagios XI 5.8.1
https://outsideit.net
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: URGENT - Weird issues after error in config

Post by WillemDH »

I stopped the postgresql and crond. My colleague Michiel had a nice word for our problem. "Orphaned notifications".

Didn't see any new mails coming in since I stopped crond and postgresql. Gonna wait for 2 more minutes to be sure.
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: URGENT - Weird issues after error in config

Post by tmcdonald »

Emails in XI use at least postgres and I believe cron at least once somewhere along the line, so there is probably something not getting updated or cleared to say the email has been sent. Let's see if they come back after the restart.
Former Nagios employee
DigNetwerk
Posts: 40
Joined: Fri Oct 25, 2013 7:29 am

Re: URGENT - Weird issues after error in config

Post by DigNetwerk »

You are correct tmdonald. We didn't get any emails while crond and postgres were shutdown.

EDIT: I'm am Willem's colleague :)
Last edited by DigNetwerk on Tue Oct 27, 2015 10:37 am, edited 1 time in total.
DigNetwerk
Posts: 40
Joined: Fri Oct 25, 2013 7:29 am

Re: URGENT - Weird issues after error in config

Post by DigNetwerk »

We restarted crond and postgres and now we are getting emails again (from the non-existent services).
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: URGENT - Weird issues after error in config

Post by WillemDH »

/etc/cron.d/nagiosxi contains:

Code: Select all

# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi

# Backup MySQL & PostgreSQL Databases
0   7 * * * root   /root/scripts/automysqlbackup
0   8 * * * root   /root/scripts/autopostgresqlbackup

*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01  * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php > /usr/local/nagiosxi/var/deadpool.log 2>&1
How can we remove these orphaned notifications? Should I make a support ticket for this?
Nagios XI 5.8.1
https://outsideit.net
DigNetwerk
Posts: 40
Joined: Fri Oct 25, 2013 7:29 am

Re: URGENT - Weird issues after error in config

Post by DigNetwerk »

Hello,

I've look around some more and found out that there is nothing regarding notifications in the Postgres db, but there is in the Mysql db (logged on as ndoutils in the nagios database).

There is a notifications table, where there are exactly as many notifications regarding "traceroute" as we have hosts in that hostgroup (x2 for 2 services):

Code: Select all

SELECT * FROM nagios_notifications WHERE lower(output) LIKE '%traceroute%';
Can we just do

Code: Select all

DELETE FROM nagios_notifications WHERE lower(output) LIKE '%traceroute%';
Or are there other dependencies that we should remove as well?
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: URGENT - Weird issues after error in config

Post by WillemDH »

Sent an email to [email protected] to create a support ticket for this. Did not receive any confirmation though. Did you guys get my email? I(m asking as I had some issues with my changed work email address in the past)
Nagios XI 5.8.1
https://outsideit.net
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: URGENT - Weird issues after error in config

Post by tmcdonald »

We got it, however you won't receive any automated response until one of our techs has assigned it to themselves (or I assign it to someone). Since it's in email, mind if I lock this up?
Former Nagios employee
User avatar
WillemDH
Posts: 2320
Joined: Wed Mar 20, 2013 5:49 am
Location: Ghent
Contact:

Re: URGENT - Weird issues after error in config

Post by WillemDH »

Since it's in email, mind if I lock this up?
Sure
Nagios XI 5.8.1
https://outsideit.net
Locked