Page 1 of 1
Notifications Emails Have Stopped Sending
Posted: Mon Sep 02, 2019 7:45 am
by benzen78
Hi,
We have today noticed that we have suddenly stopped receiving notifications. For years we have not had any issues but suddenly we are and I can't work out why?
Our setup is to send email notifications to our exchange server. This has always worked, but we've noticed that we haven't been receiving them for over a month but can't work out what may have changed.
When we go to Email Settings and click on Send a Test Email, this works fine and we receive the email.
When I force a notification, I get no email?
I check the Event log and can see a service notification event but no email sent, I take a look at phpmailer.log but see no entries relating to this notification.
I've checked contacts and the contacts have the xi_contact_generic template which are using the two required notification commands and nothing has been changed here so I'm stuck as to what is causing this.
I've checked exchange and it doesn't look like they make it this far but have not been rejected with an error.
Can anyone help in identifying the issue, I'm pulling my hair out?
Kind Regards,
Ben
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 8:24 am
by scottwilkerson
Can you show the output of the following
Code: Select all
ll /usr/local/nagiosxi/var/event*
tail -50 /usr/local/nagiosxi/var/eventman.log
Also, is everything green under Admin -> System Component Status ?
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 8:33 am
by benzen78
ll /usr/local/nagiosxi/var/event*
returns:
-rw-r--r-- 1 nagios users 0 Jul 26 12:41 /usr/local/nagiosxi/var/event_handler.lock
-rw-r--r-- 1 nagios nagios 2633748 Sep 3 14:29 /usr/local/nagiosxi/var/event_handler.log
-rw-r--r-- 1 nagios nagios 6058992 Sep 17 2017 /usr/local/nagiosxi/var/event_handler.log-20170917
-rw-r--r-- 1 nagios nagios 6863886 Sep 24 2017 /usr/local/nagiosxi/var/event_handler.log-20170924
-rw-r--r-- 1 nagios nagios 6059788 Oct 1 2017 /usr/local/nagiosxi/var/event_handler.log-20171001
-rw-r--r-- 1 nagios nagios 6629251 Oct 8 2017 /usr/local/nagiosxi/var/event_handler.log-20171008
-rw-r--r-- 1 nagios nagios 160130 Aug 11 03:39 /usr/local/nagiosxi/var/event_handler.log-20190811.gz
-rw-r--r-- 1 nagios users 271763 Sep 3 14:29 /usr/local/nagiosxi/var/eventman.log
-rw-r--r-- 1 nagios nagios 6353290 Sep 17 2017 /usr/local/nagiosxi/var/eventman.log-20170917
-rw-r--r-- 1 nagios nagios 7160748 Sep 24 2017 /usr/local/nagiosxi/var/eventman.log-20170924
-rw-r--r-- 1 nagios nagios 6342274 Oct 1 2017 /usr/local/nagiosxi/var/eventman.log-20171001
-rw-r--r-- 1 nagios nagios 6923489 Oct 8 2017 /usr/local/nagiosxi/var/eventman.log-20171008
-rw-r--r-- 1 nagios nagios 38166 Aug 23 03:18 /usr/local/nagiosxi/var/eventman.log-20190823.gz
tail -50 /usr/local/nagiosxi/var/eventman.log
Returns:
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
....................PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
.
PROCESSED 0 EVENTS
...................
PROCESSED 0 EVENTS
PHP Warning: Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/nagiosim/nagiosim.inc.php on line 491
Yes everything is green under System Component Status
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 8:40 am
by scottwilkerson
looks like you have a stuck lock file
Let's run the following
Code: Select all
rm -f /usr/local/nagiosxi/var/event_handler.lock
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 9:04 am
by benzen78
Perfect, that did it, Notifications are now flooding through..
Is there a way of monitoring this kind of issue in Nagios?
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 9:12 am
by scottwilkerson
benzen78 wrote:Perfect, that did it, Notifications are now flooding through..
Is there a way of monitoring this kind of issue in Nagios?
You could monitor the file's age with the Folder Watch wizard, but to be honest I have only seen this happen one time before and I thought the latest versions of XI had some fail-safes to catch that..
What version of XI are you running?
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 9:27 am
by benzen78
We are currently running 5.6.0 with plans to update in a few weeks so maybe the fail safes were added after this version?
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 9:33 am
by scottwilkerson
benzen78 wrote:We are currently running 5.6.0 with plans to update in a few weeks so maybe the fail safes were added after this version?
I actually believe they were.
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 9:49 am
by benzen78
Thanks for your help, this can now be closed.
Kind Regards,
Ben
Re: Notifications Emails Have Stopped Sending
Posted: Tue Sep 03, 2019 10:15 am
by scottwilkerson
benzen78 wrote:Thanks for your help, this can now be closed.
Kind Regards,
Ben
Great!
Locking