NagiosXI Emails Stoped working

[spcmidrange]

Good day:

However, the failed connections are over either SSL or TLS on port 25. Usually encyrptied connections use another port such as 587 or 465. Can you verify the port settings with your mail admin?

On my second post of this thread:

Errors in this log about ssl/tls were us testing,

There is no SSL enabled/enforced. We were doing testing enabling them to see if thats why we were having problems

the phpmailer.log is the entire log, the notification never was sent to us

Code: Select all

[root@lp-dc1-nagn1 etc]# tail -n 50 /usr/local/nagiosxi/tmp/phpmailer.log
[11-19-2019 13:54:00] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 13:54:19] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 13:59:48] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:06:55] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:15:42] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:16:40] Message sent! (method=smtp;host=172.30.4.111;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:18:50] Message sent! (method=smtp;host=172.30.4.112;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:22:09] Message sent! (method=smtp;host=172.30.4.112;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:30:40] Message sent! (method=sendmail), Referer: admin/testemail.php
[11-19-2019 14:32:11] Message sent! (method=sendmail), Referer: admin/testemail.php
[11-19-2019 14:42:23] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-19-2019 14:48:10] SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=tls), Referer: admin/testemail.php
[11-19-2019 14:48:24] SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=ssl), Referer: admin/testemail.php
[11-20-2019 08:26:24] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-20-2019 09:25:33] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: admin/testemail.php
[11-20-2019 09:39:22] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: account/testnotification.php > PHPmailer Test
[11-20-2019 09:39:42] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: account/testnotification.php > PHPmailer Test
[11-20-2019 12:21:56] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: account/testnotification.php > PHPmailer Test
[11-21-2019 08:09:45] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: account/testnotification.php > PHPmailer Test
[root@lp-dc1-nagn1 etc]# date
Thu Nov 21 12:34:21 CST 2019

You keep mention the phpmailer troubleshooting page, but where would i see these debug messages to troubleshoot? If i run the command by hand for notifications, i get no output no errors nor know where i should see them. /var/log/messages? nagios.log? Please advise troubleshooting steps that i should do because would love to troubleshoot this to help get this going. Its already been a week since it stopped working

[benjaminsmith]

Hello,

My apologies for the back and forth but on occasion notifications issues can take some time to troubleshoot. If the test email is going through, normally it is an issue with the configurations. If Nagios is firing the notification then it's either a connection issue with the SMTP server or the service is blocking or filtering the message.

The phpmailer log doesn't have enough data to determine the error so far. Run the following tail command and leave it running.

Code: Select all

tail -F /var/log/maillog /usr/local/nagiosxi/tmp/phpmailer.log /usr/local/nagiosxi/var/eventman.log &> support.txt

Next go to Home > Details > Service Status, in the search box in the upper right, search for the following service:

NRPE Client SSL Expiration

The midrangeteam should be set to use the xi service notificaiton handlers (SMTP).

Then follow the steps below force a notification to be sent then send me the full output of the tail command above and any errors you see on the screen.

For Services
----------------

Go to Home > Service Status:
- Find the Service and click on it
- Click the + Advanced tab

Note these two rows:
State Type: Hard
Current Check: 1 of 4

Those columns tell the current State Type and the Current Check number. In order to generate a notification for a service, you will need to submit MULTIPLE problem check results (the number that you need to submit is determined by the last number in the Current Check column, that is the max_check_attempts setting).

For services, when you submit a passive check result, each result that you submit will be a SOFT state until you submit enough to hit the Max Check Attempts setting that you've defined on the service, only then will the service enter a HARD problem state which will generate the notification (just remember, notifications are only sent on HARD states).

Once the service has entered a HARD state, upload the log file /tmp/support.txt to the thread. Thanks.

[spcmidrange]

Good Day

I am at a loss for words. Over night, all the emails that were not being sent out, were sent out. Got 5 days worth of emails. The system is setup to not notify after work hours (so only mon-fri 8-5) so the alerts over night is also a strange one

The phpmailer log shows now them being sent out over night:

Code: Select all

[root@lp-dc1-nagn1 etc]# tail -n 50 /usr/local/nagiosxi/tmp/phpmailer.log
[11-21-2019 21:36:02] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:03] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:12] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:14] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:15] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:15] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:15] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:16] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:17] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:20] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:21] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:22] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:23] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:25] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:36] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:36] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:36:38] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:38:13] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:42:43] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:42:55] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:43:07] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:43:23] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:43:24] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:45:38] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:45:49] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:45:59] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:08] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:09] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:10] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:10] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:12] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:16] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:17] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:18] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:30] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:30] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:32] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:33] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:43] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:45] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:46] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:46] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:47] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:49] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:50] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:54] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:54] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-21-2019 21:46:56] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-22-2019 08:00:22] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[11-22-2019 08:01:13] Message sent! (method=smtp;host=smtpmailroute.saskpower.sk.ca;port=25;security=none), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email

The email includes event time, and that matches the timestamps on the log. But these alerts were from the past 5 days. The real ones, plus the custom ones all trickled out over 3 hours last night. Custom and actual notification emails are coming through now

this is super strange. Its not like they were qued at the exchange server. phpmailer log shows it sent them out over night. Why did it stop?

Please advise if you would want any more logs for troubleshooting. If not, I would like to keep an eye on this till prob monday before saying its resolved

Thank you for your assistance with this

Cheers!

[benjaminsmith]

Hello @spcmidrange,

That is good news. Let's keep this thread open until next week and let me know if you have any issues, and we can take a look at the xi_eventqueue table in the nagiosxi database. It may be necessary to truncate this table and run the repair script.

Have a great weekend.

Benjamin

[spcmidrange]

Good Day

We have had no more issues, so i guess we can mark this as resolved by the boogie man hahah

Thank you for all your assistance!

Cheers!

[scottwilkerson]

Good Day

We have had no more issues, so i guess we can mark this as resolved by the boogie man hahah

Thank you for all your assistance!

Cheers!

Great!

Locking thread