Hello Nagios XI Support Team,
I'm just now noticing that Nagios XI email notifications have not worked since July 24th on my platform. And no one reported it!
We've upgraded to 5.7.X (currently 5.7.2) over the summer, but I'm not sure which dates we did which version.
I can send emails via the command line with or without a -r. Weekly scheduled reports are working. Also, sending test notification emails & SMSs from the user's notifications preferenceswork as well.
Here's what's in our phpmailer.log from when it stopped working:
/usr/local/nagiosxi/tmp/phpmailer.log:
[07-24-2020 07:41:31] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[07-24-2020 07:41:32] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Mobile Text
[07-24-2020 08:50:44] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[07-24-2020 08:50:44] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Mobile Text
[07-24-2020 09:51:47] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Email
[07-24-2020 09:51:47] Message sent! (method=sendmail), Referer: includes/components/xicore/xicore.inc.php > Event Handler Notification Mobile Text
[07-24-2020 22:26:54] Message sent! (method=sendmail), Referer: admin/users.php > Account Creation
[07-27-2020 01:00:47] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[07-27-2020 02:00:41] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
...
...
...
[09-07-2020 08:00:08] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-07-2020 12:00:16] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-07-2020 12:00:16] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-08-2020 19:12:01] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-09-2020 02:15:03] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-09-2020 16:43:28] Message sent! (method=sendmail), Referer: admin/users.php > Clone User - Account Creation
[09-10-2020 03:23:45] Message sent! (method=sendmail), Referer: account/testnotification.php > PHPmailer Test
[09-10-2020 03:23:46] Message sent! (method=sendmail), Referer: account/testnotification.php > Mobile Test
[09-10-2020 03:41:13] Message sent! (method=sendmail), Referer: send_user_phone_vkey()
[09-10-2020 03:41:45] Message sent! (method=sendmail), Referer: verify_phone_vkey()
[09-10-2020 09:52:54] Message sent! (method=sendmail), Referer: login.php > Password Reset Email
[09-10-2020 10:44:21] Message sent! (method=sendmail), Referer: login.php > Password Reset Email
[09-11-2020 15:55:08] Message sent! (method=sendmail), Referer: admin/users.php > Administrator Password Reset
[09-14-2020 01:00:48] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 01:00:48] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 02:00:45] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 02:00:45] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 08:00:09] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 12:00:05] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-14-2020 12:00:05] Message sent! (method=sendmail), Referer: includes/components/scheduledreporting > Scheduled Report
[09-15-2020 15:43:06] Message sent! (method=sendmail), Referer: account/testnotification.php > PHPmailer Test
[09-15-2020 15:43:07] Message sent! (method=sendmail), Referer: account/testnotification.php > Mobile Test
[09-15-2020 18:46:33] Message sent! (method=sendmail), Referer: send_user_phone_vkey()
[09-15-2020 18:46:52] Message sent! (method=sendmail), Referer: send_user_phone_vkey()
[09-15-2020 18:47:15] Message sent! (method=sendmail), Referer: verify_phone_vkey()
Here's what's in the syslogs, maillogs after testing a services notification on one of my servers:
root@nagios-b log]# tail -f messages | grep luczy
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynjROTA;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:07:19 nagios-b nagios: SERVICE ALERT: GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;SOFT;1;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:07:31 nagios-b nagios: SERVICE ALERT: GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;SOFT;2;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:08:33 nagios-b nagios: SERVICE ALERT: GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;SOFT;3;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:09:33 nagios-b nagios: SERVICE ALERT: GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;SOFT;4;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: youngr;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: toolea;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynjROTA;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Sep 15 19:10:31 nagios-b nagios: SERVICE ALERT: GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;HARD;5;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
[root@nagios-b log]# grep luczynj maillog
Sep 15 15:49:37 nagios-b sendmail[7404]: 08FFnaFd007404: from=luczynj, size=31809, class=0, nrcpts=1, msgid=<[email protected]>, relay=root@localhost
Sep 15 15:49:37 nagios-b sendmail[7550]: 08FFnbSn007550: from=<[email protected]>, size=32104, class=0, nrcpts=1, msgid=<[email protected]>, proto=ESMTP, daemon=MTA, relay=localhost [127.0.0.1] (may be forged)
Sep 15 15:49:37 nagios-b sendmail[7404]: 08FFnaFd007404: to=[email protected], ctladdr=luczynj (501/506), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=61809, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (08FFnbSn007550 Message accepted for delivery)
Sep 15 15:49:39 nagios-b sendmail[7793]: 08FFnbSn007550: to=<[email protected]>, ctladdr=<[email protected]> (501/506), delay=00:00:02, xdelay=00:00:02, mailer=relay, pri=152104, relay=rspsmtp2.reston.concert.com. [168.125.8.217], dsn=2.0.0, stat=Sent ( <[email protected]> Queued mail for delivery)
There's not an etry in maillog for the service clearing, and that is in my preferences.
Thank you for your support. Please help!
Regards,
JL
Sep 15 19:13:58 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;OK;xi_service_notification_handler;DISK OK - free space: / 9039 MB (45% inode=89%):
Sep 15 19:13:58 nagios-b nagios: SERVICE NOTIFICATION: luczynjROTA;GMV-OSS-RC-TPOSSA01;/ Disk Usage;OK;xi_service_notification_handler;DISK OK - free space: / 9039 MB (45% inode=89%):
Email Notifications Are Not Working
-
benjaminsmith
- Posts: 5324
- Joined: Wed Aug 22, 2018 4:39 pm
- Location: saint paul
Re: Email Notifications Are Not Working
Hi,
Thank you for contacting the Nagios Support Team.
Just wondering if you continued to receive the scheduled reports the entire time? Also, is there nothing in the phpmailerlog between 7/27 and 9/7?
Looking at the output the logs posted, the contact is configured to use the XI mailer handler or the phpmailer library. Did you receive this notification? I don't see the corresponding entry in the log.
Also, can you send over the system profile and I'll check the configurations on this contact/service?
Regards,
Benjamin
To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message or upload it to the post/ticket, and then reply to this post to bring it up in the queue.
Reference:
Nagios XI - Notification Problems
Thank you for contacting the Nagios Support Team.
Just wondering if you continued to receive the scheduled reports the entire time? Also, is there nothing in the phpmailerlog between 7/27 and 9/7?
Looking at the output the logs posted, the contact is configured to use the XI mailer handler or the phpmailer library. Did you receive this notification? I don't see the corresponding entry in the log.
Code: Select all
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
Regards,
Benjamin
To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message or upload it to the post/ticket, and then reply to this post to bring it up in the queue.
Reference:
Nagios XI - Notification Problems
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Be sure to check out our Knowledgebase for helpful articles and solutions!
Be sure to check out our Knowledgebase for helpful articles and solutions!
Re: Email Notifications Are Not Working
Hello,
Thanks for the resposne.
Yes, the scheduled reports have been working.
No, I didn't receive the email for this notification:
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
The /usr/local/nagiosxi/tmp/phpmailer.log shows the last Event Handler notification on [07-24-2020 09:51:47].
After that, it's password resets, PHPmailer tests, Mobile Tests, Account Creation, etc.. No more Event Handlers.
Regards,
Joe
Thanks for the resposne.
Yes, the scheduled reports have been working.
No, I didn't receive the email for this notification:
Sep 15 19:10:31 nagios-b nagios: SERVICE NOTIFICATION: luczynj;GMV-OSS-RC-TPOSSA01;/ Disk Usage;CRITICAL;xi_service_notification_handler;DISK CRITICAL - free space: / 1867 MB (9% inode=89%):
The /usr/local/nagiosxi/tmp/phpmailer.log shows the last Event Handler notification on [07-24-2020 09:51:47].
After that, it's password resets, PHPmailer tests, Mobile Tests, Account Creation, etc.. No more Event Handlers.
Regards,
Joe
You do not have the required permissions to view the files attached to this post.
Re: Email Notifications Are Not Working
Your contact is set to use the backend MTA:
define contact {
contact_name l*****j
alias J***********i
service_notification_period 24x7
host_notification_period 24x7
service_notification_options r,w,u,c,f,s
host_notification_options r,d,u,f,s
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
Go to Configure > Core Config Manager > Contacts:
- Edit your contact
- Click the Alert Settings tab
- Click the Host Notification Commands button and REMOVE notify-host-by-email and ADD xi_host_notification_handler
- Click OK
- Click the Service Notification Commands button and REMOVE notify-service-by-email and ADD xi_service_notification_handler
- Click OK
- Click Save and Apply Configuration
Now it will use the proper XI settings in Admin > Email Settings instead of the backend MTA.
You can either do that OR If you really want it to use the backend MTA (postfix likely in this case) you would need to add a relayhost to your /etc/postfix/main.cf and restart the postfix service so it will relay through the server you want (there are a ton of guides on google for this). Then it should relay properly as well. I would probably do this anyways (but still change your contact) because distribution lists won't have users either and those will be required to use notify-host/service-by-email (because they don't have an associated user in Admin > Manage Users with the same name which is a requirement of the xi_host/service_notification_handlers).
define contact {
contact_name l*****j
alias J***********i
service_notification_period 24x7
host_notification_period 24x7
service_notification_options r,w,u,c,f,s
host_notification_options r,d,u,f,s
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
Go to Configure > Core Config Manager > Contacts:
- Edit your contact
- Click the Alert Settings tab
- Click the Host Notification Commands button and REMOVE notify-host-by-email and ADD xi_host_notification_handler
- Click OK
- Click the Service Notification Commands button and REMOVE notify-service-by-email and ADD xi_service_notification_handler
- Click OK
- Click Save and Apply Configuration
Now it will use the proper XI settings in Admin > Email Settings instead of the backend MTA.
You can either do that OR If you really want it to use the backend MTA (postfix likely in this case) you would need to add a relayhost to your /etc/postfix/main.cf and restart the postfix service so it will relay through the server you want (there are a ton of guides on google for this). Then it should relay properly as well. I would probably do this anyways (but still change your contact) because distribution lists won't have users either and those will be required to use notify-host/service-by-email (because they don't have an associated user in Admin > Manage Users with the same name which is a requirement of the xi_host/service_notification_handlers).
Re: Email Notifications Are Not Working
Hello.
Over the weekend a colleague found that /usr/local/nagiosxi and /usr/local/nagiosxi/var were set to 755, which stopped the event handler from sending the mail through.
It appears that this happened ~around~ the time of our 5.7.2 upgrade. Is this a known issue? What should the permissions of this file system be?
[root@nagios-b /]# ls -lsa /usr/local
4 drwxr-xr-x. 20 root root 4096 Apr 25 2019 .
4 drwxr-xr-x. 16 root root 4096 Oct 3 2018 ..
4 drwxr-xr-x. 2 root root 4096 Jun 18 12:47 bin
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 etc
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 games
4 drwxr-xr-x. 3 root root 4096 Aug 23 2018 include
4 drwxr-xr-x. 3 root root 4096 Aug 23 2018 lib
4 drwxr-xr-x. 3 root root 4096 Jul 10 2017 lib64
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 libexec
4 drwxr-xr-x 9 root root 4096 Feb 22 2018 nagios
4 drwxr-xr-x. 6 root root 4096 Jul 11 2017 nagiosmobile
4 drwxr-xr-x 12 root nagios 4096 May 5 2016 nagiosxi
If you would like all of his troubleshooting notes, let me know.
Regards,
Joe
Over the weekend a colleague found that /usr/local/nagiosxi and /usr/local/nagiosxi/var were set to 755, which stopped the event handler from sending the mail through.
It appears that this happened ~around~ the time of our 5.7.2 upgrade. Is this a known issue? What should the permissions of this file system be?
[root@nagios-b /]# ls -lsa /usr/local
4 drwxr-xr-x. 20 root root 4096 Apr 25 2019 .
4 drwxr-xr-x. 16 root root 4096 Oct 3 2018 ..
4 drwxr-xr-x. 2 root root 4096 Jun 18 12:47 bin
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 etc
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 games
4 drwxr-xr-x. 3 root root 4096 Aug 23 2018 include
4 drwxr-xr-x. 3 root root 4096 Aug 23 2018 lib
4 drwxr-xr-x. 3 root root 4096 Jul 10 2017 lib64
4 drwxr-xr-x. 2 root root 4096 Sep 23 2011 libexec
4 drwxr-xr-x 9 root root 4096 Feb 22 2018 nagios
4 drwxr-xr-x. 6 root root 4096 Jul 11 2017 nagiosmobile
4 drwxr-xr-x 12 root nagios 4096 May 5 2016 nagiosxi
If you would like all of his troubleshooting notes, let me know.
Regards,
Joe
Re: Email Notifications Are Not Working
I haven't seen this reported other than from your other post on this, here is what mine shows:
Are you just notifying us again on this or do you still have issues?
Code: Select all
[root@xid local]# ls -lsa /usr/local
total 0
0 drwxr-xr-x. 18 root root 219 Dec 10 2019 .
0 drwxr-xr-x. 13 root root 155 Aug 14 2018 ..
0 drwxr-xr-x. 2 root root 187 Jun 24 12:02 bin
0 drwxr-xr-x. 2 root root 6 Apr 10 2018 etc
0 drwxr-xr-x. 2 root root 6 Apr 10 2018 games
0 drwxr-xr-x. 3 root root 23 Oct 11 2019 include
0 drwxr-xr-x. 2 root root 112 Oct 11 2019 lib
0 drwxr-xr-x. 3 root root 19 Oct 11 2019 lib64
0 drwxr-xr-x. 2 root root 6 Apr 10 2018 libexec
0 drwxr-xr-x 9 root root 94 Oct 11 2019 nagios
0 drwxr-xr-x. 6 root root 227 Oct 11 2019 nagiosmobile
0 drwxr-xr-x 10 root nagios 102 Oct 11 2019 nagiosxi
0 drwxr-xr-x 5 apache nagios 70 Oct 11 2019 nagvis
0 drwxr-xr-x 7 nagios nagios 121 Oct 11 2019 nrdp
0 drwxr-xr-x. 4 root root 30 Oct 11 2019 nsis
0 drwxr-xr-x. 2 root root 6 Apr 10 2018 sbin
0 drwxr-xr-x. 6 root root 62 Oct 11 2019 share
0 drwxr-xr-x. 2 root root 6 Apr 10 2018 srcRe: Email Notifications Are Not Working
Hey there. Thanks for correcting this.
I had mistakenly posted on the other forum post, thinking it was this one.
The issue is fixed now due to the group write permission on /usr/local/nagiosxi/var wasn't set.
I'm curious if this occurred during our 5.7.2 upgrade, since it occurred on both our primary and backup server around the same time that we upgraded from 5.7.1 to 5.7.2.
Thanks again.
JLu
I had mistakenly posted on the other forum post, thinking it was this one.
The issue is fixed now due to the group write permission on /usr/local/nagiosxi/var wasn't set.
I'm curious if this occurred during our 5.7.2 upgrade, since it occurred on both our primary and backup server around the same time that we upgraded from 5.7.1 to 5.7.2.
Thanks again.
JLu
Re: Email Notifications Are Not Working
I haven't heard of anyone else reporting this, this also didn't happen on my system when testing.
I checked the code and didn't see any permissions changes that should impact that.
What is the output of these commands as root?
I checked the code and didn't see any permissions changes that should impact that.
What is the output of these commands as root?
Code: Select all
umask
su - nagios
umask