Page 1 of 1

enable_notifications not working

Posted: Tue Aug 07, 2018 11:49 am
by aanataliya
I am trying to disable all notifications on my nagios core server. Initially I changed nagios.conf with enable_notifications=0. It worked and when I go to GUI, I was able to see red notification saying "notifications are disabled". Then I reverted changes. It worked perfectly and red message from UI was disappeared.

After that, from GUI, I went to System->Process Info->Disable Notification. Command seemed success but I was unable to see red message on GUI so I went again System->Process Info->Enable Notification. After that I am unable to disable notifications. Even if I change nagios.conf and restart service, It doesn't make any difference. Neither it display red message on GUI nor notifications stops. Can you please help me understand

1. Why this behavior?
2. How to reset any GUI command (like enable/disable from GUI)

Thanks.

Re: enable_notifications not working

Posted: Tue Aug 07, 2018 4:15 pm
by lmiltchev
What is the version of Nagios Core that you are currently running? OS/architecture? Run the following commands and show the output:

Code: Select all

/usr/local/nagios/bin/nagios -V
uname -a
cat /etc/*release
grep nag /etc/group
Run the following command from the command line to see if you have multiple nagios processes running:

Code: Select all

ps -ef | grep nagios.cfg | grep -v grep
...Command seemed success but I was unable to see red message on GUI...
Have you tried refreshing the page in your browser?

You can temporarily enable debugging in the nagios.cfg file to further troubleshoot the issue. Read more about possible debug levels that you could use here:

https://assets.nagios.com/downloads/nag ... gmain.html

After you modify the main config file and restart nagios, try enabling/disabling notifications from the GUI, then open the "/usr/local/nagios/var/nagios.debug" file and see if you could find any clues. If you are unsure what's relevant, post the debug file on the forum.

Re: enable_notifications not working

Posted: Wed Aug 08, 2018 5:35 am
by aanataliya
Thanks for reply. Please find below required details.

Code: Select all

/usr/local/nagios/bin/nagios -V
Nagios Core 4.2.0

uname -a
Linux mjc2zwyyzwvjymm 4.14.33-51.37.amzn1.x86_64 #1 SMP Thu May 3 20:07:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/*release
NAME="Amazon Linux AMI"
VERSION="2018.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2018.03"
PRETTY_NAME="Amazon Linux AMI 2018.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2018.03

grep nag /etc/group
nagios:x:501:apache,nrpe
nagcmd:x:502:nagios,apache,nobody

ps -ef | grep nagios.cfg | grep -v grep
nagios    8994     1  0 Aug07 ?        00:00:37 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios    9002  8994  0 Aug07 ?        00:00:01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
Yes it is running two processes but not sure why. I tried to kill/stop processes but as soon as I start service, it again comes back with two processes.

Re: enable_notifications not working

Posted: Wed Aug 08, 2018 8:35 am
by lmiltchev
Yes it is running two processes but not sure why. I tried to kill/stop processes but as soon as I start service, it again comes back with two processes.
Actually, this is normal. The second process is the "child" of the first process (the parent). You can tell by the PID and PPID... They are the same.
nagios 8994 1 0 Aug07 ? 00:00:37 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 9002 8994 0 Aug07 ? 00:00:01 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
Have you tried enabling the debugging, and reviewing the logs?

Re: enable_notifications not working

Posted: Wed Aug 08, 2018 11:01 am
by aanataliya
Oh ok. Thanks.

Below is the log with debug_level=32(notification) and after performing action disable notification from GUI.

Code: Select all

[1533742511.946919] [032.0] [pid=8280] ** Service Notification Attempt ** Host: '10.x.x.x', Service: 'AD Web Service', Type: NORMAL, Options: 0, Current State: 3, Last Notification: Wed Aug  8 15:33:11 2018
[1533742511.946947] [032.1] [pid=8280] Notifications are disabled, so service notifications will not be sent out.
[1533742511.946953] [032.0] [pid=8280] Notification viability test failed.  No notification will be sent out.
[1533742514.953144] [032.0] [pid=8280] ** Service Notification Attempt ** Host: '10.x.x.x', Service: 'AD Domain NTDS Service', Type: NORMAL, Options: 0, Current State: 3, Last Notification: Wed Aug  8 15:33:14 2018
[1533742514.953177] [032.1] [pid=8280] Notifications are disabled, so service notifications will not be sent out.
[1533742514.953183] [032.0] [pid=8280] Notification viability test failed.  No notification will be sent out.
[1533742518.972027] [032.0] [pid=8280] ** Service Notification Attempt ** Host: ''10.x.x.x' Service: 'Intersite Messaging Service', Type: NORMAL, Options: 0, Current State: 3, Last Notification: Wed Aug  8 15:33:18 2018
[1533742518.972058] [032.1] [pid=8280] Notifications are disabled, so service notifications will not be sent out.
[1533742518.972065] [032.0] [pid=8280] Notification viability test failed.  No notification will be sent out.
And below log after performing enable notification from GUI.

Code: Select all

[1533742962.016241] [032.0] [pid=8280] ** Service Notification Attempt ** Host: 10.x.x.x', Service: 'Netlogon Service', Type: NORMAL, Options: 0, Current State: 2, Last Notification: Wed Aug  8 15:34:42 2018
[1533742962.016268] [032.0] [pid=8280] Notification viability test passed.
[1533742962.016273] [032.1] [pid=8280] Current notification number: 726 (incremented)
[1533742962.016278] [032.1] [pid=8280] Service notification will NOT be escalated.
[1533742962.016281] [032.1] [pid=8280] Adding normal contacts for service to notification list.
[1533742962.016430] [032.0] [pid=8280] 2 contacts were notified.  Next possible notification time: Wed Aug  8 15:44:42 2018
[1533742962.016446] [032.0] [pid=8280] 2 contacts were notified.
[1533742963.035465] [032.0] [pid=8280] ** Service Notification Attempt ** Host:10.x.x.x', Service: 'AD Web Service', Type: NORMAL, Options: 0, Current State: 3, Last Notification: Wed Aug  8 15:34:43 2018
And I am able to see red message on GUI as expected. However, I am not able to receive emails. But I am guessing it could be issue with my mail server so we can ignore as of now. but I can also see below log after couple of lines very frequently.

Code: Select all

[1533743535.011883] [032.1] [pid=8280] We haven't waited long enough to re-notify contacts about this service.
[1533743535.011889] [032.1] [pid=8280] Next valid notification time: Wed Aug  8 15:52:16 2018
[1533743535.011893] [032.0] [pid=8280] Notification viability test failed.  No notification will be sent out.

After that, I tried to change enable_notification=0 in nagios.cfg file and restarted service. log is same as enable notification log. and no change on GUI. I can see log showing "2 contacts were notified". So basically, nagios is just ignoring "enable_notifications" flag value. and thats my actual problem.

Re: enable_notifications not working

Posted: Wed Aug 08, 2018 11:34 am
by aanataliya
Oh Finally I found root cause.

As per nagios documentation https://assets.nagios.com/downloads/nag ... ifications, enable_notifications flag value will be ignored if state retention enabled.

I am not sure if it is a good idea to disable this option. Hence I have used use_retained_program_state=0 . Now it is working as expected.

Thanks for your help. Have a nice day! :)

Re: enable_notifications not working

Posted: Wed Aug 08, 2018 11:49 am
by lmiltchev
I am glad your issue has been resolved! I am closing this thread.