Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

Notifications are disabled

https://support.nagios.com/forum/viewtopic.php?t=29323

Page 1 of 3

Notifications are disabled

Posted: Wed Sep 24, 2014 4:07 pm
by tonyleatwork
Hi -

My notifications are reported as being disabled. I noticed that when making certain changes in the CCM, it may come back and it may go away again. I've tried to back out all of my changes (last changes being additional users through CCM) but it's still 'stuck'.

Nagios Log
[root@nwd2ng01 var]# tail -40 nagios.log
[1411592555] ndomod registered for aggregated status data'
[1411592555] ndomod registered for retention data'
[1411592555] ndomod registered for contact data'
[1411592555] ndomod registered for contact notification data'
[1411592555] ndomod registered for acknowledgement data'
[1411592555] ndomod registered for state change data'
[1411592555] ndomod registered for contact status data'
[1411592555] ndomod registered for adaptive contact data'
[1411592555] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.
[1411592555] Warning: Duplicate definition found for service 'Memory Usage' on host 'Nwd2spcbdev2_WMI' (config file '/usr/local/nagios/etc/services/Windows WMI DEV QA Group Memory Usage.cfg', starting on line 16)
[1411592555] Warning: Host 'drbw4.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'drcrm6.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'drerp3.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'drprd3.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'drptl4.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'drsap9.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2app6.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2bw1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2crm4.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2dri.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2erp1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ora1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ora2.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ora3.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ora4.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ora6.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2pft1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2pft2' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2prd1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2prd3corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2ptl4.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2sbx1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2sm3.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2tws1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2wms1.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2wms2.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Warning: Host 'nwd2wmsd.corp.analog.com' has no default contacts or contactgroups defined!
[1411592555] Successfully launched command file worker with pid 25976
[1411592669] SERVICE ALERT: Nwd2spcbdev3_WMI;CPU Usage;UNKNOWN;SOFT;1;UNKNOWN - Plugin Timed out (15 sec)
[1411592675] SERVICE NOTIFICATION: nagiosadmin;Nwd2epmdev1_WMI;Drive C: Disk Usage;WARNING;xi_service_notification_handler;WARNING - [Triggered by _Used%>80] - C: Total=19.99GB, Used=16.06GB (80.3%), Free=3.94GB (19.7%)
[root@nwd2ng01 var]#
Thanks.

Re: Notifications are disabled

Posted: Wed Sep 24, 2014 4:46 pm
by lmiltchev
It seems like some of your hosts don't have contacts or contact groups defined. I would recommend assigning contacts or a contact groups in the CCM to these hosts, in order to fix the issue.
Next, run the Write Config Tool prior to applying configuration to check for config errors. Once you don't have any config errors, you can apply configuration.

Please, read our documentation on how to configure email and text notifications here:

http://assets.nagios.com/downloads/nagi ... ios-XI.pdf

Let us know if you run into more issues.

Re: Notifications are disabled

Posted: Wed Sep 24, 2014 4:52 pm
by abrist
The log above shows a notification going out after the service restart. Are you noticing disabled notifications directly on objects, or program wide?

Re: Notifications are disabled

Posted: Wed Sep 24, 2014 4:59 pm
by tmcdonald
Also, are your notifications actually disabled or does it just say they are? Do you still get email/sms alerts?

What does the following command reveal?

Code: Select all

grep "enable_notifications" /usr/local/nagios/var/status.dat

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 9:46 am
by tonyleatwork
lmiltchev wrote:It seems like some of your hosts don't have contacts or contact groups defined. I would recommend assigning contacts or a contact groups in the CCM to these hosts, in order to fix the issue.
Next, run the Write Config Tool prior to applying configuration to check for config errors. Once you don't have any config errors, you can apply configuration.

Please, read our documentation on how to configure email and text notifications here:

http://assets.nagios.com/downloads/nagi ... ios-XI.pdf

Let us know if you run into more issues.
That's correct, only some hosts are enabled for notifications directly.

The other hosts are in host groups with services defined against the group. I've also created a host template with specific alert criteria and applied it to the group as well. What I'm trying to achieve is to configure notifications group-wide instead of per host.

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 9:48 am
by tonyleatwork
abrist wrote:The log above shows a notification going out after the service restart. Are you noticing disabled notifications directly on objects, or program wide?
The notifications seem to be working but slowly, the indication I got that it is program wide are the 3 exclamation marks in the Nagios XI console. When clicking on one of those and forcing the monitoring engine to run, it's still stuck.

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 9:51 am
by tonyleatwork
tmcdonald wrote:Also, are your notifications actually disabled or does it just say they are? Do you still get email/sms alerts?

What does the following command reveal?

Code: Select all

grep "enable_notifications" /usr/local/nagios/var/status.dat
Notifications are not disabled, I did notice some email alerts came through.

grep enabled_notifications /usr/local/nagios/var/status.dat

shows nothing, but notifications_enabled grep showed a ton:

enable_notifications=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
notifications_enabled=1
no_more_notifications=0
(truncated)

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 9:56 am
by tonyleatwork
Update:

Today, the notifications look OK again. It wasn't OK for at least an hour before I left work yesterday. The only abnormal behavior I noticed is that ndo2db is running at high cpu.. could this be the cause? It's consistently running at 100% and has been for a while. Reboot attempts have not done anything.

It is on a VM with 4 cpus and 8GB of ram, san storage back end.

25935 nagios 20 0 51744 3236 1008 R 100.0 0.0 1061:55 ndo2db

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 1:49 pm
by tonyleatwork
Update:

I tried to add a notification contact to a host template to apply to a host and the same issue is happening again. The "Notifications are disabled" blue exclamation mark along with "Active Service Checks" and "Host Service Checks" in the Nagios XI console.

Re: Notifications are disabled

Posted: Thu Sep 25, 2014 5:10 pm
by abrist
What version of XI are you running?
Are any of your db tables crashed?

Code: Select all

grep crashed /var/log/mysqld.log
How big are your tables?

Code: Select all

ls -lahS /var/lib/mysql/nagios | head -10
All times are UTC-05:00
Page 1 of 3

Powered by phpBB® Forum Software © phpBB Limited