Page 1 of 1

Notifications going to incorrect people

Posted: Thu Aug 28, 2014 11:29 am
by da1701d
Hello,

I have a host in the system configure like the following. The "mvhosttemplate" used below on it has a contact group called "mv" configured in it so host notifications will go to the group when a host notifications sent out. The secondary "standard_host_win" template is also setup to notify a specific contact and add this host to a hostgroup called "basic_checks_windows".

define host {
host_name MVStaging
use mvhosttemplate,standard_host_win
alias mvstaging.wz.hasbro.com
address x.x.x.x (omitted the IP for security reasons)
register 1
}

Yesterday, I created a new service check (shown below) and applied it. The first service template "live_ops_notifications" configure the check settings and one specific contact to send the service notification to. The second template (memcheck_windows_template) holds the check command settings only. I have also listed both template configs here.

### Service check ###
define service {
service_description check-mem-WIN
use live_ops_notifications,memcheck_windows_template
hostgroup_name basic_checks_windows
register 1
}

### Service template ###
define service {
name memcheck_windows_template
service_description memcheck_windows_template
display_name windows snmp template
servicegroups basic_service_checks
check_command check_winmem!snmpCommName -t 30!80!90!!!!!
register 0
}

### Service template ###
define service {
name live_ops_notifications
service_description live_ops_notifications
max_check_attempts 3
check_interval 10
retry_interval 3
check_period 24x7
notification_interval 60
notification_period 24x7
notification_options w,c,u,r,
notifications_enabled 1
contacts liveopsmonitoring
register 0
}

Once this new service check went active after applying the nagios config from XI interface, this host memory was over utilized so it promptly sent out a crit alert but besides going to the "liveopsmonitoring" contact, it also went all other people listed on the MV contact group configured on the "mvhosttemplate".

How is that possible? I thought service check notifications only went out to the contacts or contact groups configured on the service check themselves or any templates used in that service check and host notificaitons would only go out to the contacts or contactgroups configured on the host object or any templates used on the host object.

Any help would be appeciated.

--
[email protected]
Wizards of the Coast

Re: Notifications going to incorrect people

Posted: Thu Aug 28, 2014 11:54 am
by lmiltchev
What contacts do you see defined for "check-mem-WIN" in the "/usr/local/nagios/var/objects.cache"?

Re: Notifications going to incorrect people

Posted: Thu Aug 28, 2014 12:19 pm
by da1701d
In that file I found the host config as follows.....

define host {
host_name MVStaging
alias mvstaging.wz.hasbro.com
address 10.1.1.211
check_period 24x7
check_command check-host-alive
contacts liveopsmonitoring
contact_groups mv
notification_period 24x7
initial_state o
check_interval 5.000000
retry_interval 1.000000
max_check_attempts 10
active_checks_enabled 1
passive_checks_enabled 1
obsess_over_host 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options o,d,u
freshness_threshold 0
check_freshness 0
notification_options d,u,r
notifications_enabled 1
notification_interval 120.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
failure_prediction_enabled 1
retain_status_information 1
retain_nonstatus_information 1
}

And the check-mem-WIN service check as follows....
define service {
host_name MVStaging
service_description check-mem-WIN
display_name windows snmp template
check_period 24x7
check_command check_winmem!Blueg3 -t 30!80!90!!!!!
contacts liveopsmonitoring
contact_groups mv
notification_period 24x7
initial_state o
check_interval 10.000000
retry_interval 3.000000
max_check_attempts 3
is_volatile 0
parallelize_check 1
active_checks_enabled 1
passive_checks_enabled 1
obsess_over_service 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options o,w,u,c
freshness_threshold 0
check_freshness 0
notification_options u,w,c,r
notifications_enabled 1
notification_interval 60.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
failure_prediction_enabled 1
retain_status_information 1
retain_nonstatus_information 1
}

I do notice that both the liveopsmonitoring contact and the "mv" listed after contacts_groups are there. That seems incorrect to me when on that service check config no contactgroups are added in the config.

Re: Notifications going to incorrect people

Posted: Thu Aug 28, 2014 4:42 pm
by slansing
Would you be able to nab a screenshot of both of these notifications going out from Home > Notifications? Out of curiosity, does this happen when you navigate to service details on one of these services, then click the advanced tab, and send out a custom notification?

Re: Notifications going to incorrect people

Posted: Thu Aug 28, 2014 4:53 pm
by lmiltchev
I do notice that both the liveopsmonitoring contact and the "mv" listed after contacts_groups are there. That seems incorrect to me when on that service check config no contactgroups are added in the config.
What happens when you add the "liveopsmonitoring" contact directly to the " check-mem-WIN" service (not via the " live_ops_notifications" template)? I suspect the "mv" contact group gets inherited from the host via implied inheritance, even though, technically it shouldn't, as the "liveopsmonitoring" contact is inherited from the template. I am not sure why this is happening. Do you have multiple instances of nagios running? Try:

Code: Select all

service nagios stop
killall nagios
service ndo2db stop
service ndo2db start
service nagios start

Re: Notifications going to incorrect people

Posted: Thu Aug 28, 2014 5:56 pm
by da1701d
What happens when you add the "liveopsmonitoring" contact directly to the " check-mem-WIN" service (not via the " live_ops_notifications" template)? I suspect the "mv" contact group gets inherited from the host via implied inheritance, even though, technically it shouldn't, as the "liveopsmonitoring" contact is inherited from the template. I am not sure why this is happening. Do you have multiple instances of nagios running?
Thanks. I read through link for the implied inhertance that you indicated and I think that is EXACTLY what is happening. I only have one instance of nagios running this host. I will just make adjustments to the host objects and take away directly configuring a contact their for notfication since we the services are what we mostly care about.

Dan

Re: Notifications going to incorrect people

Posted: Fri Aug 29, 2014 12:28 pm
by abrist
Sounds good. Let us know if you have any further questions.