PROBLEM: Host Alert: <host> is UP

[fsbeaunix]

I am unable to find the elusive setting in either Core or XI to stop the following recurring message. This message only appears for hosts that have an actual Service Problem, whether handled or not. All hosts without Service Problem are fine and do not report their UP status.

** PROBLEM Host Alert: psappd1 is UP **

***** Nagios Monitor XI Alert *****

Notification Type: PROBLEM
Host: <host>
State: UP
Address: 10.132.59.26
Info: OK - 10.132.59.26: rta 0.280ms, lost 0%

Date/Time: Mon Apr 20 15:54:13 EDT 2015

My configuration:

- Contact Template for xi_contact_generic/Alert Settings: Host notification options, only 'd' selected
- Host Templates/Alert Settings: Host notification options, only 'd' selected
- Contact Settings/Alert Settings, Host notification options, only 'd' selected
- Hosts are configured using Host Templates

I looked for and any Alert Settings I can find and they are all only configured to alert when a host is down. But what's weird is the message above has both PROBLEM and UP in the body. Obviously being UP isn't normally a PROBLEM.

Any tips how to find the elusive setting or misconfiguration to disable this alert?

Here are some config's that might help (I only copied relevant info and left the fluff out):

contacts.cfg:
define contact {
contact_name nagiosadmin
alias Nagios Administrator
host_notifications_enabled 0
service_notifications_enabled 0
host_notification_period 24x7
service_notification_period nagiosadmin_notification_times
host_notification_options d,
service_notification_options w,u,c,
host_notification_commands xi_host_notification_handler
service_notification_commands xi_service_notification_handler
email nagios@localhost
use xi_contact_generic
}

define contact {
contact_name unixpager
alias Unix Pagers
host_notifications_enabled 1
service_notifications_enabled 1
host_notification_period unixpager_notification_times
service_notification_period unixpager_notification_times
host_notification_options d,
service_notification_options w,u,c,
host_notification_commands notify-host-by-email
service_notification_commands notify-host-by-email
can_submit_commands 1
retain_status_information 1
retain_nonstatus_information 1
email [email protected]
use xi_contact_generic
}

contactgroups.cfg:
define contact {
name generic-contact
host_notification_period 24x7
service_notification_period 24x7
host_notification_options d,
service_notification_options w,u,c,
host_notification_commands notify-host-by-email
service_notification_commands notify-service-by-email
register 0
}

define contact {
name xi_contact_generic
host_notification_period xi_timeperiod_24x7
service_notification_period xi_timeperiod_24x7
host_notification_options d,
service_notification_options w,u,c,
host_notification_commands xi_host_notification_handler
service_notification_commands xi_service_notification_handler
register 0
}

contacttemplate.cfg:
define host {
name RHEL_SuSE_production_template
alias RHEL SuSE Production Template
hostgroups RHEL-SuSE-Linux-Prod-Group
check_command check-host-alive!!!!!!!!
use RHEL_SuSE_production_template
initial_state d
max_check_attempts 5
check_interval 5
retry_interval 1
active_checks_enabled 1
passive_checks_enabled 1
check_period 24x7
obsess_over_host 0
event_handler notify-host-by-email
event_handler_enabled 1
flap_detection_enabled 1
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
contacts unixemail,unixpager
notification_interval 30
notification_period 24x7
first_notification_delay 1
notification_options d,
notifications_enabled 1
register 0

}

define host {
name RHEL_SuSE__lowers_template
alias RHEL SuSE Linux Lowers Template
hostgroups RHEL-SuSE-Linux-Lowers-Group
check_command check-host-alive!!!!!!!!
use RHEL_SuSE__lowers_template
initial_state u
max_check_attempts 5
check_interval 5
retry_interval 1
active_checks_enabled 1
check_period workhours
obsess_over_host 0
event_handler notify-host-by-email
event_handler_enabled 1
flap_detection_enabled 1
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
contacts unixemail
notification_interval 30
notification_period workhours
first_notification_delay 1
notification_options d,
notifications_enabled 1
register 0

}

hosttemplate.cfg:
define host {
name RHEL_SuSE_production_template
alias RHEL SuSE Production Template
hostgroups RHEL-SuSE-Linux-Prod-Group
check_command check-host-alive!!!!!!!!
use RHEL_SuSE_production_template
initial_state d
max_check_attempts 5
check_interval 5
retry_interval 1
active_checks_enabled 1
passive_checks_enabled 1
check_period 24x7
obsess_over_host 0
event_handler notify-host-by-email
event_handler_enabled 1
flap_detection_enabled 1
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
contacts unixemail,unixpager
notification_interval 30
notification_period 24x7
first_notification_delay 1
notification_options d,
notifications_enabled 1
register 0

}

define host {
name RHEL_SuSE__lowers_template
alias RHEL SuSE Linux Lowers Template
hostgroups RHEL-SuSE-Linux-Lowers-Group
check_command check-host-alive!!!!!!!!
use RHEL_SuSE__lowers_template
initial_state u
max_check_attempts 5
check_interval 5
retry_interval 1
active_checks_enabled 1
check_period workhours
obsess_over_host 0
event_handler notify-host-by-email
event_handler_enabled 1
flap_detection_enabled 1
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
contacts unixemail
notification_interval 30
notification_period workhours
first_notification_delay 1
notification_options d,
notifications_enabled 1
register 0

}

<host>.cfg:
define host {
host_name psappd1
use RHEL_SuSE__lowers_template
address 10.132.59.26
hostgroups RHEL-SuSE-Linux-Lowers-Group
register 1
}

Thanks in advance!

[jdalrymple]

Why do your templates use themselves? I'm not saying that's the problem, but it's definitely strange and I doubt it's helping.

When is it that you see this notification - when a service alert is raised? When a service recovers?

Your behavior is not something I've experienced.

[Box293]

Is this Nagios Core or Nagios XI?

Can you look in /usr/local/nagios/var/objects.cache and post one host and service definition for a host that experiences this problem.

[fsbeaunix]

First, I cleaned up the templates so they do not use themselves. It doesn't seem to have hurt anything, but not the right way to do it. No need to lose any more sleep over that.

What I did to resolve the issue was:

• Go into CCM and double check all my logic. It was a little clumsy, I admit.

• But I found nothing really critical to cause the problem.

• I went into the XI interface and configured the Notification and Host/Service Handlers. This was key.

• I configured a custom E-Mail Alert message under Notification Management. Not critical, but made things easier to read.

• Back in CCM, I configured the Service and Contacts to use xi_host/service_handlers. This was key too.

Shazaam. It works as expected.

I knew it had to be a config issue. I've been maintaining Nagios Core via text files for awhile. I didn't realize the extra dynamics added with XI. Time to take the training wheels off I guess.