Can't Get Host Notifications To Work

An open discussion forum for obtaining help with Nagios Core. Nagios Core users of all experience levels are welcome here. Subforum have been created for the discussion of Nagios Core and Nagios Plugin development.

NOTE: The SourceForge.net mailing lists have been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

Can't Get Host Notifications To Work

Postby gohanman » Thu Jun 15, 2017 4:19 pm

I've looked through the documentation extensively and cannot figure out why my service status notifications are working 100% of the time and my host status notifications are not working 100% of the time.

All the services in question use this template. I always get notifications on these.
Code: Select all
define service{                                       
        name                            generic-service
        active_checks_enabled           1             
        passive_checks_enabled          1             
        parallelize_check               1             
        obsess_over_service             1             
        check_freshness                 0             
        notifications_enabled           1             
        event_handler_enabled           1             
        flap_detection_enabled          1             
        failure_prediction_enabled      1             
        process_perf_data               1             
        retain_status_information       1             
        retain_nonstatus_information    1             
        is_volatile                     0             
        check_period                    24x7           
        max_check_attempts              3             
        normal_check_interval           10             
        retry_check_interval            2             
        contact_groups                  admins         
        notification_options            w,u,c,r       
        notification_interval           0             
        notification_period             24x7           
        register                        0             
        }                                             


All hosts uses these templates. I never get notifications on these.
Code: Select all
define host{                                   
    name                            generic-host
    notifications_enabled           1           
    event_handler_enabled           1           
    flap_detection_enabled          1           
    failure_prediction_enabled      1           
    process_perf_data               1           
    retain_status_information       1           
    retain_nonstatus_information    1           
    notification_period             24x7       
    register                        0           
    }     
define host{                               
    name                    linux-server   
    use                     generic-host   
    check_period            24x7           
    check_interval          5               
    retry_interval          1               
    max_check_attempts      10             
    check_command           check-host-alive
    notification_period     24x7           
    notification_interval   0               
    notification_options    d,u,r           
    contact_groups          admins         
    register                0               
    }                                                                           


The only notification setting I can see that differs is "notification_options" and per the documentation it seems like these should be different for hosts vs services. I am not overriding any notification* or contact* parameters in individual host or service definitions inherited from these templates.

The template for the contact seems to have both host and service notifications enabled:
Code: Select all
define contact{                                           
    name                            generic-contact       
    service_notification_period     24x7                   
    host_notification_period        24x7                   
    service_notification_options    w,u,c,r,f,s           
    host_notification_options       d,u,r,f,s             
    service_notification_commands   notify-service-by-email
    host_notification_commands      notify-host-by-email   
    register                        0                     
    }                                                     


And the command definitions for notify-service-by-email and notify-host-by-email seem reasonably similar, ultimately just using /bin/mail to email $CONTACTEMAIL$
Code: Select all
define command{
    command_name    notify-host-by-email
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
    }
define command{
    command_name    notify-service-by-email
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
    }


I'm not seeing anything in nagios.log to indicate that it's trying to send a host down notification and encountering an error.

Can anyone point out what I'm overlooking here or how I might further debug what's happening here?
gohanman
 
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Re: Can't Get Host Notifications To Work

Postby tgriep » Fri Jun 16, 2017 12:04 pm

The settings look like the host notifications should be sent out.
There is a file called objects.cache and in there are all of the configs parsed out and that is what the Nagios daemon uses when running the checks, etc...
Can you open that file, find one of the hosts that is not sending email notifications and post the configuration here so we can check it?
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 6019
Joined: Thu Oct 30, 2014 9:02 am

Re: Can't Get Host Notifications To Work

Postby gohanman » Fri Jun 16, 2017 1:37 pm

Sure. I did pull the IP address but the rest is verbatim from object.cache

Code: Select all
define host {
    host_name   station2
    alias   station2
    address redacted
    check_period    24x7
    check_command   check-host-alive
    contact_groups  admins
    notification_period 24x7
    initial_state   o
    check_interval  5.000000
    retry_interval  1.000000
    max_check_attempts  10
    active_checks_enabled   1
    passive_checks_enabled  1
    obsess_over_host    1
    event_handler_enabled   1
    low_flap_threshold  0.000000
    high_flap_threshold 0.000000
    flap_detection_enabled  1
    flap_detection_options  o,d,u
    freshness_threshold 0
    check_freshness 0
    notification_options    d,u,r
    notifications_enabled   1
    notification_interval   0.000000
    first_notification_delay    0.000000
    stalking_options    n
    process_perf_data   1
    failure_prediction_enabled  1
    retain_status_information   1
    retain_nonstatus_information    1
    }


I'm now seeing that could leave a 10-15 minute delay for the first notification, right? Up to 5 minutes before the regularly scheduled check fails and then nine more minutes for retries before it hits a HARD state?
gohanman
 
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Re: Can't Get Host Notifications To Work

Postby dwhitfield » Fri Jun 16, 2017 2:22 pm

If you run the following from the command line (but with an actual email address), what happens?

Code: Select all
/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$


It worked for me, and there's no reason to believe it won't work for you since your services work, but just in case. Also, can you attach or put in code blocks your nagios.cfg?
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
dwhitfield
The Doctor
 
Posts: 3756
Joined: Wed Sep 21, 2016 10:29 am
Location: Nagios Enterprises, LLC

Re: Can't Get Host Notifications To Work

Postby gohanman » Fri Jun 16, 2017 2:42 pm

It looks like is was max_check_attempts. My own fault for not setting that low enough and/or not being patient enough. Thanks for the pointers.
gohanman
 
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Re: Can't Get Host Notifications To Work

Postby tmcdonald » Mon Jun 19, 2017 10:37 am

Did you have any further (related) questions or are we okay to close this thread?
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tmcdonald
Support Lead / Operations Engineer
 
Posts: 8683
Joined: Mon Sep 23, 2013 8:40 am


Return to Nagios Core

Who is online

Users browsing this forum: Bing [Bot] and 4 guests