Can't Get Host Notifications To Work

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
gohanman
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Can't Get Host Notifications To Work

Post by gohanman »

I've looked through the documentation extensively and cannot figure out why my service status notifications are working 100% of the time and my host status notifications are not working 100% of the time.

All the services in question use this template. I always get notifications on these.

Code: Select all

define service{                                        
        name                            generic-service
        active_checks_enabled           1              
        passive_checks_enabled          1              
        parallelize_check               1              
        obsess_over_service             1              
        check_freshness                 0              
        notifications_enabled           1              
        event_handler_enabled           1              
        flap_detection_enabled          1              
        failure_prediction_enabled      1              
        process_perf_data               1              
        retain_status_information       1              
        retain_nonstatus_information    1              
        is_volatile                     0              
        check_period                    24x7           
        max_check_attempts              3              
        normal_check_interval           10             
        retry_check_interval            2              
        contact_groups                  admins         
        notification_options            w,u,c,r        
        notification_interval           0              
        notification_period             24x7           
        register                        0              
        }                                              
All hosts uses these templates. I never get notifications on these.

Code: Select all

define host{                                    
    name                            generic-host
    notifications_enabled           1           
    event_handler_enabled           1           
    flap_detection_enabled          1           
    failure_prediction_enabled      1           
    process_perf_data               1           
    retain_status_information       1           
    retain_nonstatus_information    1           
    notification_period             24x7        
    register                        0           
    }      
define host{                                
    name                    linux-server    
    use                     generic-host    
    check_period            24x7            
    check_interval          5               
    retry_interval          1               
    max_check_attempts      10              
    check_command           check-host-alive
    notification_period     24x7            
    notification_interval   0               
    notification_options    d,u,r           
    contact_groups          admins          
    register                0               
    }                                                                            
The only notification setting I can see that differs is "notification_options" and per the documentation it seems like these should be different for hosts vs services. I am not overriding any notification* or contact* parameters in individual host or service definitions inherited from these templates.

The template for the contact seems to have both host and service notifications enabled:

Code: Select all

define contact{                                            
    name                            generic-contact        
    service_notification_period     24x7                   
    host_notification_period        24x7                   
    service_notification_options    w,u,c,r,f,s            
    host_notification_options       d,u,r,f,s              
    service_notification_commands   notify-service-by-email
    host_notification_commands      notify-host-by-email   
    register                        0                      
    }                                                      
And the command definitions for notify-service-by-email and notify-host-by-email seem reasonably similar, ultimately just using /bin/mail to email $CONTACTEMAIL$

Code: Select all

define command{
    command_name    notify-host-by-email
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
    }
define command{
    command_name    notify-service-by-email
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
    }
I'm not seeing anything in nagios.log to indicate that it's trying to send a host down notification and encountering an error.

Can anyone point out what I'm overlooking here or how I might further debug what's happening here?
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Can't Get Host Notifications To Work

Post by tgriep »

The settings look like the host notifications should be sent out.
There is a file called objects.cache and in there are all of the configs parsed out and that is what the Nagios daemon uses when running the checks, etc...
Can you open that file, find one of the hosts that is not sending email notifications and post the configuration here so we can check it?
Be sure to check out our Knowledgebase for helpful articles and solutions!
gohanman
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Re: Can't Get Host Notifications To Work

Post by gohanman »

Sure. I did pull the IP address but the rest is verbatim from object.cache

Code: Select all

define host {
    host_name   station2
    alias   station2
    address redacted
    check_period    24x7
    check_command   check-host-alive
    contact_groups  admins
    notification_period 24x7
    initial_state   o
    check_interval  5.000000
    retry_interval  1.000000
    max_check_attempts  10
    active_checks_enabled   1
    passive_checks_enabled  1
    obsess_over_host    1
    event_handler_enabled   1
    low_flap_threshold  0.000000
    high_flap_threshold 0.000000
    flap_detection_enabled  1
    flap_detection_options  o,d,u
    freshness_threshold 0
    check_freshness 0
    notification_options    d,u,r
    notifications_enabled   1
    notification_interval   0.000000
    first_notification_delay    0.000000
    stalking_options    n
    process_perf_data   1
    failure_prediction_enabled  1
    retain_status_information   1
    retain_nonstatus_information    1
    }
I'm now seeing that could leave a 10-15 minute delay for the first notification, right? Up to 5 minutes before the regularly scheduled check fails and then nine more minutes for retries before it hits a HARD state?
dwhitfield
Former Nagios Staff
Posts: 4583
Joined: Wed Sep 21, 2016 10:29 am
Location: NoLo, Minneapolis, MN
Contact:

Re: Can't Get Host Notifications To Work

Post by dwhitfield »

If you run the following from the command line (but with an actual email address), what happens?

Code: Select all

/usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
It worked for me, and there's no reason to believe it won't work for you since your services work, but just in case. Also, can you attach or put in code blocks your nagios.cfg?
gohanman
Posts: 4
Joined: Sat Dec 13, 2014 8:39 am

Re: Can't Get Host Notifications To Work

Post by gohanman »

It looks like is was max_check_attempts. My own fault for not setting that low enough and/or not being patient enough. Thanks for the pointers.
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Can't Get Host Notifications To Work

Post by tmcdonald »

Did you have any further (related) questions or are we okay to close this thread?
Former Nagios employee
Locked