Page 2 of 2
Re: Allow service notifications when host status is DOWN
Posted: Sun Mar 20, 2016 3:45 am
by emarton
tmcdonald wrote:In addition to what
@tgriep said, I dug into the notification code and found this:
https://github.com/NagiosEnterprises/na ... ons.c#L567
I could have sworn as well that service notifications could go out when the host is down, but we must have been collectively mistaken. This is something I might consider adding in a flag for.
I am going to check in with the devs on this and get confirmation that this is in fact expected behavior.
Thank you guys for the insight in this, i knew for sure at some older version the host status didn't matter when sending service notification, which was indeed annoying and generated a lot of emails in case the host was down.
In the setup i'm doing now, I'm not that much interested in the host status, but i need to get notifications for each service under the host regardless of the host state.
Adding a flag or some optional setting though which this behavior can be controlled would be excellent.
Please update me back once you hear from the devs, if there will be no change to the current behavior then I would need to go with event handlers to get the desired behavior.
Thanks.
Re: Allow service notifications when host status is DOWN
Posted: Mon Mar 21, 2016 2:09 pm
by lmiltchev
In the setup i'm doing now, I'm not that much interested in the host status, but i need to get notifications for each service under the host regardless of the host state.
If you don't care about the host's state, you could disable active checks for this host. Make sure the host is in "UP" state. If not - you can always submit a passive check result ("UP") to change the state.
Alternatively, you could replace the existing host check command with "check_dummy", and pass "0" as an argument.
In both cases, the host's status will always be "UP", and service notifications will be sent.
Adding a flag or some optional setting though which this behavior can be controlled would be excellent.
Please update me back once you hear from the devs...
This will take some time to get implemented (if ever). You are welcome to post a feature request on our tracker here:
http://tracker.nagios.org
Re: Allow service notifications when host status is DOWN
Posted: Mon Mar 21, 2016 2:19 pm
by eloyd
Bingo. That's what I'd do if you don't want to write event handlers. Use the check_dummy command to always have the host come back as OK. You can use a check_ping service check to actually tell you if the host is not reachable (since that's also the default host check).
Then you'll get service notifications every time, regardless of the state of the host.
If you want to get notified for EVERY service check result when it changes state (not just after Nagios decides it's been bad long enough to warrant a notification) you'll want to look at the is_volatile service definition parameter as well.
Re: Allow service notifications when host status is DOWN
Posted: Tue Mar 22, 2016 10:58 am
by bwallace
Looks like lmiltchev and eloyd have come up with a solution - were there any other questions or may we lock this thread?
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 12:10 am
by emarton
Just one last question,
I was testing the event handlers in an attempt to replicate the default nagios notification.
So I added the following in the configuration:
Command:
Code: Select all
define command {
command_name event-notify
command_line $USER1$/eventhandlers/event-notify $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $NOTIFICATIONTYPE$ $SERVICEDESC$ $HOSTALIAS$ $HOSTADDRESS$ $LONGDATETIME$ $SERVICEOUTPUT$ $NOTIFICATIONTYPE$ $CONTACTEMAIL$
}
Service:
Code: Select all
define service {
host_name xxx.xxx.com
service_description Check Multi Event
check_period 24x7
check_command check_multi_icmp!20,20%!100,50%!xx.xx.xx.xx,xx.xx.xx.xx!0!0
event_handler event-notify
contact_groups customer1group
notification_period 24x7
initial_state o
importance 0
check_interval 1.000000
retry_interval 1.000000
max_check_attempts 3
is_volatile 0
parallelize_check 1
active_checks_enabled 1
passive_checks_enabled 1
obsess 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options a
freshness_threshold 0
check_freshness 0
notification_options r,w,u,c,s
notifications_enabled 1
notification_interval 10.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
}
I modified the event handler example for the doc to this:
Code: Select all
#!/bin/sh
# What state is the HTTP service in?
case "$1" in
OK)
;;
WARNING)
;;
UNKNOWN)
;;
CRITICAL)
case "$2" in
SOFT)
;;
HARD)
# Notify contacts, service in Hard Critical state, using notify-by-email command from nagios
/usr/bin/printf "%b" "***** Nagios *****\\n\\nNotification Type: $4\\n\\nService: $5\\nHost: $6\\nAddress: $7\\nState: $1\\n\\nDate/Time: $8\\n\\nAdditional Info:\\n\\n$9" | /bin/mail -s "** $10 Service Alert: $6/$5 is $1 **" $11
;;
esac
;;
esac
exit 0
The script is executed correctly, when the service state goes to critical / hard, however the variables are not passed over (except the first 3). I know the CONTACTEMAIL, NOTIFICATIONTYPE and some other macros are not available for service event handlers, but is there any way to get the same service notification as nagios has using event handlers ?
I do have this in the main nagios configuration, not sure if that's the cause though: enable_environment_macros=0
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 5:07 am
by emarton
Actually I found the answer to the problem, you cannot use contact macros for service event handlers.
The only way I found to pass over contacts is by using the $CONTACTGROUPMEMBERS:customer1group$ on-demand macro, however for this the contact group name needs to be hardcoded in the event handler command, so this is not a solution for me as I would need to create individual commands for each service event handler.
From what I found you cannot pass $ARGn$ to event_handler definition inside the service definition, any ideas if this can be done through some other method ?
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 7:07 am
by eloyd
You can use the nagios API to query the list of contacts for a particular service if you're using 4.0.7 or higher. See
https://labs.nagios.com/2014/06/19/expl ... -7-part-1/ for details. You're going to hate me for this, but once you master this sort of thing below, then all the configuration information within Nagios is available to you at the command line as well.
Here is an example to grab the JSON data that shows the contacts for a service:
Code: Select all
http://NAGIOSSERVER/nagios/cgi-bin/objectjson.cgi?query=service&hostname=SOMEHOST&servicedescription=SOMESERVICE
This will return JSON data that looks like this the big list I've posted at the bottom. You could then parse this in your event handler to extract the line after "contacts" (in bash, that would be: contacts=`echo $JSON_OUTPUT | egrep -A 1 "\"contacts\""`), and then use the same APIs to go get each contact's email address with:
Code: Select all
http://NAGIOSSERVER/nagios/cgi-bin/objectjson.cgi?query=contact&contactname=SOMECONTACT
Then grab the "email" field, which would be an email addres and use it in your event handler to send email.
Code: Select all
http://NAGIOSSERVER/nagios/cgi-bin/objectjson.cgi?query=service&hostname=SOMEHOST&servicedescription=SOMESERVICE
{
"format_version": 0,
"result": {
"query_time": 1458734327000,
"cgi": "objectjson.cgi",
"user": "nagiosadmin",
"query": "service",
"query_status": "released",
"program_start": 1458643403000,
"last_data_update": 1458643403000,
"type_code": 0,
"type_text": "Success",
"message": ""
},
"data": {
"service": {
"host_name": "SOMEHOST",
"description": "SOMESERVICE",
"display_name": "SOMESERVICE",
"check_command": "check_nrpe!check_init_service!httpd",
"event_handler": "",
"initial_state": 0,
"check_interval": 5,
"retry_interval": 1,
"max_attempts": 4,
"parallelize": true,
"contact_groups": [
"voip-support"
],
"contacts": [
"SOMECONTACT"
],
"notification_interval": 60,
"first_notification_delay": 0,
"notifications_options": 15,
"stalking_options": 0,
"is_volatile": false,
"notification_period": "24x7",
"check_period": "24x7",
"flap_detection_enabled": true,
"low_flap_threshold": 0,
"high_flap_threshold": 0,
"flap_detection_options": -1,
"process_performance_data": true,
"check_freshness": false,
"freshness_threshold": 0,
"accept_passive_checks": true,
"event_handler_enabled": true,
"checks_enabled": true,
"retain_status_information": true,
"retain_nonstatus_information": true,
"notifications_enabled": true,
"obsess": true,
"hourly_value": 0,
"parents": [
],
"children": [
],
"notes": "",
"notes_url": "",
"action_url": "",
"icon_image": "",
"icon_image_alt": "",
"custom_variables": {
}
}
}
}
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 7:22 am
by emarton
Thanks Eric,
I never though about the API
We do have nagrestconf installed and we are going to use that to setup services. hosts etc
And yes, it can also be used to get the configuration values in json just like you mentioned with the nagios API.
Thank you all for the help on this, thread can be locked.
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 7:29 am
by eloyd
Excellent! Glad I could help. Feel free to nominate me for an MVP award like it says in my signature. See you at the Nagios World Conference in September!
Re: Allow service notifications when host status is DOWN
Posted: Wed Mar 23, 2016 9:15 am
by rkennedy
Thanks @eloyd!
Closing this one out, but feel free to make a new thread if you ever need any assistance in the future!