Disable Recovery Notifications After Warning State

[bferris]

Hello,

I am trying to find a way to stop receiving RECOVERY notificaitons when a service/host goes from WARNING to OK

I still want to get notifications of RECOVERY when a service/host goes from CRITICAL to OK.

Any ideas?

[cdienger]

My initial thought is two checks - one that sends out warning notifications and the other that sends out critical/recover notifications.

[bferris]

My initial thought is two checks - one that sends out warning notifications and the other that sends out critical/recover notifications.

I am not sending any warning notifications. However I still receive Recovery notifications.

I noticed this because I am getting Recovery notifications with out ever getting a Problem notification.


Here is a service definition from one of my hosts:
define service {
host_name HOST1
service_description Memory Usage
check_command check_ncpa!-t '#########' -P 12489 -M memory/virtual -w 78 -c 98 -u Gi
max_check_attempts 5
check_interval 5
retry_interval 1
check_period 24x7
notification_interval 5
notification_period 24x7
notification_options u,c,r
contacts nagiosadmin,pagerduty
register 1
}

Here is from my notifications or contacts config file:

define contact {

name notification-contact ; The name of this contact template
service_notification_period 24x7 ; service notifications can be sent anytime
host_notification_period 24x7 ; host notifications can be sent anytime
# service_notification_options c,r,f,s ; send notifications for all service states, flapping events, and scheduled downtime events
host_notification_options d,u,r,f,s ; send notifications for all host states, flapping events, and scheduled downtime events
service_notification_commands notify-service-by-email ; send service notifications via email
host_notification_commands notify-host-by-email ; send host notifications via email
register 0 ; DON'T REGISTER THIS DEFINITION - ITS NOT A REAL CONTACT, JUST A TEMPLATE!
}

define contact {

contact_name nagiosadmin ; Short name of user
use notification-contact ; Inherit default values from generic-contact template (defined above)
alias Nagios Admin ; Full name of user
email [email protected] ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
service_notification_options u,c,r,f,s
}

[scottwilkerson]

I'm going to propose something a little different...

Change your service notification command to something like this using the $LASTSERVICESTATE$ to

Code: Select all

[ "$LASTSERVICESTATE$" != "WARNING" ] && /usr/bin/printf "%b" "***** Nagios Monitor XI Alert *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$

This will check to see if the last service state is not WARNING before continuing to to send emails.

[bferris]

I'm going to propose something a little different...

Change your service notification command to something like this using the $LASTSERVICESTATE$ to

Code: Select all

[ "$LASTSERVICESTATE$" != "WARNING" ] && /usr/bin/printf "%b" "***** Nagios Monitor XI Alert *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$

This will check to see if the last service state is not WARNING before continuing to to send emails.

scottwilkerson - THANK YOU VERY MUCH! This look exactly like what I was thinking. I will set it up and let it run over the weekend to see if there are any issues.

[scottwilkerson]

Sounds good!

[bferris]

Sounds good!

Scott - it looks like this was the fix I needed. Thank you very much!

I am no longer getting the notification emails.

However, I am still seeing log entries in nagios.log that show notifications were sent.

Under the Notifications area in Nagios I see that an entry was logged saying it did sent out a recovery notification.


NOT a big deal at all, it seems to be working they way I would like it to but I just wanted to see if that was expected.

Thanks

[scottwilkerson]

Sounds good!

Scott - it looks like this was the fix I needed. Thank you very much!

I am no longer getting the notification emails.

However, I am still seeing log entries in nagios.log that show notifications were sent.

Under the Notifications area in Nagios I see that an entry was logged saying it did sent out a recovery notification.


NOT a big deal at all, it seems to be working they way I would like it to but I just wanted to see if that was expected.

Thanks

Yes, this change just prevents the notification from actually sending but they would still be logged in the notification log. There isn't really any way around this as the notification is logged when the notification command is fired, this update is just short-circuiting the actual sending of the email.