Page 1 of 1

Custom service notification "exception"

Posted: Tue Jun 05, 2012 10:20 am
by blueline
I'm hoping to avoid a hacked together mishmash to achieve something. I know it can be done with a mishmash but let's see if I'm missing a SIMPLE, easy way.

I have a service. That service is checked 24x7x365. Notifications are sent 24x7x365, on WARNING and also on CRITICAL.

That is good--that is what I want.

However...now I want one single exception to that *notification* setup. NOTE: I do not want an exception to the *monitoring* setup--I want the console to always show the correct status, 24x7. I just want to make one exception for the *notification* (via email) on this service.

Here is the exception:

IF service state is WARN *AND* time of day is between 0300 and 0600, do NOT notify.

That's it. If it's CRITICAL, email-notify 24x7 (as it already does). If it's not between 3 and 6 a.m., notify regardless of WARN vs. CRIT (as it already does). The only exception is WARNING *and* 3-6 a.m.

Background: This is because we have maintenance that occurs every night between 3 and 6, which we've customized to produce a WARNING (not CRITICAL). I want notifications any time outside of this (admin may have accidentally launched maint in middle of day), and I want CRITICAL *any* time. I don't want to simply skip CHECKS during that time because I do want the console to be correct (a big bunch of yellows 0300-0600).

So, anyway, seems like I can kludge together a bunch of constructs but does anybody have a simple way to define this one "boolean AND" condition to the notification (only) schedule?

Thanks,
D

Re: Custom service notification "exception"

Posted: Tue Jun 05, 2012 7:37 pm
by jsmurphy
There are a few ways you could probably go about this, but none of them are particularly nice.

I think the solution I would go with faced with the same issue where this is my only option, is create a script that runs when this would notify that checks the state and time before sending the notification. It's probably the easiest to implement and doesn't require a configuration hodge-podge.

Re: Custom service notification "exception"

Posted: Wed Jun 06, 2012 10:23 am
by blueline
Ah yes, thank you. Excellent. We have a custom script with custom Nagios command for determining the status itself, but I'd forgotten we can actually create a custom *notify* script as well. That would do it just fine. Thanks!

Re: Custom service notification "exception"

Posted: Wed Jun 06, 2012 11:34 am
by agriffin
Good to see that everything worked out!