Sending events when downtime ends

[Fred Kroeger]

Is there a way to trigger sending any active events when a scheduled downtime ends?
The scenario I have is that we want to suppress sending any events during a scheduled downtime but want an alert generated if the host/service is "not OK" when the downtime ends.

regards... Fred

[slansing]

This should happen by default, when nagios runs the next scheduled check on your hosts that were in downtime it should pick the critical up and push out a notification just as it normally would, is this not happening in your environment? It happens this way because in downtime your hosts will undoubtedly be in a critical or unreachable state unless they are not actually down, thus, if you were to send a notification immediately after leaving downtime and they were still coming back up you would essentially get a false notification.

[Fred Kroeger]

Thanks - I need to do some further testing then as I am relying on a Global event handler instead of notifications. Does the Global event handler get triggered again as well as the notifications when it comes out of downtime?
I am using the check_dummy command to generate the critical.
My testing sequence is.
1) Put Host in scheduled downtime
2) Change the check_dummy parameter from 0 to 2 (Check interval 5min)
3) Apply config

In the event log I can see:
- Scheduled downtime starting
- Critical alert for Host
- Critical alert processed by Global event handler
- Scheduled downtime stopping

There are no further messages to indicate that the Global Event handler has received another event after that.
BTW - the event handler ignores any event that is in downtime, which is why I need to send another event if it is still active when it comes out of downtime.

Fred

[sreinhardt]

Ya the event handler would not do this, simply because it is only going to get called when a state change occurs or if a host\service has isvolatile set making each return a state change. Unfortunately, I don't think simply setting a contact with notifications will truly do what you want without duplicating traps that are sent.

[Fred Kroeger]

Thanks Spenser
Would it be possible to setup the scheduled downtime start/end routines to trigger the event handler?
ie: When Downtime starts and ends, it send the current state.

The event handler I am using interfaces into HP's SM9 to generate tickets. So we now have a situation where we could have a critical event but nobody knows about it as we don't use the notifications. How does the Nagios IM handle this scenario?

reagrsd... Fred

[abrist]

If you were using the notification handlers, you could send a notification when downtime ends, parse the email and take action.
You could pass $HOSTDOWNTIME$ or $SERVICEDOWNTIME$ macros to your event handler script to check the objects downtime depth.

IM handles this through an integrated XI component, with considerable logic for these types of issues. I will have to look into the php more before I have a more specific answer.

[Fred Kroeger]

The event handler that HP provides works exactly as it should, in that it is passing the downtime status along with the state so that the end process can determine whether to take any action or not.
I don't think any condiderable logic is required for this - all I need is the event handler to be triggered when downtime ends. That way the current state will always be sent.

regards... Fred

[sreinhardt]

The only way that I could think to do this, would be to create a contact specifically for it, use your event handler as the host and service notification command probably with some minor changes to how you call it, and finally set this contact to only ever notify on downtime actions. Unfortunately a pure event handler will not work, as they are only executed on state change, and downtime does not cause a state change the same as a check result will.

As abrist mentioned, there is a very large amount of logic with the IM component for XI. Essentially it collects and processes just about every actionable event within your nagios system, determines how it should act for both system and does so. Neither event handlers nor notifications have near this same level of logic in any form of a controllable or callable manor, that I know of. If it's not protected you are more than welcome to take a peek at the IM component and see if there is a hook or two you could implement with the snmptrap component to work better for you.

[Fred Kroeger]

I've had a bit more of a think about this and perhaps I can define my requirements a bit better.

Essentially, I would like Nagios to alert on any Non-OK Host or Service state that is active when a Host comes out of Downtime.

Because the notification was suppressed during the downtime, the Admins have no way of knowing if there was any alert apart from having to log in to Nagios to view the current status of the monitors for that host.

Is that possible? ..... Fred

[Box293]

After downtime ends this will happen automatically on the next notification_interval of the host or service object (as long as the object was not acknowledged).