Page 1 of 1

Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:15 pm
by mbeebe
Hello,

We're using a service handler to send alerts to a third-party product when a critical alert fires. Our general service monitoring setup is to check every 15 minutes, then if there's a problem, check once every minute for five minutes before firing an alert.

We ran into an issue today: the service handler in question was firing every minute on SOFT alerts instead of only once for the HARD alert. This resulted in a ton of spam being sent to the team and on-call person.

Is there any way to have the service handler only fire on the final HARD alert?

As a nb: the service check in question is binary: it's either CRITICAL or it's not; there's no WARN state. I don't know if this makes a difference, but I thought I'd throw it out there anyway.

Thanks,

-- Mike Beebe

Re: Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:20 pm
by ssax
What version of XI are you running? You can grab it from the bottom left hand side of the web interface.

You will need to update your event handler to not fire if it's a SOFT state, you would need to use the available macro such as $SERVICESTATETYPE$ which indicates HARD or SOFT to determine that, see here for all available macros and when they are available:

https://assets.nagios.com/downloads/nag ... estatetype

Re: Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:23 pm
by mbeebe
Hello,

We're running 5.6.7.

I'll look into that macro.

Thanks,

-- Mike Beebe

Re: Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:27 pm
by ssax
Should be good with that version.

How are you running the event handler? From the service or from Admin > Manage Components > Global Event Handlers?

Re: Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:38 pm
by mbeebe
The event handler is set in the service management for the services in question.

Re: Service handler triggered by SOFT alerts

Posted: Mon Oct 14, 2019 4:54 pm
by ssax
Ok, yeah, the problem is in your script then, please update it to look at the macro I previously referenced.

Re: Service handler triggered by SOFT alerts

Posted: Tue Oct 15, 2019 1:44 pm
by mbeebe
Hi Ssax,

That worked great -- a little simple logic in the script and we're now not seeing the additional alert emails.

Thanks!

-- Mike Beebe

Re: Service handler triggered by SOFT alerts

Posted: Tue Oct 15, 2019 2:45 pm
by benjaminsmith
Hi Mike,

Glad to hear that worked. Did you have any other questions or may we close this thread?

Re: Service handler triggered by SOFT alerts

Posted: Wed Oct 16, 2019 1:14 pm
by mbeebe
Nope -- please close the thread, thanks.