Alert on value (not status) change

[KurtAlden]

Hello all

We have an application that one of my customers wants an alert setting up for which is fine, however the requirement is a little bit complicated. Instead of being alerted on a status change, they want to know if the numerical value of the service changes. It is kind of their work queue and they might have, say, 3 things in there and they'd like to know when a new one arrives. I've not been able to crack this in Nagios, I'm not sure if it's even possible as, to be fair, it's not really a standard function. I was wondering though if anyone had any thoughts as to how I might be able to accomplish this. NB The client uses Nagstamon.

Cheers in advance
K

[tmcdonald]

Is there some sort of string example you could give for what you are testing against? For example, is it displaying "We have 6 jobs in the queue" and you want to know when "6" becomes "7"?

[mickem]

NSClient++ supports stateful scripts so it could be done there Using lua or python).
But other options would be to manage state manually outside Nagios.

// Michael Medin

[slansing]

Is this application "service" on a Windows system or a *nix based system?

[tmcdonald]

But other options would be to manage state manually outside Nagios.

I was going to suggest this, but wanted to make sure that was the right solution beforehand.

I am working on documentation right now for a utility called bischeck. It was made to monitor dynamically changing data, such as bandwidth throughout the day. It's a bit tricky, but you might be able to use it for what you need. I have been testing it locally using dummy data, so if I can get a few samples of what you're actually looking at I could maybe get you pointed in the right direction.

[KurtAlden]

Hello, thank you for your replies

The text response is "WARNING: Search resulted in 8 documents (warning is >= 0 documents, critical is >= 99999 documents)" and the performance data is "Count=8"

I am using the check_domino plugin to get this data, it is the number of documents in a Domino database from a view search.

So, what I am trying to do is alert when the value changes. Managing state change outside Nagios is interesting but that gave me another thought. Instead of running a check_domino call as a service, I can run a perl script instead and call it from a new command which passes the value the last time it ran using the $SERVICEPERFDATA$ on-call macro. The perl script then runs the check_domino plug-in, gets the perf data value and compares to the $SERVICEPERFDATA$ value. The perl script can then return warn/crit if these values are different. Think that'll crack it, I'll let you know if you're interested.

[tmcdonald]

I'd be interested in seeing what you come up with. One problem I ran into was that yes, you can get it to alert on state change, however the next time the check is run it will go back to OK since the state is now considered the same.

[KurtAlden]

Hello

I'm back to working on this as we now have a specific requirement in a very similar vein to check that a value has changed since the last check. As in, if the last check returns 50 then the next check has to return 51 or over. I am trying to avoid having to modify the check_domino plugin and I'd rather try to avoid writing perl scripts too, I've come up with a solution but there's a wrinkle that I can't solve. I'm trying to use the $SERVICEPERFDATA$ macro as a paramater.

So, instead of using

host_name TestHost
service_description TestService
check_command <general paramaters> -w 50 -c 100

I thought I could do...

host_name TestHost
service_description TestService
check_command <general paramaters> -w $SERVICEPERFDATA:TestHost:TestService$ -c 100

I thought that would, in theory, send the last value returned as the warning threshold so that if the last values is the same as the current value it fires. However, of course the performance data also includes the text of the performance data, ie "Count=5" instead of just "5" and my plugin falls over as it's expecting a value it can parse as an integer not a string. Do you have any idea as to how I might be able to solve this without a Perl script or having to modify my plugin?

Also, you've mentioned that you got this working, could you tell me how please?

[tmcdonald]

The documentation I wrote is available here: https://assets.nagios.com/downloads/nag ... ios-XI.pdf

But I wrote that over 2 years ago and it's not at all fresh in my mind and bischeck has changed quite a lot since then so you'll need to use the exact versions mentioned in the doc to get it working.

Honestly though, I think the easiest route is going to be to modify your plugin. Have it run the check and save the data to a file in /tmp. Then the next time it runs, compare the current data with what was saved to disk. That's how a lot of other plugins do it and it's not difficult at all.