Page 1 of 1

Dynamic Thresholds

Posted: Thu Jan 01, 2015 8:37 am
by reincarne
Hi,
I'm pretty sure this question was raised before, but I will try my luck.
Is there a way to create thresholds based on time periods?
For example I want X & Y Thresholds for morning hours and X & Y thresholds for night but still remain in the same service check.
I know I can create two checks with different time periods but then I have two problems here:
1) I lose the total perf data as its devided by two different checks.
2) Sometimes the current state is stuck on critical until the next cycle of night hours. For example, my check time period ends at 7:00 AM, and the last check was critical, once I reach 7:00, this check is scheduled for the next cycle which starts at 23:00. Until 23;00 I'm stuck with critical state (I just acknowledge them).

It would be great if you add the ability to add time periods for thresholds.

Re: Dynamic Thresholds

Posted: Fri Jan 02, 2015 1:48 am
by Box293
What I would suggest here is creating a wrapper script that detects the time that the wrapper script was executed and based on that use the relevant thresholds for that time period when executing the real check.

Here is an example:
  • You have a CPU Usage check in the format "check_cpu -w 50 -c 70"
    During the day this works fine
    During night when backups happen it triggers thresholds, during this time period you would prefer "-w 80 -c 90"
    You create a wrapper script called timeperiod_wrapper_check
    It accepts two arguments, one is the name of the real check "check_cpu" and the other argument has the timeperiods and thresholds, for example "w:50:c:70:s:08:e:18,w:80:c:90:S:18:e:08"
    w: the warning threshold
    c: the critical threshold
    s: the start of the time period in 24 hour format
    e: the end of the time period in 24 hour format
    , = a seperator between the two time periods and thresholds
    So the wrapper script would be executed like: "timeperiod_wrapper_check check_cpu w:50:c:70:s:08:e:18,w:80:c:90:s:18:e:08"
    The wrapper script splits the two two time periods and thresholds (the , is the separator)
    It determines if this check is being executed between 8am - 6pm or 6pm - 8am
    With this information it executes "check_cpu -w xx -c yy" with the thresholds defined for that timeperiod
There is a bit of coding work required there however it would give you the solution you are after. This is just one of the ways you could do this.