Disk space check that alert on % rate off growth

[twelsh]

Hi All,

First post in a while and I'm on the search for some information.

Nagios Environment:
Nagios 3.2.1
PnP4Nagios 0.6.19-R.36
OS - RHEL 5.8

Scenario:
I have a mixed Operating Systems environment consisting of Linux, AIX, and Windows 2000/2003/2008.

From time to time we have a situation where any one of our servers will suddenly eat through all its disk space in a period of time and by the time someone notices the critical alert has gone off its too late to do remedial actions. Yes we could use warnings but they are still struggling to get the concept of dealing with critical alerts in a timely manner.

We want to look at the rate of growth on a disk and see if it is trending at a level where the disk may fill up IF its projected trajectory continues.

Yes, this may raise a few false positives as we tune our timeperiods / growth rates but we feel it may be useful in the long term.

I wish to create a new Nagios check/plugin to measure the trajectory of disk space use over a period of time.

check_disk_trajectory

This plugin looks at the rate of disk space consumption over a user defined time period and alerts if this threshold breaches. im thinking of something like the following

./check_disk_trajectory -H HOSTNAME -p <disk/partition> -t <trajectory time period specified in minutes> -w <warning level in % change> -c <critical level in % change>

Question:
1. Has anyone implemented something like this before?
2. Is there a plugin that does this already?
3. I am thinking that as I already use PnP4Nagios there may be an option to do some RRD calculations on the back off an event handler after the scheduled disk space check has been carried out.

[sreinhardt]

Hi twelsh,

This is an excellent question! Nagios XI does contain the ability to extrapolate various usage statistics through existing data. It is actually built into the reporting section. To my knowledge, we do not currently have a plugin to create warnings off the projected data. Without reading the code, I do believe that our current calculations do exactly as you thought, work through the existing rrd data and base future usage from that. I will speak with one of our developers this week and see if he has anything in the works or already done, and if not, if he could add this to his queue. This would be a great check to have!

[twelsh]

Hi Sreinhardt,

Thanks for the reply. I use Nagios Core, but have used Xi in the past.

I don't really want to do reports on each drive/partition across my 1000's of hosts. This would just get get really silly.

A plugin makes much more sense as it works directly with the Nagios interface and we can also view alerts etc via Nagstamon.

A plugin could also be configured look in specific directories for the RRD data

Regards

Tom

[sreinhardt]

I completely agree, running reports on all those devices would be rather silly, and a whole lot of work. I should have been more specific, but I was going to ask if he has a plugin or could add it to his list, not create additional reports. I will report back when I have an update for you!

[twelsh]

Hi Sreinhardt

Any news on whether you developer has the code that would enable me to do this or is it in the work queue?

If i could see the code i may be able to knock something together.

Ta in Advance

Tom

[slansing]

The view that you see when you look at extrapolated data in XI is perf data read into High Charts and parsed with the Holt-Winters algorithm, I believe that is a part of the high charts suite but we have only built this functionality into XI and not core. So in essence, it's not really a plugin or a snippet of code that would work in core necessarily. Are you interested in writing one? We could pass you some documentation and guidelines on the subject.

[twelsh]

Hi Slansing,

Yep, if you dont have code to do it in core, or a plugin then i will look at doing it in a plugin. Any guidelines /docs would be really helpful.

Regards

Tom

[slansing]

Unfortunately since this is something only available in the Enterprise version of Nagios XI, "Capacity Planning" we do not have any documents or guidance on how to convert this to Nagios Core. A lot of the moving parts involved are within Nagios XI and not Nagios Core.