Treating Unknown as Critical

[cylindric]

We have a bunch of servers that are connected to an iSCSI SAN, and we monitor them for drive-space with check_nrpe:

Code: Select all

$USER1$/check_nrpe -H $HOSTADDRESS$ -c CheckDriveSize -a MinWarnFree=10% MinCritFree=5% Drive=E

What has happened now is that we had a SAN issue that disconnected that volume, bringing down a bunch of our SQL resources. Nagios ignored it, as CheckDriveSize seems to report these as 'UNKNOWN'. Can this be configured to be treated as critical?

[eloyd]

What is the configuration line in the client's nrpe.cfg file that corresponds to the command of CheckDriveSize?

[cylindric]

There isn't one.

Code: Select all

[/modules]
CheckDisk = 1
CheckEventLog = 1
CheckExternalScripts = 1
CheckHelpers = 1
CheckNSCP = 1
CheckSystem = 1
CheckWMI = 1
NRPEServer = 1
NSClientServer = 1
 
[/settings/default]
allowed hosts = 10.10.0.6
 
[/settings/NRPE/server]
allow arguments = true
allow_nasty_meta_chars = 1

[eloyd]

Somewhere on the client there needs to be a configuration file that tells NRPE what UNIX command(s) to execute when it is told to execute NRPE command "CheckDriveSize." Unfortunately, I do not recognize the format of the file you listed below, so I don't know where that is. Find it and try to find the line that talks about CheckDriveSize.

[cylindric]

The command works fine for any parameters that don't consist of a string, so I know the command is working in general. That's a standard NSClient++ config file.

[eloyd]

That's a standard NSClient++ config file.

True, but you never said you were using NSclient++. It makes more sense now.

Okay. It's been a long time since I've played with NSclient++, but I'm thinking that you need the : at the end of your drive specification. Try:

Code: Select all

$USER1$/check_nrpe -H $HOSTADDRESS$ -c CheckDriveSize -a MinWarnFree=10% MinCritFree=5% Drive=E:

(note the colon).

A typical Nagios config for this is:

Code: Select all

define command {
  command_name check_driveSize
  command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c CheckDriveSize -a ShowAll MinWarnFree=$ARG1$ MinCritFree=$ARG2$ Drive=$ARG3$
}

Note the "ShowAll." Then you pass in 10%!5!e: as your parameters. So two suggestions:

1. Try adding the colon
2. Try adding the ShowAll

[millisa]

The scenario doesn't sound wrong to me - you can't check drive size on a drive if the drive is not known. So, the check *should* come back as unknown. If you want notifications for a service in an unknown state, you need to set it with 'notification_options' for the service definition. It wouldn't surprise me if you have your notification options set to just 'w,c,r' for warning, critical recovery. You probably want w,c,r,u to get unknown alerts too. Quick excerpt on 'notification_options' from the service definition section of http://nagios.sourceforge.net/docs/3_0/ ... ml#service (emphasis mine) :

This directive is used to determine when notifications for the service should be sent out. Valid options are a combination of one or more of the following: w = send notifications on a WARNING state, u = send notifications on an UNKNOWN state, c = send notifications on a CRITICAL state, r = send notifications on recoveries (OK state), f = send notifications when the service starts and stops flapping, and s = send notifications when scheduled downtime starts and ends. If you specify n (none) as an option, no service notifications will be sent out. If you do not specify any notification options, Nagios will assume that you want notifications to be sent out for all possible states. Example: If you specify w,r in this field, notifications will only be sent out when the service goes into a WARNING state and when it recovers from a WARNING state.

Code: Select all

define service {
  use                               sometemplate_services
  host_name                         someservername
  check_command                     check_nrpe_yourcustomcommandname!awesomeargument1
  service_description               NRPE-disk-withcheese
  notification_options              w,c,r,u  #note, this would override a notification_options defined in the sometemplate_services defined in the 'use' line above
}

[cylindric]

Thanks, I've enabled "U" notifications now too. I also managed to link the failure state with an eventlog entry, so I now monitor for those too.

[tmcdonald]

cylindric, are we good to close the thread?

[cylindric]

Sure.