Disk Space - Reporting Error (incorrect)

[jcmanous]

After following the directions provided, I received this output:

Code: Select all

/usr/local/nagios/libexec/check_snmp_storage_wizard.pl -H xxx.xxx.xxx.xxx -C public --v2c -m "^/data$" -w 80 -c 95 -f
/data: 154%used(2761941MB/1794802MB) (>95%) : CRITICAL | '/data'=2761940.8789MB;1435842;1705062;0;1794802

I rolled back to the previous version of the plugin.

[jdalrymple]

We need to debug deeper and make sure this is even a plugin problem. Kindly provide the output of:

Code: Select all

snmpwalk -v2c -c public <yourhost> 1.3.6.1.2.1.25.2.3.1

[jcmanous]

Output as requested:

Code: Select all

HOST-RESOURCES-MIB::hrStorageIndex.1 = INTEGER: 1
HOST-RESOURCES-MIB::hrStorageIndex.3 = INTEGER: 3
HOST-RESOURCES-MIB::hrStorageIndex.6 = INTEGER: 6
HOST-RESOURCES-MIB::hrStorageIndex.7 = INTEGER: 7
HOST-RESOURCES-MIB::hrStorageIndex.10 = INTEGER: 10
HOST-RESOURCES-MIB::hrStorageIndex.31 = INTEGER: 31
HOST-RESOURCES-MIB::hrStorageIndex.35 = INTEGER: 35
HOST-RESOURCES-MIB::hrStorageIndex.36 = INTEGER: 36
HOST-RESOURCES-MIB::hrStorageIndex.37 = INTEGER: 37
HOST-RESOURCES-MIB::hrStorageType.1 = OID: HOST-RESOURCES-TYPES::hrStorageRam
HOST-RESOURCES-MIB::hrStorageType.3 = OID: HOST-RESOURCES-TYPES::hrStorageVirtualMemory
HOST-RESOURCES-MIB::hrStorageType.6 = OID: HOST-RESOURCES-TYPES::hrStorageOther
HOST-RESOURCES-MIB::hrStorageType.7 = OID: HOST-RESOURCES-TYPES::hrStorageOther
HOST-RESOURCES-MIB::hrStorageType.10 = OID: HOST-RESOURCES-TYPES::hrStorageVirtualMemory
HOST-RESOURCES-MIB::hrStorageType.31 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.35 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.36 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.37 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: Physical memory
HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: Virtual memory
HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: Memory buffers
HOST-RESOURCES-MIB::hrStorageDescr.7 = STRING: Cached memory
HOST-RESOURCES-MIB::hrStorageDescr.10 = STRING: Swap space
HOST-RESOURCES-MIB::hrStorageDescr.31 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.35 = STRING: /dev/shm
HOST-RESOURCES-MIB::hrStorageDescr.36 = STRING: /boot
HOST-RESOURCES-MIB::hrStorageDescr.37 = STRING: /data
HOST-RESOURCES-MIB::hrStorageAllocationUnits.1 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.3 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.6 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.7 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.10 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.31 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.35 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.36 = INTEGER: 1024 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.37 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageSize.1 = INTEGER: 396857040
HOST-RESOURCES-MIB::hrStorageSize.3 = INTEGER: 704057036
HOST-RESOURCES-MIB::hrStorageSize.6 = INTEGER: 396857040
HOST-RESOURCES-MIB::hrStorageSize.7 = INTEGER: 43123072
HOST-RESOURCES-MIB::hrStorageSize.10 = INTEGER: 307199996
HOST-RESOURCES-MIB::hrStorageSize.31 = INTEGER: 50363640
HOST-RESOURCES-MIB::hrStorageSize.35 = INTEGER: 49607130
HOST-RESOURCES-MIB::hrStorageSize.36 = INTEGER: 487652
HOST-RESOURCES-MIB::hrStorageSize.37 = INTEGER: 459469312
HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 49038464
HOST-RESOURCES-MIB::hrStorageUsed.3 = INTEGER: 49038464
HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 318212
HOST-RESOURCES-MIB::hrStorageUsed.7 = INTEGER: 43123072
HOST-RESOURCES-MIB::hrStorageUsed.10 = INTEGER: 0
HOST-RESOURCES-MIB::hrStorageUsed.31 = INTEGER: 42551841
HOST-RESOURCES-MIB::hrStorageUsed.35 = INTEGER: 1
HOST-RESOURCES-MIB::hrStorageUsed.36 = INTEGER: 80493
HOST-RESOURCES-MIB::hrStorageUsed.37 = INTEGER: 707056865

[jcmanous]

I did a little extra digging and found this:

Linux xfs 16TB filesystem with 4096 bytes large blocks will be reported as hrStorageAllocationUnits = 8192 and hrStorageSize = 2147483647, so 8192 x 2147483647 gives real size of the filesystem (=16 TB).

Setting this directive to '1' (=default) turns off this calculation and the agent reports real hrStorageAllocationUnits, but it might report wrong hrStorageSize for big drives because the value won't fit into Integer32. In this case, hrStorageAllocationUnits x hrStorageSize won't give real size of the storage.

Origination Link - http://linux.die.net/man/5/snmpd.conf

[jdalrymple]

The math is simple enough - autodectecting it isn't (possible as far as I can see).

If it were my box I'd switch to NRPE and check_disk, of course if it were MY box that's where I would have started. If that's not an option for you, there will be a wrapper script and some manual intervention involved when it comes to 16+TiB XFS filesystems.

[jcmanous]

Fair enough. Typically I tend to lean towards "pure" SNMP but I can see there are a few limitations.

In any event, I'll give this a go and see what I can find out and let you know.

[lmiltchev]

In any event, I'll give this a go and see what I can find out and let you know.

Sure. We will keep the thread open.

[jcmanous]

So, as it turns out this has sorted out the reporting problem.

I do have one question but it's not a deal-breaker - The nrpe plugin shows remaining % of space available (as opposed to what's actually in-use). Can this be reversed?

Again, not a big deal but I was curious.

[lmiltchev]

This particular plugin checks the amount of used disk space on a mounted file system and generates an alert if free space is less than one of the threshold values. There are some other 3rd party plugins that will show the used disk space. For example, I found one such plugin on Nagios Exchange here:

https://exchange.nagios.org/directory/P ... ce/details

I renamed the plugin to "check_disk_used" and copied it over to my remote box. I created a command in the "/usr/local/nagios/etc/nrpe/common.cfg":

Code: Select all

command[check_disk_used]=/usr/local/nagios/libexec/check_disk_used $ARG1$

restarted xinetd:

Code: Select all

service xinetd restart

and tested it from the XI server:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H x.x.x.x -t 30 -c check_disk_used -a '-d / -w 85 -c 90'
OK - / space used=81% | '/ usage'=81%;85;90;

Does this look like something that you might use?

[jcmanous]

I will give that a shot and see how that works. In general, I think that it would work fine.

By the way, I'm consideirng this resolved so please close the case if you like.

Many thanks.