check_disk falsely reporting 0% free Nagios Users

[salderman1]

As we continue to deploy nagios/nrpe to our Solaris SPARC database servers, I've compiled some correlating info...

Code: Select all

/# /opt/csw/libexec/nagios-plugins/check_disk -w 20% -c 10% -R "^/$|oracle$|^/u[0-9]+" -X tmpfs
DISK OK - free space: / 198350 MB (94% inode=99%); /u01 36542 MB (36% inode=99%); /u02 36542 MB (36% inode=99%); /u03 19765 MB (98% inode=99%); /u04 19765 MB (98% inode=99%); /u05 19765 MB (98% inode=99%);| /=1128
# df -h|grep u0
zpool01/u01             98G    63G    36G    64%    /u01
zpool02/u02            2.4T   1.9T   579G    77%    /u02
zpool03/u03             20G   268M    19G     2%    /u03
zpool04/u04            4.9T   2.4T   2.5T    50%    /u04
zpool05/u05            4.9T   836G   4.1T    17%    /u05

Notice above /u02, /u04 and /u04 report invalid percentages compared to the df output.

It seems to persist that all of the errant status have zfs filesystems over 1TB. I haven't found a single one of our servers with a 1TB+ zfs filesystem that does not encounter this issue.

Thanks.

[abrist]

There is a good chance this was fixed in 1.5 and beyond. But you already know this. The real trick is building the 1.5 plugins pack on your legacy hardware and there are no 1.5 bins available for your platform.

[salderman1]

There is a good chance this was fixed in 1.5 and beyond. But you already know this. The real trick is building the 1.5 plugins pack on your legacy hardware and there are no 1.5 bins available for your platform.

And even more complicated is the fact that we're using Puppet to manage nagios/nrpe packages, services, and checks via exported resources. It might be that my only recourse is to compile and replace only the check_disks plugin on a few boxes and see if the issue resolves. Otherwise Puppet is just going to point nrpe back to the original plugins package... I'll see what I can do.

[abrist]

Alright. Let us know how it goes. At least try the new version on one of the problematic boxes to see if that will resolve the issue.

[salderman1]

Alright. Let us know how it goes. At least try the new version on one of the problematic boxes to see if that will resolve the issue.

Well, I've found a SPARC T4 to build on but it's not going smoothly... Please bear with my, this is the first compiled project I've worked with from github. Here's my compile issue:

Code: Select all

Making all in plugins
make: Fatal error in reader: Makefile, line 2456: Unexpected end of line seen
Current working directory /usr/local/git/nagios-plugins/plugins
*** Error code 1
The following command caused the error:
fail= failcom='exit 1'; \
for f in x $MAKEFLAGS; do \
  case $f in \
    *=* | --[!k]*);; \
    *k*) failcom='fail=yes';; \
  esac; \
done; \
dot_seen=no; \
target=`echo all-recursive | sed s/-recursive//`; \
case "all-recursive" in \
  distclean-* | maintainer-clean-*) list='gl tap lib plugins plugins-scripts plugins-root po' ;; \
  *) list='gl tap lib plugins plugins-scripts plugins-root po' ;; \
esac; \
for subdir in $list; do \
  echo "Making $target in $subdir"; \
  if test "$subdir" = "."; then \
    dot_seen=yes; \
    local_target="$target-am"; \
  else \
    local_target="$target"; \
  fi; \
  (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make  $local_target) \
  || eval $failcom; \
done; \
if test "$dot_seen" = "no"; then \
  make  "$target-am" || exit 1; \
fi; test -z "$fail"
make: Fatal error: Command failed for target `all-recursive'
Current working directory /usr/local/git/nagios-plugins
*** Error code 1
make: Fatal error: Command failed for target `all'

I took a peek at plugins/Makefile on the line quoted and in vi it's a blank line. It doesn't appear to be problematic. Should I be checking out a specific tag before trying to build?

[abrist]

Take a look at Eris suggestion for this issue:
http://support.nagios.com/forum/viewtop ... ake#p87397
It sounds like you may need to use gmake instead of make.

[salderman1]

Seems like you're right this issue is resolved in the latest greatest. I guess I'll reach out to the OpenCSW folks and see what their timeline is or make a request. The latest 1.4 version is packages and available from all three of their repo's unstable, test and "stable" whatever they call that one now. Seems like it shouldn't be too hard to get 1.5 into unstable. In the meantime, I might just use puppet to distribute the 1.5 check_disk binary to the SPARC boxes.

Notice /u02, /u04, and u05 below. The first execution is v. 1.5, the second (faulty) execution is out of the CSW packaged 1.4...

Code: Select all

# ./check_disk -w 20% -c 10% -R "^/$|oracle$|^/u[0-9]+" -X tmpfs
DISK OK - free space: / 198350 MB (94% inode=99%); /u01 36540 MB (36% inode=99%); /u02 592830 MB (23% inode=99%); /u03 19765 MB (98% inode=99%); /u04 2562448 MB (49% inode=99%); /u05 4269915 MB (83% inode=99%);| /=11288MB;167711;188675;0;209639 /u01=64001MB;80433;90487;0;100542 /u02=1971316MB;2051316;2307731;0;2564146 /u03=267MB;16025;18028;0;20032 /u04=2566246MB;4102955;4615824;0;5128694 /u05=858723MB;4102911;4615775;0;5128639
# /opt/csw/libexec/nagios-plugins/check_disk -w 20% -c 10% -R "^/$|oracle$|^/u[0-9]+" -X tmpfs
DISK OK - free space: / 198350 MB (94% inode=99%); /u01 36540 MB (36% inode=99%); /u02 36540 MB (36% inode=99%); /u03 19764 MB (98% inode=99%); /u04 19764 MB (98% inode=99%); /u05 19764 MB (98% inode=99%);| /=11288MB;167711;188675;0;209639 /u01=64001MB;80433;90487;0;100542 /u02=64001MB;80433;90487;0;100542 /u03=267MB;16025;18028;0;20032 /u04=267MB;16025;18028;0;20032 /u05=267MB;16025;18028;0;20032
# df -h |grep u0
zpool01/u01             98G    63G    36G    64%    /u01
zpool02/u02            2.4T   1.9T   579G    77%    /u02
zpool03/u03             20G   268M    19G     2%    /u03
zpool04/u04            4.9T   2.4T   2.4T    51%    /u04
zpool05/u05            4.9T   839G   4.1T    17%    /u05

Thanks for your help!

[salderman1]

OpenCSW ticket submitted:

https://www.opencsw.org/mantis/view.php?id=5147

[abrist]

Great. Thanks for your sleuthing and patience with it.