check_memory nrpe Unable to read output

[calcazar]

Folks, I'm having a bit of an issue getting nrpe to work with one of my servers. I have CentOS 6.6, I downloaded a check_mem file, which I've attahced. I uploaded this file to everyone of my servers, centos and ubuntu and it worked fine. I've added the file to /usr/lib64/nagios/plugins and made the following changes in the configs "/etc/nagios/nrpe.cfg" for all nagios client servers. This configuration has worked on multiple CentOS servers without an issue, I've tried everything, scavenged through dozen of articles, forums and no avail. The memory command /usr/lib64/nagios/plugins/check_mem -f -w 20 -c 10 works great when run locally in the client but not from the server to the client...

Client Side:

I have the following location and permissions for this file:

Code: Select all

[root@management ~]# ls -l /usr/lib64/nagios/plugins/check_mem 
-rwxr-xr-x. 1 root root 12330 Jul  7 18:21 /usr/lib64/nagios/plugins/check_mem

Code: Select all

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=Nagios monitor server IP

dont_blame_nrpe=1

command[check_drive]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/sda
command[check_memory]=/usr/lib64/nagios/plugins/check_mem -f -w 30 -c 10

Added nrpe 5666/tcp #NRPE into the /etc/services file, the following is the debug printed out on the client side for the nrpe.

Code: Select all

Allowing connections from: xx.xx.xx.xx
Jul  8 14:37:16 api nrpe[1835]: Connection from xx.xx.xx.xx port 5784
Jul  8 14:37:16 api nrpe[1835]: Host address is in allowed_hosts
Jul  8 14:37:16 api nrpe[1835]: Handling the connection...
Jul  8 14:37:16 api nrpe[1835]: Host is asking for command 'check_memory' to be run...
Jul  8 14:37:16 api nrpe[1835]: Running command: /usr/lib64/nagios/plugins/check_mem -f -w 20 -c 10
Jul  8 14:37:16 api nrpe[1835]: Command completed with return code 3 and output:
Jul  8 14:37:16 api nrpe[1835]: Return Code: 3, Output: NRPE: Unable to read output
Jul  8 14:37:16 api nrpe[1835]: Connection from pÃ}<9c>ÿ#177 closed.

Server Side:

I can run any command from the server to client for ex. "/usr/lib64/nagios/plugins/check_nrpe -H client ip -c check_drive" and gives the expected response.

I've added the following to the command.cfg & the server.cfg, which works great.

Code: Select all

define command{
        command_name    check_space
        command_line    /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c check_drive
        }

server.cfg

Code: Select all

define service{
        use                             generic-service
        host_name                       api-webserver
        service_description             Root Partition
        check_command                   check_space
        }

I've done the same for the memory, but won't work.

Code: Select all

define command{
        command_name    check_memori
        command_line    /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c check_memory
        }

define service{
        use                            generic-service
        host_name                      api-webserver
        service_description            Memory
        check_command                  check_memori
        }

Running the command from the local client side:

Code: Select all

[root@api ~]# /usr/lib64/nagios/plugins/check_mem -f -w 20 -c 5  
OK - 84.2% (6785808 kB) free.|TOTAL=8059740KB;;;; USED=1273932KB;6447792;7656753;; FREE=6785808KB;;;; CACHES=1024644KB;;;;

When I run the commands from the command line from the server side I get

Code: Select all

[root@management ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19 -c check_drive
DISK OK - free space: / 11463 MB (86% inode=94%);| /=1716MB;11113;12502;0;13892
[root@management ~]/usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19 -c check_memory
NRPE: Unable to read output

Please help.. It's kind of annoying that it worked on every server except one!!

[abrist]

You may find this doc useful:
https://assets.nagios.com/downloads/nag ... utions.pdf
Try running the check from the remote host directly:

Code: Select all

/usr/lib64/nagios/plugins/check_mem -f -w 30 -c 10
echo $?

Next, try running it through nrpe locally:

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_memory
echo $?

Also, could you post the full nrpe.cfg from the remote host in code wraps here?

[calcazar]

/usr/lib64/nagios/plugins/check_mem -f -w 30 -c 10
echo $?

Code: Select all

[root@api ~]# /usr/lib64/nagios/plugins/check_mem -f -w 30 -c 10
OK - 84.1% (6781584 kB) free.|TOTAL=8059740KB;;;; USED=1278156KB;5641818;7253766;; FREE=6781584KB;;;; CACHES=1026604KB;;;;
[root@api ~]# echo $?
0

/usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_memory
echo $?

Code: Select all

[root@api ~]# /usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_memory 
-bash: /usr/lib64/nagios/plugins/check_nrpe: No such file or directory

I'm seeing that check_nrpe is not installed in this server, which I thought that it would be installed when I ran yum nagios nrpe nagios-plugins-all. But for comparison, none of the other centos boxes that are working have check_nrpe plugin, but still are working.

Code: Select all

#############################################################################
# Sample NRPE Config File 
# Written by: Ethan Galstad (nagios@nagios.org)
# 
# Last Modified: 11-23-2007
#
# NOTES:
# This is a sample configuration file for the NRPE daemon.  It needs to be
# located on the remote host that is running the NRPE daemon, not the host
# from which the check_nrpe client is being executed.
#############################################################################


# LOG FACILITY
# The syslog facility that should be used for logging purposes.

log_facility=daemon



# PID FILE
# The name of the file in which the NRPE daemon should write it's process ID
# number.  The file is only written if the NRPE daemon is started by the root
# user and is running in standalone mode.

pid_file=/var/run/nrpe.pid



# PORT NUMBER
# Port number we should wait for connections on.
# NOTE: This must be a non-priviledged port (i.e. > 1024).
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

server_port=5666



# SERVER ADDRESS
# Address that nrpe should bind to in case there are more than one interface
# and you do not want nrpe to bind on all interfaces.
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

#server_address=127.0.0.1



# NRPE USER
# This determines the effective user that the NRPE daemon should run as.  
# You can either supply a username or a UID.
# 
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

nrpe_user=nagios



# NRPE GROUP
# This determines the effective group that the NRPE daemon should run as.  
# You can either supply a group name or a GID.
# 
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

nrpe_group=nagios



# ALLOWED HOST ADDRESSES
# This is an optional comma-delimited list of IP address or hostnames 
# that are allowed to talk to the NRPE daemon. Network addresses with a bit mask
# (i.e. 192.168.1.0/24) are also supported. Hostname wildcards are not currently 
# supported.
#
# Note: The daemon only does rudimentary checking of the client's IP
# address.  I would highly recommend adding entries in your /etc/hosts.allow
# file to allow only the specified host to connect to the port
# you are running this daemon on.
#
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

allowed_hosts=192.168.0.26
 


# COMMAND ARGUMENT PROCESSING
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments to commands that are executed.  This option only works
# if the daemon was configured with the --enable-command-args configure script
# option.  
#
# *** ENABLING THIS OPTION IS A SECURITY RISK! *** 
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow arguments, 1=allow command arguments

dont_blame_nrpe=1



# BASH COMMAND SUBTITUTION
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments that contain bash command substitutions of the form
# $(...).  This option only works if the daemon was configured with both 
# the --enable-command-args and --enable-bash-command-substitution configure 
# script options.
#
# *** ENABLING THIS OPTION IS A HIGH SECURITY RISK! *** 
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow bash command substitutions, 
#         1=allow bash command substitutions

allow_bash_command_substitution=0



# COMMAND PREFIX
# This option allows you to prefix all commands with a user-defined string.
# A space is automatically added between the specified prefix string and the
# command line from the command definition.
#
# *** THIS EXAMPLE MAY POSE A POTENTIAL SECURITY RISK, SO USE WITH CAUTION! ***
# Usage scenario: 
# Execute restricted commmands using sudo.  For this to work, you need to add
# the nagios user to your /etc/sudoers.  An example entry for alllowing 
# execution of the plugins from might be:
#
# nagios          ALL=(ALL) NOPASSWD: /usr/lib/nagios/plugins/
#
# This lets the nagios user run all commands in that directory (and only them)
# without asking for a password.  If you do this, make sure you don't give
# random users write access to that directory or its contents!

# command_prefix=/usr/bin/sudo 



# DEBUGGING OPTION
# This option determines whether or not debugging messages are logged to the
# syslog facility.
# Values: 0=debugging off, 1=debugging on

debug=1



# COMMAND TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# allow plugins to finish executing before killing them off.

command_timeout=60



# CONNECTION TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# wait for a connection to be established before exiting. This is sometimes
# seen where a network problem stops the SSL being established even though
# all network sessions are connected. This causes the nrpe daemons to
# accumulate, eating system resources. Do not set this too low.

connection_timeout=300



# WEEK RANDOM SEED OPTION
# This directive allows you to use SSL even if your system does not have
# a /dev/random or /dev/urandom (on purpose or because the necessary patches
# were not applied). The random number generator will be seeded from a file
# which is either a file pointed to by the environment valiable $RANDFILE
# or $HOME/.rnd. If neither exists, the pseudo random number generator will
# be initialized and a warning will be issued.
# Values: 0=only seed from /dev/[u]random, 1=also seed from weak randomness

#allow_weak_random_seed=1



# INCLUDE CONFIG FILE
# This directive allows you to include definitions from an external config file.

#include=<somefile.cfg>



# INCLUDE CONFIG DIRECTORY
# This directive allows you to include definitions from config files (with a
# .cfg extension) in one or more directories (with recursion).

#include_dir=<somedirectory>
#include_dir=<someotherdirectory>



# COMMAND DEFINITIONS
# Command definitions that this daemon will run.  Definitions
# are in the following format:
#
# command[<command_name>]=<command_line>
#
# When the daemon receives a request to return the results of <command_name>
# it will execute the command specified by the <command_line> argument.
#
# Unlike Nagios, the command line cannot contain macros - it must be
# typed exactly as it should be executed.
#
# Note: Any plugins that are used in the command lines must reside
# on the machine that this daemon is running on!  The examples below
# assume that you have plugins installed in a /usr/local/nagios/libexec
# directory.  Also note that you will have to modify the definitions below
# to match the argument format the plugins expect.  Remember, these are
# examples only!


# The following examples use hardcoded command arguments...

#command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
#command[check_load]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
#command[check_hda1]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/sda
#command[check_zombie_procs]=/usr/lib64/nagios/plugins/check_procs -w 5 -c 10 -s Z
#command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w 150 -c 200 
#command[check_pings]=/usr/lib/nagios/plugins/check_icmp -H localhost
#command[check_swap_space]=/usr/lib/nagios/plugins/check_swap -w 25% -c 10%

command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_total_loads]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_drive]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/sda
command[check_zombie_procs]=/usr/lib64/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w 250 -c 350 
command[check_pings]=/usr/lib64/nagios/plugins/check_icmp -H localhost
command[check_swap_space]=/usr/lib64/nagios/plugins/check_swap -w 25% -c 10%
command[check_memory]=/usr/lib64/nagios/plugins/check_mem -f -w 50 -c 20

# The following examples allow user-supplied arguments and can
# only be used if the NRPE daemon was compiled with support for 
# command arguments *AND* the dont_blame_nrpe directive in this
# config file is set to '1'.  This poses a potential security risk, so
# make sure you read the SECURITY file before doing this.

#command[check_users]=/usr/lib64/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
#command[check_load]=/usr/lib64/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
#command[check_disk]=/usr/lib64/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
#command[check_procs]=/usr/lib64/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$
#command[check_memory]=/usr/lib64/nagios/plugins/check_mem -f -w $ARG1$ -c $ARG2$ 

[abrist]

Restart the nrpe daemon on the remote host:

Code: Select all

service nrpe restart

Or:

Code: Select all

service xinetd restart

Then try to just connect to nrpe:

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H <remote host ip>
/usr/lib64/nagios/plugins/check_nrpe -H <remote host ip> -c check_memory
echo $?

Also, where did you acquire the check_mem plugin?
Can you attach it to your next post?

[calcazar]

nrpe works just fine, when I try to connect. I went through dozens of forums, sites and downloaded every script I could find until one worked, except for the one server.

Output from server to client:

Code: Select all

[root@management ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19
NRPE v2.15
[root@management ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19 -c check_drive
DISK OK - free space: / 11465 MB (86% inode=94%);| /=1714MB;11113;12502;0;13892
[root@management ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19 -c check_users
USERS OK - 1 users currently logged in |users=1;5;10;0
[root@management ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.0.19 -c check_memory
NRPE: Unable to read output
[root@management ~]# echo $?
130

Script I'm using"

Code: Select all

#!/usr/bin/perl -w

# Heavily based on the script from:
# check_mem.pl Copyright (C) 2000 Dan Larsson <dl@tyfon.net>
# heavily modified by
# Justin Ellison <justin@techadvise.com>
#
# The MIT License (MIT)
# Copyright (c) 2011 justin@techadvise.com

# Permission is hereby granted, free of charge, to any person obtaining a copy of this
# software and associated documentation files (the "Software"), to deal in the Software
# without restriction, including without limitation the rights to use, copy, modify,
# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to the following conditions:

# The above copyright notice and this permission notice shall be included in all copies
# or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
# FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
# OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.

# Tell Perl what we need to use
use strict;
use Getopt::Std;

#TODO - Convert to Nagios::Plugin
#TODO - Use an alarm

# Predefined exit codes for Nagios
use vars qw($opt_c $opt_f $opt_u $opt_w $opt_C $opt_v %exit_codes);
%exit_codes   = ('UNKNOWN' , 3,
        	 'OK'      , 0,
                 'WARNING' , 1,
                 'CRITICAL', 2,
                 );

# Get our variables, do our checking:
init();

# Get the numbers:
my ($free_memory_kb,$used_memory_kb,$caches_kb) = get_memory_info();
print "$free_memory_kb Free\n$used_memory_kb Used\n$caches_kb Cache\n" if ($opt_v);

if ($opt_C) { #Do we count caches as free?
    $used_memory_kb -= $caches_kb;
    $free_memory_kb += $caches_kb;
}

# Round to the nearest KB
$free_memory_kb = sprintf('%d',$free_memory_kb);
$used_memory_kb = sprintf('%d',$used_memory_kb);
$caches_kb = sprintf('%d',$caches_kb);

# Tell Nagios what we came up with
tell_nagios($used_memory_kb,$free_memory_kb,$caches_kb);


sub tell_nagios {
    my ($used,$free,$caches) = @_;
    
    # Calculate Total Memory
    my $total = $free + $used;
    print "$total Total\n" if ($opt_v);

    my $perf_warn;
    my $perf_crit;
    if ( $opt_u ) {
      $perf_warn = int(${total} * $opt_w / 100);
      $perf_crit = int(${total} * $opt_c / 100);
    } else {
      $perf_warn = int(${total} * ( 100 - $opt_w ) / 100);
      $perf_crit = int(${total} * ( 100 - $opt_c ) / 100);
    }
    
    my $perfdata = "|TOTAL=${total}KB;;;; USED=${used}KB;${perf_warn};${perf_crit};; FREE=${free}KB;;;; CACHES=${caches}KB;;;;";

    if ($opt_f) {
      my $percent    = sprintf "%.1f", ($free / $total * 100);
      if ($percent <= $opt_c) {
          finish("CRITICAL - $percent% ($free kB) free!$perfdata",$exit_codes{'CRITICAL'});
      }
      elsif ($percent <= $opt_w) {
          finish("WARNING - $percent% ($free kB) free!$perfdata",$exit_codes{'WARNING'});
      }
      else {
          finish("OK - $percent% ($free kB) free.$perfdata",$exit_codes{'OK'});
      }
    }
    elsif ($opt_u) {
      my $percent    = sprintf "%.1f", ($used / $total * 100);
      if ($percent >= $opt_c) {
          finish("CRITICAL - $percent% ($used kB) used!$perfdata",$exit_codes{'CRITICAL'});
      }
      elsif ($percent >= $opt_w) {
          finish("WARNING - $percent% ($used kB) used!$perfdata",$exit_codes{'WARNING'});
      }
      else {
          finish("OK - $percent% ($used kB) used.$perfdata",$exit_codes{'OK'});
      }
    }
}

# Show usage
sub usage() {
  print "\ncheck_mem.pl v1.0 - Nagios Plugin\n\n";
  print "usage:\n";
  print " check_mem.pl -<f|u> -w <warnlevel> -c <critlevel>\n\n";
  print "options:\n";
  print " -f           Check FREE memory\n";
  print " -u           Check USED memory\n";
  print " -C           Count OS caches as FREE memory\n";
  print " -w PERCENT   Percent free/used when to warn\n";
  print " -c PERCENT   Percent free/used when critical\n";
  print "\nCopyright (C) 2000 Dan Larsson <dl\@tyfon.net>\n";
  print "check_mem.pl comes with absolutely NO WARRANTY either implied or explicit\n";
  print "This program is licensed under the terms of the\n";
  print "MIT License (check source code for details)\n";
  exit $exit_codes{'UNKNOWN'}; 
}

sub get_memory_info {
    my $used_memory_kb  = 0;
    my $free_memory_kb  = 0;
    my $total_memory_kb = 0;
    my $caches_kb       = 0;

    my $uname;
    if ( -e '/usr/bin/uname') {
        $uname = `/usr/bin/uname -a`;
    }
    elsif ( -e '/bin/uname') {
        $uname = `/bin/uname -a`;
    }
    else {
        die "Unable to find uname in /usr/bin or /bin!\n";
    }
    print "uname returns $uname" if ($opt_v);
    if ( $uname =~ /Linux/ ) {
        my @meminfo = `/bin/cat /proc/meminfo`;
        foreach (@meminfo) {
            chomp;
            if (/^Mem(Total|Free):\s+(\d+) kB/) {
                my $counter_name = $1;
                if ($counter_name eq 'Free') {
                    $free_memory_kb = $2;
                }
                elsif ($counter_name eq 'Total') {
                    $total_memory_kb = $2;
                }
            }
            elsif (/^(Buffers|Cached|SReclaimable):\s+(\d+) kB/) {
                $caches_kb += $2;
            }
        }
        $used_memory_kb = $total_memory_kb - $free_memory_kb;
    }
    elsif ( $uname =~ /HP-UX/ ) {
      # HP-UX, thanks to Christoph Fürstaller
      my @meminfo = `/usr/bin/sudo /usr/local/bin/kmeminfo`;
      foreach (@meminfo) {
        chomp;
      	if (/^Physical memory\s\s+=\s+(\d+)\s+(\d+.\d)g/) {
      	  $total_memory_kb = ($2 * 1024 * 1024);
      	}
      	elsif (/^Free memory\s\s+=\s+(\d+)\s+(\d+.\d)g/) {
      	  $free_memory_kb = ($2 * 1024 * 1024);
      	}
      }
     $used_memory_kb = $total_memory_kb - $free_memory_kb;
    }
    elsif ( $uname =~ /FreeBSD/ ) {
      # The FreeBSD case. 2013-03-19 www.claudiokuenzler.com
      # free mem = Inactive*Page Size + Cache*Page Size + Free*Page Size
      my $pagesize = `sysctl vm.stats.vm.v_page_size`;
      $pagesize =~ s/[^0-9]//g;
      my $mem_inactive = 0;
      my $mem_cache = 0;
      my $mem_free = 0;
      my $mem_total = 0;
      my $free_memory = 0;
        my @meminfo = `/sbin/sysctl vm.stats.vm`;
        foreach (@meminfo) {
            chomp;
            if (/^vm.stats.vm.v_inactive_count:\s+(\d+)/) {
            $mem_inactive = ($1 * $pagesize);
            }
            elsif (/^vm.stats.vm.v_cache_count:\s+(\d+)/) {
            $mem_cache = ($1 * $pagesize);
            }
            elsif (/^vm.stats.vm.v_free_count:\s+(\d+)/) {
            $mem_free = ($1 * $pagesize);
            }
            elsif (/^vm.stats.vm.v_page_count:\s+(\d+)/) {
            $mem_total = ($1 * $pagesize);
            }
        }
        $free_memory = $mem_inactive + $mem_cache + $mem_free;
        $free_memory_kb = ( $free_memory / 1024);
        $total_memory_kb = ( $mem_total / 1024);
        $used_memory_kb = $total_memory_kb - $free_memory_kb;
        $caches_kb = ($mem_cache / 1024);
    }
    elsif ( $uname =~ /joyent/ ) {
      # The SmartOS case. 2014-01-10 www.claudiokuenzler.com
      # free mem = pagesfree * pagesize
      my $pagesize = `pagesize`;
      my $phys_pages = `kstat -p unix:0:system_pages:pagestotal | awk '{print \$NF}'`;
      my $free_pages = `kstat -p unix:0:system_pages:pagesfree | awk '{print \$NF}'`;
      my $arc_size = `kstat -p zfs:0:arcstats:size | awk '{print \$NF}'`;
      my $arc_size_kb = $arc_size / 1024;

      print "Pagesize is $pagesize" if ($opt_v);
      print "Total pages is $phys_pages" if ($opt_v);
      print "Free pages is $free_pages" if ($opt_v);
      print "Arc size is $arc_size" if ($opt_v);

      $caches_kb += $arc_size_kb;

      $total_memory_kb = $phys_pages * $pagesize / 1024;
      $free_memory_kb = $free_pages * $pagesize / 1024;
      $used_memory_kb = $total_memory_kb - $free_memory_kb;
    }
    elsif ( $uname =~ /SunOS/ ) {
        eval "use Sun::Solaris::Kstat";
        if ($@) { #Kstat not available
            if ($opt_C) {
                print "You can't report on Solaris caches without Sun::Solaris::Kstat available!\n";
                exit $exit_codes{UNKNOWN};
            }
            my @vmstat = `/usr/bin/vmstat 1 2`;
            my $line;
            foreach (@vmstat) {
              chomp;
              $line = $_;
            }
            $free_memory_kb = (split(/ /,$line))[5] / 1024;
            my @prtconf = `/usr/sbin/prtconf`;
            foreach (@prtconf) {
                if (/^Memory size: (\d+) Megabytes/) {
                    $total_memory_kb = $1 * 1024;
                }
            }
            $used_memory_kb = $total_memory_kb - $free_memory_kb;
            
        }
        else { # We have kstat
            my $kstat = Sun::Solaris::Kstat->new();
            my $phys_pages = ${kstat}->{unix}->{0}->{system_pages}->{physmem};
            my $free_pages = ${kstat}->{unix}->{0}->{system_pages}->{freemem};
            # We probably should account for UFS caching here, but it's unclear
            # to me how to determine UFS's cache size.  There's inode_cache,
            # and maybe the physmem variable in the system_pages module??
            # In the real world, it looks to be so small as not to really matter,
            # so we don't grab it.  If someone can give me code that does this, 
            # I'd be glad to put it in.
            my $arc_size = (exists ${kstat}->{zfs} && ${kstat}->{zfs}->{0}->{arcstats}->{size}) ?
                 ${kstat}->{zfs}->{0}->{arcstats}->{size} / 1024 
                 : 0;
            $caches_kb += $arc_size;
            my $pagesize = `pagesize`;
    
            $total_memory_kb = $phys_pages * $pagesize / 1024;
            $free_memory_kb = $free_pages * $pagesize / 1024;
            $used_memory_kb = $total_memory_kb - $free_memory_kb;
        }
    }
    elsif ( $uname =~ /AIX/ ) {
        my @meminfo = `/usr/bin/vmstat -vh`;
        foreach (@meminfo) {
            chomp;
            if (/^\s*([0-9.]+)\s+(.*)/) {
                my $counter_name = $2;
                if ($counter_name eq 'memory pages') {
                    $total_memory_kb = $1*4;
                }
                if ($counter_name eq 'free pages') {
                    $free_memory_kb = $1*4;
                }
                if ($counter_name eq 'file pages') {
                    $caches_kb = $1*4;
                }
                if ($counter_name eq 'Number of 4k page frames loaned') {
                    $free_memory_kb += $1*4;
                }
            }
        }
        $used_memory_kb = $total_memory_kb - $free_memory_kb;
    }
    else {
        if ($opt_C) {
            print "You can't report on $uname caches!\n";
            exit $exit_codes{UNKNOWN};
        }
    	my $command_line = `vmstat | tail -1 | awk '{print \$4,\$5}'`;
    	chomp $command_line;
        my @memlist      = split(/ /, $command_line);
    
        # Define the calculating scalars
        $used_memory_kb  = $memlist[0]/1024;
        $free_memory_kb = $memlist[1]/1024;
        $total_memory_kb = $used_memory_kb + $free_memory_kb;
    }
    return ($free_memory_kb,$used_memory_kb,$caches_kb);
}

sub init {
    # Get the options
    if ($#ARGV le 0) {
      &usage;
    }
    else {
      getopts('c:fuCvw:');
    }
    
    # Shortcircuit the switches
    if (!$opt_w or $opt_w == 0 or !$opt_c or $opt_c == 0) {
      print "*** You must define WARN and CRITICAL levels!\n";
      &usage;
    }
    elsif (!$opt_f and !$opt_u) {
      print "*** You must select to monitor either USED or FREE memory!\n";
      &usage;
    }
    
    # Check if levels are sane
    if ($opt_w <= $opt_c and $opt_f) {
      print "*** WARN level must not be less than CRITICAL when checking FREE memory!\n";
      &usage;
    }
    elsif ($opt_w >= $opt_c and $opt_u) {
      print "*** WARN level must not be greater than CRITICAL when checking USED memory!\n";
      &usage;
    }
}

sub finish {
    my ($msg,$state) = @_;
    print "$msg\n";
    exit $state;
}

[abrist]

The "Unable to read output" error is only caused by a few things:
1. NRPE not running.
2. Incorrect plugin path in nrpe.cfg
3. Permission issues with the plugin.

Lets take a look at these:

1. NRPE not running.
This is obviously not the case as other nrpe plugins are check-able through nrpe.

2. Incorrect plugin path in nrpe.cfg
This is probably not the case as your plugin is located at (and you have successfully run it):

Code: Select all

/usr/lib64/nagios/plugins/check_mem

And the nrpe.cfg command states:

Code: Select all

command[check_memory]=/usr/lib64/nagios/plugins/check_mem -f -w 50 -c 20

So those both look ok.

3. Permission issues with the plugin.
Lets check the permissions:

Code: Select all

ls -la /usr/lib64/nagios/plugins/check_mem

And try running it as nagios:

Code: Select all

su nagios
/usr/lib64/nagios/plugins/check_mem -f -w 50 -c 20
echo $?

[calcazar]

I thought it might have been a permission issue, but I've also tried multiple combinations of permissions. 777, root:root, nagios:nagios,nrpe:nagios and possibly a couple others..... "I've also disabled selinux"

Code: Select all

[root@api ~]# ls -la /usr/lib64/nagios/plugins/check_mem
-rwxr-xr-x. 1 root root 12330 Jul  8 13:02 /usr/lib64/nagios/plugins/check_mem

Code: Select all

[root@api ~]# su nagios
bash-4.1$ /usr/lib64/nagios/plugins/check_mem -f -w 50 -c 20
OK - 84.1% (6781292 kB) free.|TOTAL=8059740KB;;;; USED=1278448KB;4029870;6447792;; FREE=6781292KB;;;; CACHES=1026804KB;;;;
bash-4.1$ echo $?
0

[abrist]

There must be a control character issue. Are you editing the nrpe.cfg file with windows through filezilla/winscp/etc?
Can you post the nrpe.cfg as an attached file (instead of in code wraps)?

[calcazar]

I edit the file on the server using vim... I've attached the nrpe.cfg..

[abrist]

File looks clean. dang.
Can you post a long tail of /var/log/messages?

Code: Select all

tail -50 /var/log/messages