Page 1 of 2

NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 11:56 am
by lee.krause
I have written a script to check for the size of /oracle/SID/oraarch on our Solaris servers.
When I run the command as the nagios user it works on the client. When I run the command from the nagios server it fails with NRPE: Unable to read output.

Below is the script.

Code: Select all

#!/bin/bash

while test -n "$1"; do
    case "$1" in
        -w)
            WarnInodes=$2
            shift
            ;;
        -c)
            CritInodes=$2
            shift
            ;;
        -p)
            Partition=$2
            shift
            ;;
    esac
    shift
done

sid=`sudo df -h |awk '{print $NF}' | grep oraarch`
if [ $sid -z ];
        then
        echo "No oraarch."
        exit 0
else

inode=`sudo df -h $sid | awk '{print $5}' | grep -vi Cap | sed 's/%//g'`
#inode=81
#echo $inode $WarnInodes $CritInodes
if [ "$inode" -lt "$WarnInodes" ];
        then
        echo "$sid disk space $inode% used"
        exit 0
fi
if [ "$inode" -gt "$WarnInodes" -a "$inode" -lt "$CritInodes" ];
        then
        echo "$sid disk space $inode% used"
        exit 1
fi
if [ "$inode" -gt "$CritInodes" ];
        then
        echo "$sid disk space $inode% used"
        exit 2
fi
fi
Here is the nrpe.cfg:
command[check_orarch_space]=/etc/nagios/nrpe/check_orarch_space $ARG1$

Command on client:
root@client nagios # sudo su - nagios
-bash-3.2$ /etc/nagios/nrpe/check_orarch_space -w 10 -c 80
No oraarch.
-bash-3.2$ echo $?
0

Command on nagios server:
[root@nagios /]# /usr/local/nagios/libexec/check_nrpe -H 136.205.105.86 -t 30 -c check_orarch_space -a '-w 89 -c 90'
NRPE: Unable to read output


Permissions are all atleast 700 nagios on all the scripts.

The NRPE agent is talking to the nagios server and getting some results:

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 1:12 pm
by lmiltchev
This error is usually due to path issues, or sudo issues. Your path looks fine - lets check the sudoer tty settings:

Code: Select all

cat /etc/sudoers | grep tty
Do you see any errors in the system logs on the client? Does it help if you add the script to sudoers?

Code: Select all

nagios ALL=(ALL) NOPASSWD:/etc/nagios/nrpe/check_orarch_space *

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 1:19 pm
by gormank
Is the command defined correctly in nrpe.cfg or commands.cfg?

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 1:47 pm
by bwallace
Just checking to see if you've had a chance to try the suggestions from 'lmiltchev' and 'gormank' - let us know of the outcome and if you have any questions. We'll check in again on Monday unless we hear from you sooner, thanks.

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:06 pm
by lee.krause
root@client nrpe # cat /etc/sudoers | grep tty
root@client nrpe #

root@client nrpe # cat /etc/sudoers | grep check_
nagios ALL=(ALL) NOPASSWD:/etc/nagios/nrpe/check_orarch_space *
root@client nrpe #

Here is the nrpe.cfg:
command[check_orarch_space]=/etc/nagios/nrpe/check_orarch_space $ARG1$

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:18 pm
by tgriep
Can you run the following as root on the remote server and post the output?

Code: Select all

ls -lR /etc/nagios/
Can you post your full nrpe.cfg file so we can review it?

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:22 pm
by lee.krause
root@client nrpe # ls -lR /etc/nagios/
/etc/nagios/:
total 34
drwxr-xr-x 2 nagios nagios 512 Feb 11 20:45 nrpe
-rwx------ 1 nagios nagios 7604 Feb 12 16:21 nrpe.cfg
-rw------- 1 nagios nagios 7934 Jan 18 2013 nrpe.cfg.save

/etc/nagios/nrpe:
total 8
-rwx------ 1 nagios nagios 918 Feb 12 17:15 check_disk_inodes
-rwx------ 1 nagios nagios 888 Feb 12 17:15 check_orarch_space
-rw-r--r-- 1 nagios nagios 1040 Feb 3 16:23 common.cfg

Code: Select all

root@client nagios # cat nrpe.cfg
#############################################################################
# Sample NRPE Config File
# Written by: Ethan Galstad ([email protected])
#
# Last Modified: 11-23-2007
#
# NOTES:
# This is a sample configuration file for the NRPE daemon.  It needs to be
# located on the remote host that is running the NRPE daemon, not the host
# from which the check_nrpe client is being executed.
#############################################################################


# LOG FACILITY
# The syslog facility that should be used for logging purposes.

log_facility=daemon



# PID FILE
# The name of the file in which the NRPE daemon should write it's process ID
# number.  The file is only written if the NRPE daemon is started by the root
# user and is running in standalone mode.

pid_file=/var/run/nrpe.pid



# PORT NUMBER
# Port number we should wait for connections on.
# NOTE: This must be a non-priviledged port (i.e. > 1024).
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

server_port=5666



# SERVER ADDRESS
# Address that nrpe should bind to in case there are more than one interface
# and you do not want nrpe to bind on all interfaces.
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

#server_address=127.0.0.1



# NRPE USER
# This determines the effective user that the NRPE daemon should run as.
# You can either supply a username or a UID.
#
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

nrpe_user=nagios



# NRPE GROUP
# This determines the effective group that the NRPE daemon should run as.
# You can either supply a group name or a GID.
#
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

nrpe_group=nagios



# ALLOWED HOST ADDRESSES
# This is an optional comma-delimited list of IP address or hostnames
# that are allowed to talk to the NRPE daemon. Network addresses with a bit mask
# (i.e. 192.168.1.0/24) are also supported. Hostname wildcards are not currently
# supported.
#
# Note: The daemon only does rudimentary checking of the client's IP
# address.  I would highly recommend adding entries in your /etc/hosts.allow
# file to allow only the specified host to connect to the port
# you are running this daemon on.
#
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

allowed_hosts=127.0.0.1,xxx.xxx.xxx.xxx



# COMMAND ARGUMENT PROCESSING
# This option determines whether or not the NRPE daemon will allow clients
# to specify arguments to commands that are executed.  This option only works
# if the daemon was configured with the --enable-command-args configure script
# option.
#
# *** ENABLING THIS OPTION IS A SECURITY RISK! ***
# Read the SECURITY file for information on some of the security implications
# of enabling this variable.
#
# Values: 0=do not allow arguments, 1=allow command arguments

dont_blame_nrpe=1



# COMMAND PREFIX
# This option allows you to prefix all commands with a user-defined string.
# A space is automatically added between the specified prefix string and the
# command line from the command definition.
#
# *** THIS EXAMPLE MAY POSE A POTENTIAL SECURITY RISK, SO USE WITH CAUTION! ***
# Usage scenario:
# Execute restricted commmands using sudo.  For this to work, you need to add
# the nagios user to your /etc/sudoers.  An example entry for alllowing
# execution of the plugins from might be:
#
# nagios          ALL=(ALL) NOPASSWD: /usr/lib/nagios/plugins/
#
# This lets the nagios user run all commands in that directory (and only them)
# without asking for a password.  If you do this, make sure you don't give
# random users write access to that directory or its contents!

# command_prefix=/usr/bin/sudo



# DEBUGGING OPTION
# This option determines whether or not debugging messages are logged to the
# syslog facility.
# Values: 0=debugging off, 1=debugging on

debug=0



# COMMAND TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# allow plugins to finish executing before killing them off.

command_timeout=60



# CONNECTION TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# wait for a connection to be established before exiting. This is sometimes
# seen where a network problem stops the SSL being established even though
# all network sessions are connected. This causes the nrpe daemons to
# accumulate, eating system resources. Do not set this too low.

connection_timeout=300



# WEEK RANDOM SEED OPTION
# This directive allows you to use SSL even if your system does not have
# a /dev/random or /dev/urandom (on purpose or because the necessary patches
# were not applied). The random number generator will be seeded from a file
# which is either a file pointed to by the environment valiable $RANDFILE
# or $HOME/.rnd. If neither exists, the pseudo random number generator will
# be initialized and a warning will be issued.
# Values: 0=only seed from /dev/[u]random, 1=also seed from weak randomness

#allow_weak_random_seed=1



# INCLUDE CONFIG FILE
# This directive allows you to include definitions from an external config file.

#include=<somefile.cfg>
include=/etc/nagios/nrpe/common.cfg



# INCLUDE CONFIG DIRECTORY
# This directive allows you to include definitions from config files (with a
# .cfg extension) in one or more directories (with recursion).

#include_dir=<somedirectory>
#include_dir=<someotherdirectory>



# COMMAND DEFINITIONS
# Command definitions that this daemon will run.  Definitions
# are in the following format:
#
# command[<command_name>]=<command_line>
#
# When the daemon receives a request to return the results of <command_name>
# it will execute the command specified by the <command_line> argument.
#
# Unlike Nagios, the command line cannot contain macros - it must be
# typed exactly as it should be executed.
#
# Note: Any plugins that are used in the command lines must reside
# on the machine that this daemon is running on!  The examples below
# assume that you have plugins installed in a /usr/local/nagios/libexec
# directory.  Also note that you will have to modify the definitions below
# to match the argument format the plugins expect.  Remember, these are
# examples only!


# The following examples use hardcoded command arguments...

#command[check_users]=/opt/nagios/libexec/check_users -w 5 -c 10
#command[check_load]=/opt/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
#command[check_hda1]=/opt/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
#command[check_zombie_procs]=/opt/nagios/libexec/check_procs -w 5 -c 10 -s Z
#command[check_total_procs]=/opt/nagios/libexec/check_procs -w 150 -c 200


# The following examples allow user-supplied arguments and can
# only be used if the NRPE daemon was compiled with support for
# command arguments *AND* the dont_blame_nrpe directive in this
# config file is set to '1'.  This poses a potential security risk, so
# make sure you read the SECURITY file before doing this.

#command[check_users]=/opt/nagios/libexec/check_users -w $ARG1$ -c $ARG2$
#command[check_load]=/opt/nagios/libexec/check_load -w $ARG1$ -c $ARG2$
#command[check_disk]=/opt/nagios/libexec/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
#command[check_procs]=/opt/nagios/libexec/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$
command[nag_passwdexpchk.sh]=/etc/nagios/nrpe/nag_passwdexpchk.sh
command[df.nag.sh]=/etc/nagios/nrpe/df.nag.sh
command[check_disk_inodes]=/etc/nagios/nrpe/check_disk_inodes $ARG1$
command[check_orarch_space]=/etc/nagios/nrpe/check_orarch_space $ARG1$

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:24 pm
by jolson
What happens if you log into your remote box and issue the following:

Code: Select all

su - nagios
/etc/nagios/nrpe/check_orarch_space *
echo $?

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:25 pm
by lee.krause
I also turned on debugging earlier and there's the log:
Feb 12 16:10:44 client nrpe[25182]: [ID 881351 daemon.debug] Host is asking for command 'check_disk_inodes' to be run...
Feb 12 16:10:44 client nrpe[25182]: [ID 462736 daemon.debug] Running command: /etc/nagios/nrpe/check_disk_inodes -w 89 -c 90
Feb 12 16:10:45 client nrpe[25182]: [ID 757686 daemon.debug] Command completed with return code 1 and output:
Feb 12 16:10:45 client nrpe[25182]: [ID 869297 daemon.debug] Return Code: 1, Output: NRPE: Unable to read output
Feb 12 16:10:45 client nrpe[25182]: [ID 903583 daemon.debug] Connection from xxx.xxx.xxx.xxx closed.

Re: NRPE: Unable to read output only fails on custom script

Posted: Fri Feb 12, 2016 3:27 pm
by lee.krause
root@client nagios # su - nagios
-bash-3.2$ /etc/nagios/nrpe/check_orarch_space *
No oraarch
-bash-3.2$ echo $?
0
-bash-3.2$