NRPE: Unable to read output

[saracm]

I have this problem with NRPE. I'm trying to run check_oracle_health (https://labs.consol.de/nagios/check_oracle_health/) plugin through NRPE.
When i exicute the script it works:
/usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus
OK - 1 connected users | connected_users=1;30;50
but when I'm trying to run it through NRPE i got this error:
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c test
NRPE: Unable to read output

nrpe.cfg

log_facility=daemon
debug=0
pid_file=/usr/local/nagios/var/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
dont_blame_nrpe=1
allow_bash_command_substitution=0
command_timeout=60
connection_timeout=300
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 3.5 -c 4
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
command[check_data]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /mnt/data
command[check_root]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 500 -c 550
command[check_swap]=/usr/local/nagios/libexec/check_swap -w 20% -c 10%
command[test]=sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus

i tried with /bin/sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus
and it still doesn't work.

Without sudo permission, it still doesn't work through NRPE

/etc/sudoers
nagios ALL=(ALL) NOPASSWD: ALL

getsebool -a | grep nagios
logging_syslogd_run_nagios_plugins --> on
nagios_run_pnp4nagios --> off
nagios_run_sudo --> on
nagios_use_nfs --> on

I edited check_oracle_health plugin adding these lines in the begin to always use specific environment:
$ENV{'ORACLE_HOME'}="/usr/lib/oracle/11.2/client64";
$ENV{'LD_LIBRARY_PATH'}="/usr/lib/oracle/11.2/client64/lib:/lib:/usr/lib";
$ENV{'TNS_ADMIN'}="/usr/lib/oracle/11.2/client64/network/admin";

Installed packages:
oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm
oracle-instantclient11.2-devel-11.2.0.4.0-1.x86_64.rpm
oracle-instantclient11.2-sqlplus-11.2.0.4.0-1.x86_64.rpm

ps -ef | grep nrpe
nagios 27100 1 0 09:23 ? 00:00:00 /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -f

/usr/local/nagios/libexec/check_nrpe --version
NRPE Plugin for Nagios
Version: 3.2.1

It works with other checks:
/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c check_root
DISK OK - free space: / 15 GB (39.57% inode=96%);| /=23GB;30;34;0;38

With debug option set to 1 i got this log in /var/log/messages
Apr 24 10:16:35 app_server nrpe[28584]: CONN_CHECK_PEER: checking if host is allowed: 127.0.0.1 port 36042
Apr 24 10:16:35 app_server nrpe[28585]: WARNING: my_system() seteuid(0): Operation not permitted
Apr 24 10:16:35 app_server systemd: Created slice User Slice of root.
Apr 24 10:16:35 app_server systemd: Started Session c27 of user root.
Apr 24 10:16:36 app_server systemd: Removed slice User Slice of root.

Nrpe is installed from the Source. I used --enable-command-args with the installation

Thank you in advance.
Milos

[gormank]

What's the test command?

/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c test
NRPE: Unable to read output

I think that unable to read output means there was no output.
A command is one defined in nrpe.cfg or the nrpe include dir if defined.

[benjaminsmith]

Thanks @gormank helping out on the Community Forum.

When you're running the plugin locally and it works. Are you logged in as root or the nagios user? Pretty sure this is a permissions or sudoers issue.

We have a helpful troubleshooting guide for this error message on our KB. Please follow the steps and let me know if are able to get it going.

NRPE - NRPE: Unable To Read Output

[saracm]

Hi guys,

I will try to describe better:

Nagios
nrpe
|
|
V
app_server -------------------------> db_server
check_oracle_health

I am trying to test the oracle connection between app_server and db_server via the check_oracle_health plugin written in Perl form Nagios via NRPE.
I installed everything that is needed on app_server to use the check_oracle_health plugin and to test db_server health.
From app_server I CAN get the output. And everything is working correctly.
The command is defined in nagios.cfg on app_server:
command[test]=sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus
I tried as root and as nagios user. I get the same output.
I also added the ENV variables directly in the script:
$ENV{'ORACLE_HOME'}="/usr/lib/oracle/11.2/client64";
$ENV{'LD_LIBRARY_PATH'}="/usr/lib/oracle/11.2/client64/lib:/lib:/usr/lib";
$ENV{'TNS_ADMIN'}="/usr/lib/oracle/11.2/client64//network/admin";

@benjaminsmith I tried everything from that article but I had no luck.

Thanks!

[raosri1992]

Hi,

Which version of NRPE agent is installed on the remote server.?

I'm having same issue with NRPE v4.0.3. I have submitted similar issue today. I'll be testing with NRPE 3.2.1 now.

Thanks,
Srikanth

[raosri1992]

HI Benjimin Smith,

I having same problem. I'm testing the scripts from this article https://github.com/colebrooke/kubernetes-nagios

I tried two versions of NRPE agent 3.2.1 & 4.0.3 too. Same problem for both versions.

Any one can help us find the root cause ..

Thanks,
Srikanth

[ssax]

Do the base nrpe command work from the nagios server to the remote NRPE server? Please send the output of these commands:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H X.X.X.X
/usr/local/nagios/libexec/check_nrpe -H X.X.X.X -n
/usr/local/nagios/libexec/check_nrpe -H X.X.X.X -2

Are you seeing any errors in /var/log/messages or /var/log/secure on the remote NRPE system?

Try changing your command to this:

Code: Select all

command[test]=sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus 2>&1

Then restart NRPE, run this again, and send us the full output:

Code: Select all

/usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c test

[saracm]

@ssax this is the output:
[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server
NRPE v3.2.1

[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server -n
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).

[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server -2
NRPE v3.2.1

Output in /var/log/messages on the app_server:
Error: (!log_opts) Could not complete SSL handshake with public_ip: 1
May 14 12:54:11 app_server sshd[6430]: Connection closed by public_ip port 60270 [preauth]

I already changed command on my app_server to:
command[test]=sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus 2>&1
but this didnt help

From Nagios:
[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server -c test
NRPE: Unable to read output

From app_server:
$ /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c test
NRPE: Unable to read output

I'm using the same versions of NRPE on every server. (NRPE v3.2.1)

Thanks

[raosri1992]

I'm having the same problem, but my usecase is little different,. I have opened a support request here: https://support.nagios.com/forum/viewto ... =7&t=58573

Let me know if anyone needs more information.

Thanks,
Srikanth

@ssax this is the output:
[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server
NRPE v3.2.1

[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server -n
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).

[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H app_server -2
NRPE v3.2.1

Output in /var/log/messages on the app_server:
Error: (!log_opts) Could not complete SSL handshake with public_ip: 1
May 14 12:54:11 app_server sshd[6430]: Connection closed by public_ip port 60270 [preauth]

I already changed command on my app_server to:
command[test]=sudo /usr/local/nagios/libexec/check_oracle_health -t 240 --connect=DEV --user=nagios --password='xxxxxxxx' --mode=connected-users --warning 30 --critical 50 --method sqlplus 2>&1
but this didnt help

From Nagios:
[root@nagios ~]# /usr/local/nagios/libexec/check_nrpe -H uq-dev-app -c test
NRPE: Unable to read output

From app_server:
$ /usr/local/nagios/libexec/check_nrpe -H 127.0.0.1 -c test
NRPE: Unable to read output

I'm using the same versions of NRPE on every server. (NRPE v3.2.1)

Thanks