Page 1 of 2
NRPE Not Returning Value
Posted: Wed Jan 10, 2018 1:54 pm
by danielgaraventa_ats
Hello,
We have a custom check that we have written for our Oracle database servers to check the number of days left until passwords expire via an SQL command. We are currently testing on moving our databases to RHEL 7 from SUSE 11. Running on SUSE 11 the check works as expected when we run the script through NRPE. On RHEL 7 the output from the SQL command does not seem to be being passed to NRPE It does work fine however when I run the check locally on the RHEL 7 server.
Code: Select all
#!/bin/bash
# Nagios return codes
UNKNOWN=3
CRIT=2
WARN=1
OK=0
OUT=$(/opt/oracle/product/11.2.0.4/bin/sqlplus -S redacted/redacted@svr-rhel7/SID << EOF
set head off;
select nvl( min(trunc(expiry_date) - trunc(sysdate)),90) days_remaining from dba_users u where expiry_date is not null and account_status = 'OPEN';
exit
EOF
)
export OUT
if test "$OUT" -gt 30; then
echo "OK - Passwords are not set to expire for another "$OUT" days"
exit $OK
elif test "$OUT" -gt 14; then
echo "WARN - Passwords are scheduled to expire in "$OUT" days"
exit $WARN
else
echo "CRIT - Passwords are scheduled to expire in "$OUT" days"
exit $CRIT
fi
When I run the script locally on the RHEL 7 box I get the following output.
Code: Select all
[root@svr-rhel7 plugins]# runuser -u nrpe /usr/lib64/nagios/plugins/test2
OK - Passwords are not set to expire for another 90 days
When I run the check_nrpe on our Nagios XI server I see the following output.
Code: Select all
[root@monitor libexec]# ./check_nrpe -H svr-rhel7 -t 60 -p 5666 -4 -c test2
CRIT - Passwords are scheduled to expire in days
We are currently running Nagios XI 5.4.11 and I have also disabled SELinux on the RHEL 7 server.
Thank you for any advice,
Daniel
Re: NRPE Not Returning Value
Posted: Wed Jan 10, 2018 2:34 pm
by npolovenko
Hello,
@danielgaraventa_ats.
What're the permissions on this folder?
Code: Select all
/opt/oracle/product/11.2.0.4/bin/sqlplus
When you run it locally it runs as a sudo user but if you run it with Nagios it runs as a Nagios user.
I'd probably do
chmod +x on that folder to see if that resolves the problem.
Re: NRPE Not Returning Value
Posted: Wed Jan 10, 2018 2:58 pm
by danielgaraventa_ats
Hello npolovenko,
sqlplus is a file in /opt/oracle/product/11.2.0.4/bin which has the following permissions.
Code: Select all
[root@ats-db-04 ~]# ls -la /opt/oracle/product/11.2.0.4/bin/sqlplus
-rwxr-x--x. 1 oracle oinstall 13575 Jun 24 2014 /opt/oracle/product/11.2.0.4/bin/sqlplus
/opt/oracle/product/11.2.0.4/bin has the following permissions set.
Code: Select all
[root@ats-db-04 ~]# ls -lad /opt/oracle/product/11.2.0.4/bin/
drwxr-xr-x. 2 oracle oinstall 12288 Nov 30 08:56 /opt/oracle/product/11.2.0.4/bin/
I have added the nagios and the nrpe user to the oinstall group.
Code: Select all
[root@ats-db-04 ~]# groups nrpe
nrpe : nrpe oinstall nagios
Code: Select all
[root@ats-db-04 ~]# groups nagios
nagios : nagios oinstall
I still seem to get the same results sadly.
Thanks,
Daniel
Re: NRPE Not Returning Value
Posted: Wed Jan 10, 2018 4:07 pm
by npolovenko
@danielgaraventa_ats, On the remote server, can you change the username to Nagios:
And then attempt to run the script locally. Does it work?
Are you running nrpe as a standalone service or under xinetd daemon?
Re: NRPE Not Returning Value
Posted: Wed Jan 10, 2018 4:26 pm
by danielgaraventa_ats
If I su to the nagios user locally the script runs normally.
Code: Select all
[nagios@ats-db-04 plugins]$ ./test2
OK - Passwords are not set to expire for another 90 days
As for the nrpe version on the rhel box I am using the standalone version I believe. It is the one that you can install from EPEL.
Code: Select all
yum list installed nrpe
nrpe.x86_64 3.2.0-6.el7 @epel
Re: NRPE Not Returning Value
Posted: Wed Jan 10, 2018 5:14 pm
by npolovenko
@danielgaraventa_ats, Actually, I think your version of nrpe runs checks under nrpe user. You can make sure by opening /usr/local/nagios/nrpe.cfg file, and looking for the lines that say nrpe_user= and nrpe_group=
The first thing you could try is to create a script on the remote server that will call the test2 script and put its output to a local text file ./test2 >1.txt. Then you'd run the parent script with nrpe. That way you will get to know whether results get lost during NRPE transaction or maybe nrpe user still has permission problems. Or I guess you could just save $OUT to a local text file. But I hope you get my point?
Also, you may give this a shot:
Keep us updated.
Re: NRPE Not Returning Value
Posted: Thu Jan 11, 2018 8:37 am
by danielgaraventa_ats
I stripped down the script to be the following. Taking the SQL command out just for testing purposes.
Code: Select all
#!/bin/bash
# Nagios return codes
UNKNOWN=3
CRIT=2
WARN=1
OK=0
OUTTEST=`cat test.txt`
export OUTTEST
if test "$OUTTEST" -gt 30; then
echo "OK - Passwords are not set to expire for another "$OUTTEST" days"
exit $OK
elif test "$OUTTEST" -gt 14; then
echo "WARN - Passwords are scheduled to expire in "$OUTTEST" days"
exit $WARN
else
echo "CRIT - Passwords are scheduled to expire in "$OUTTEST" days"
exit $CRIT
fi
test.txt contains the number 90. Running the script locally I see this.
Code: Select all
[nrpe@svr-rhel7 plugins]$ ./test2
OK - Passwords are not set to expire for another 90 days
Running the test from our Nagios server via nrpe the 90 is not populated.
Code: Select all
[root@monitor libexec]# ./check_nrpe -H svr-rhel7 -t 60 -p 5666 -4 -c test2
CRIT - Passwords are scheduled to expire in days
Thanks,
Daniel
Re: NRPE Not Returning Value
Posted: Thu Jan 11, 2018 8:46 am
by danielgaraventa_ats
I forgot to add test.txt is owned by nrpe and I change the permissions to 777.
Thanks,
Daniel
Re: NRPE Not Returning Value
Posted: Thu Jan 11, 2018 11:35 am
by npolovenko
@danielgaraventa_ats, If you change this line in the script:
to
Code: Select all
OUTTEST=`cat /full/path/to/test.txt`
Then it's going to work. I think the main problem with your original script is that sqlplus runs as a sudo, or as a nagios user but it doesn't run as nrpe user. Either because the nrpe user doesn't have a shell, or because of the other permissions problem. Sometimes you also need to include environment variables in order for the script to run.
Re: NRPE Not Returning Value
Posted: Thu Jan 11, 2018 12:37 pm
by danielgaraventa_ats
I believe I found the issue. I added the following the the script and I am now getting results back.
Code: Select all
ORACLE_BASE=/opt/oracle
ORACLE_HOME=$ORACLE_BASE/product/11.2.0.4
export ORACLE_BASE ORACLE_HOME
Code: Select all
[root@monitor libexec]# ./check_nrpe -H svr-rhel7 -t 60 -p 5666 -4 -c test2
OK - Passwords are not set to expire for another 90 days
Thank you for all your help! I will mark this post as solved.
Daniel