New Nagios Build - unable to get NRPE working

[thetechguy]

I have a new VM with Nagios 4.0.2 installed.. I am having trouble with getting NRPE to work correctly. Currently I am getting the error (Return code of 255 is out of bounds).

A bit of history..

I installed the master plugin package and then also nrpe..
I set up a server to check for load and drive space.
Received an error that check_nrpe was not found..
Checked the plugin library folder and it was indeed missing.

I ran a make install all on the nrpe 2.15 folder after follow some other forum advise entries.
The check_nrpe was now in the /usr/local/nagios/libexec director..

Then this new error of (Return code of 255 is out of bounds) started to show up..

I have tried this on two servers and they are both showing the same error.

Any help is appreciated.

Thank you

Jon

[lmiltchev]

Let's start with something simple:

1. See if you can run the check locally on the client (remote box):

Code: Select all

./check_load -w 15,10,5 -c 30,20,10

2. See if you have "check_load" defined in the nrpe.cfg file. You may have something like this:

Code: Select all

command[check_load]=/usr/local/nagios/libexec/check_load -w $ARG1$ -c $ARG2$

3. Test from the CLI on the nagios server:

Code: Select all

./check_nrpe -H <client ip>
./check_nrpe -H <client ip> -c check_load -a 15,10,5 30,20,10

[thetechguy]

Thank you for your quick reply..

I received the following after executing your suggestions

./check_load -w 15,10,5 -c 30,20,10 - locally on remote client.
OK - load average: 0.38, 0.29, 0.21|load1=0.380;15.000;30.000;0; load5=0.290;10.000;20.000;0; load15=0.210;5.000;10.000;0;

Test from the CLI on the Nagios server:
./check_nrpe -H <remote host ip>
CHECK_NRPE: Error - Could not complete SSL handshake. ---- (hmm that looks like a problem...)

# The following examples use hardcoded command arguments...

command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/lib64/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w 150 -c 200

(looks like the check_nrpe as well as other checks are in two folders is that a problem? /usr/lib64/nagios/plugins/ and /usr/local/nagios/libexec/)

Thank you

[abrist]

Try running the nrpe test on the remote host:

Code: Select all

./check_nrpe -H localhost

What method did you use to install nrpe? (source or package from repo)

[thetechguy]

abrist

ran that command on the remote host and got this..

[root@delta plugins]# ./check_nrpe -H localhost
./check_nrpe: error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory (that seems like a problem too)

I installed it using the following commands

wget dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum install -y nrpe nagios-plugins-all openssl

Thank you

Jon

[abrist]

yum install -y nrpe nagios-plugins-all openssl

You should have installed openssl first. additionally, your need to reinstall nrpe once you have installed openssl. If you are installing nrpe from source, you will also need the openssl-devel package.

[millisa]

Based on the notes above, it looks like you are using CentOS6, x86_64, with the EPEL repo.
I've setup a CentOS6.5 Minimal x86_64 install for a test 'remote' system and included all steps to get a remote check_nrpe to work so you can compare to the steps you've taken.

This shouldn't be an issue with openssl in this case; even the minimal CentOS install has openssl by default (it'll get updated though when you pull in the nagios rpm's from EPEL)

If I had to guess, I'm wagering either the allowed_hosts in the nrpe.cfg isn't set to allow the nagios server, or you don't have iptables open for port 5666.

Setup notes below, so you can compare against what you've setup:
#####
note: iptables & selinux are on by default, only tcp22 is open,
localhost is in hosts file by default:

Code: Select all

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

(I only imagine this might matter when you are testing with 'localhost')

Code: Select all

yum install wget nano
wget dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum install -y nrpe nagios-plugins-all openssl

(pulls in about 80 packages, updates openssl)
(openssl should be installed by default, even in minimal config on centos)


Edit /etc/nagios/nrpe.cfg, default port should already be set to 5666.
Update allowed_hosts= to have localhost and nagios server IP:

Code: Select all

allowed_hosts=127.0.0.1,192.168.1.123

(assuming 192.168.1.123 is your nagios server)


Start nrpe:

Code: Select all

/etc/init.d/nrpe start

(you'll want to set it to autostart)

Open iptables 5666 to nagios server (assuming 192.168.1.123 is your nagios server):

Code: Select all

/sbin/iptables -I INPUT 4 -m state --state NEW -m tcp -p tcp --dport 5666 -s 192.168.1.123 -j ACCEPT

On a default CentOS 6 install, this will make your iptables look similar to this (your nagios server IP would be different, mine was 192.168.176.41 in this sample):

Code: Select all

[root@nagiostestd ~]# /etc/init.d/iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  192.168.176.41       0.0.0.0/0           state NEW tcp dpt:5666 
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
6    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Save to retain on reboot:

Code: Select all

/etc/init.d/iptables save

On the remote's local console, this should now work

Code: Select all

[root@nagiostestd ~]# /usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,20,10
OK - load average: 0.00, 0.01, 0.00|load1=0.000;15.000;30.000;0; load5=0.010;10.000;20.000;0; load15=0.000;5.000;10.000;0; 

If you install the check_nrpe plugin from EPEL with:

Code: Select all

yum install nagios-plugins-nrpe

this will also work locally at this point (the epel nrpe.cfg should already have check_load defined)

Code: Select all

[root@nagiostestd ~]# /usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_load
OK - load average: 0.03, 0.02, 0.00|load1=0.030;15.000;30.000;0; load5=0.020;10.000;25.000;0; load15=0.000;5.000;20.000;0; 

On the nagios server, this should work (path assumes it is also a centos6 64bit system with nagios-plugins-nrpe from EPEL):

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H REMOTEIPHERE -c check_load

Code: Select all

[root@nagiostestA ~]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.176.44 -c check_load
OK - load average: 0.00, 0.00, 0.00|load1=0.000;15.000;30.000;0; load5=0.000;10.000;25.000;0; load15=0.000;5.000;20.000;0; 

If this doesn't work, make sure you can reach port 5666 on the remote system from the nagios server.
(nmap is the easiest way to verify your open/reachable ports, replace 192.168.176.44 with your remote server IP)

Code: Select all

yum install nmap
[root@nagiostestA ~]# nmap -p 5666 192.168.176.44

Starting Nmap 5.51 ( http://nmap.org ) at 2014-02-13 21:17 CST
Nmap scan report for 192.168.176.44
Host is up (0.00074s latency).
PORT     STATE SERVICE
5666/tcp open  nrpe

A closed (bad) setup would look like this:

Code: Select all

[root@nagiostestA ~]# nmap -p 5666 192.168.176.44

Starting Nmap 5.51 ( http://nmap.org ) at 2014-02-13 21:23 CST
Nmap scan report for 192.168.176.44
Host is up (0.00068s latency).
PORT     STATE    SERVICE
5666/tcp filtered nrpe

If 5666 doesn't show open, check your iptables rules, check your port in the nrpe.cfg, verify you don't have other firewalls between your nagios server and the remote host.

[slansing]

Holy smokes millisa,

That was a very detailed reply, thanks for helping out! I agree with it either being allowed_hosts, or iptables rules, those two usually trip some people up.

[thetechguy]

Since starting this post the server I was working with is now a boat anchor.. maybe related to my problems .. not sure. I appreciate all of the replies, I am moving onto trying it again on other server and will refer to this to assist with the installation and testing.. I will update the thread when I have success.

Thank you all for your help

Jon

[tmcdonald]

We'll keep this thread open until you return.