Nagios Support Forum

Hello,

I've tried everything I can think of and I can't figure out what's going on. The NRPE service checks for my Nagios XI test server are reporting "connection refused" on port 5666. I know without a doubt this isn't a network permit issue.

It started after I manually performed a restore of XI from the backup of my failover server. This involved restoring a profile from a CentOS 6 box to a CentOS 7 box. After the restore I ran the repair script and everything came up fine with the exception of the NRPE service check.

Here's the command and the resulting message:

/usr/local/nagios/libexec/check_tcp -H 10.131.86.60 -p 5666
connect to address 10.131.86.60 and port 5666: Connection refused

I think it has to do with a configuration somewhere on the box, but I can't find it.

rferebee wrote:This involved restoring a profile from a CentOS 6 box to a CentOS 7 box.

Just a quick verification, is this new CentOS 7 box's IP address is listed in the NRPE client's allowed hosts?

If so, then we'll want to verify that there's no firewall running on the host you're trying to monitor. Is this a Windows host? I know Microsoft likes to tinker with firewall settings during Windows updates.

Also verify that the NRPE client is actually running.

I know, these are all of the easy steps. But sometimes it helps to retrace them just in case one was missed.

The box has the same IP address as the box it replaced. We built a new box, installed Nagios XI on it, restored the profile from the old CentOS 6 box (repaired after restore), shutdown the CentOS 6 box and moved it's network config over to this CentOS 7 box.

The firewall is not running on the host. It's a Linux host.

I checked the status of the NRPE service and got the following:

root@nagiostest:/root>systemctl status nrpe
● nrpe.service - Nagios Remote Plugin Executor
Loaded: loaded (/usr/lib/systemd/system/nrpe.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Docs: http://www.nagios.org/documentation
root@nagiostest:/root>systemctl start nrpe
root@nagiostest:/root>systemctl status nrpe
● nrpe.service - Nagios Remote Plugin Executor
Loaded: loaded (/usr/lib/systemd/system/nrpe.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2019-08-27 13:46:58 PDT; 3s ago
Docs: http://www.nagios.org/documentation
Process: 10957 ExecStopPost=/bin/rm -f /usr/local/nagios/var/nrpe.pid (code=exited, status=0/SUCCESS)
Process: 10954 ExecStart=/usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -f (code=exited, status=2)
Main PID: 10954 (code=exited, status=2)

Aug 27 13:46:58 nagiostest.state.nv.us systemd[1]: Started Nagios Remote Plugin Executor.
Aug 27 13:46:58 nagiostest.state.nv.us nrpe[10954]: Starting up daemon
Aug 27 13:46:58 nagiostest.state.nv.us nrpe[10954]: Cannot write to pidfile '/var/run/nrpe.pid' - check your privileges.
Aug 27 13:46:58 nagiostest.state.nv.us systemd[1]: nrpe.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 27 13:46:58 nagiostest.state.nv.us systemd[1]: Unit nrpe.service entered failed state.
Aug 27 13:46:58 nagiostest.state.nv.us systemd[1]: nrpe.service failed.

Is that how I would start NRPE if I'm not using the binary package?

Aug 27 13:46:58 nagiostest.state.nv.us nrpe[10954]: Cannot write to pidfile '/var/run/nrpe.pid' - check your privileges.
Aug 27 13:46:58 nagiostest.state.nv.us systemd[1]: nrpe.service: main process exited, code=exited, status=2/INVALIDARGUMENT

Well that's something. It looks like you're starting the service as root, so there shouldn't be a problem with permissions. Maybe do a ps -aux | grep -i nrpe and see if anything comes back. The only thing that should come back is your grep command. If more than that comes back, I bet there's a ghost process out there. Or maybe if there's already a /var/run/nrpe.pid?

I ran that grep and got this:

root@nagiostest:/root>ps -aux | grep -i nrpe
nagios 69613 0.0 0.0 40528 2500 ? S 14:30 0:00 /usr/local/nagios/libexec/check_nrpe -2 -H 10.131.143.154 -t 30 -c check_memory -a type=physical warn=free<20% crit=free<10%
root 70073 0.0 0.0 112712 996 pts/0 S+ 14:30 0:00 grep --color=auto -i nrpe

Try running this command as toot to see if the NRPE agent is getting started by xinetd. If it shows that xinetd starts it, make sure the Nagios XI server's IP address is in the /etc/xinetd.d/nrpe folder.

Also, look at this KB article for some troubleshooting steps.
https://support.nagios.com/kb/article/n ... t-622.html

It doesn't look like my server is even listening on port 5666.

When I look at a netstat output on the other two servers that are working, I see it listening on port 5666 any/any basically. But, when I do netstat from my test server (the one that stopped working) I don't see the line that says it's listening on port 5666.

I have confirmed that xinetd is running, so I'm not sure why it isn't listening. We're not using the firewall on this box, so there's nothing to configure in that arena.

Can you run the following commands and show the output in code wraps? Also, post the nrpe.cfg file on the forum.

PM sent. Thank you.

I think my test server has the binary package installed where as the other servers do not.

The servers that are working don't even have /etc/nagios/nrpe.cfg