Problems using NAGIOS XI to monitor NAGIOS NA

[HIINNS]

NAGIOSNA_on_NAGIOSXI.docx

All,
I am having difficulty understanding why when I try to monitor our NAGIOS NA server using NAGIOS XI that I get a (No output on stdout) stderr: connect to address xxx.xxx.xxx.xxx (sorry my installation does not allow me to send IP addresses outside of our installation) port 5666: No route to host. The problem is the problem corrects itself on the plugin, and then some other plugin may report the same problem. See attached.

I read the following writeup-> https://support.nagios.com/forum/viewto ... 20&t=47406 but don't understand how port 5666 works for some plugins but not all, and how during the next collection cycle a different plugin(s) shows the same error message.

[benjaminsmith]

Hello,

The problem is the problem corrects itself on the plugin

Do you mean this issue is intermittent (comes and goes)? Also, please check the following items.

1. Is NRPE running on the remote host?

2. Run a nmap on the NNA server from XI and post the output.

Code: Select all

nmap <nna ip address>

3. Please upload nrpe.cfg from the remote server.

Code: Select all

/usr/local/nagios/etc/nrpe.cfg

[HIINNS]

NRPE is executed through the xinetd service on this server. NMAP is not on this server Red Hat 7.

[HIINNS]

Sorry, forgot to answer the 1st question. The problem is a little more than intermittent. Every 5 minutes different plugins get the message that the plugin can't communicate across port 5666. I will note that this server is virtual, and that it takes a while to complete a login. I have looked at the CPU, Memory, Disk, and Swap utilization on the server. All are not stressing the server.

[benjaminsmith]

Hello @HIINNS,

The fact that it cannot connect about every 5 minutes suggests a network issue. However, if it's timing out there would be a different error message. Are you using a proxy server?

Please run the check command directly from the shell and post the output. ( see How To Test Check Commands From The Command-li ).

Can you upload /etc/xinetd.d/nrpe from the remote server? Thanks.

[HIINNS]

No, I am not using a proxy server. I ran the CPU stats plugin multiple times. Normally, it returns like below:

nagios@nnagsr01 ~]$ /usr/local/nagios/libexec/check_nrpe -H SERVER1 -t 30 -c check_cpu_stats -a '-w 85 -c 95'
CPU STATISTICS OK: user=0.05% system=0.05% iowait=0.00% idle=99.90% cpu=.10%| user=0.0% system=0.0% iowait=0.0%;85;95 idle=99.90%

Occasionally, it will just return this:

[nagios@nnagsr01 ~]$ /usr/local/nagios/libexec/check_nrpe -H SERVER1 -t 30 -c check_cpu_stats -a '-w 85 -c 95'

[benjaminsmith]

Hello,

Thanks for posting the results of running the check command.

The port 5666: No route to host is returning back as there is no path to the host. If you are using a domain name in the check command, try using the IP address instead as it could be DNS issue.

You can also try increasing the timeout in the check command by adding a -t 30 ( see NRPE - CHECK_NRPE: Socket Timeout After n Seconds for instructions).

[HIINNS]

First, sorry for taking so long to get back to you. I changed from the DNS name to the IP address, without success. Additionally, I doubled the time for the command to execute without success. Any additional ideas. Could it be that because the server is virtual, and shares a NIC card, that the traffic through the NIC is what is holding the command up?

[mbellerue]

If there are enough virtual machines, with enough network traffic going over the same physical NIC, then it's possible that the NIC could be overloaded. I would think that if this was the case, then you might see times when more than 1 check fails at a time, or no checks fail. You would also see network related issues on other virtual machines on the host.

Going back to the error, No Route To Host, you mentioned that there is no proxy between the remote server and the Nagios XI instance, but is the remote server on a different subnet from the Nagios XI instance?

[HIINNS]

The problem is multiple plugins fail. I have seen up to 8 fail, then the next cycle a smaller number, then 8 again. The Nagios XI server is on a different subnet from the Nagios NA server.