check_nrpe : No route to host

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

check_nrpe : No route to host

Post by nos09 »

Hi, everybody.

I am trying to deploy a monitored server, before I do that on production machine again I was hoping to configure it on my virtual lab. I am following this guide http://www.tecmint.com/how-to-add-linux ... ed-server/

I have installed Nagios Core on Ubuntu Machine and the machine being monitored is a Centos Machine (dont ask about weird compination ... ;)

Apperently, I can't pass remote-host's nrpe check. Although both machines can talk to each other using ssh, when testing it says "no route to host on port 5666".

I performed nrpe-check from nagios-serv and got its request on port 5666 on Monitorig-node using tcpdump(which i can clearly see ... )

Nagios-Server is 192.168.0.154, monitored Node is 192.168.0.106.

From nagios-serv

Code: Select all

 
root@nagios-srv:/home/x# /usr/local/nagios/libexec/check_nrpe -H localhost 
connect to address ::1 port 5666: Connection refused
connect to address ::1 port 5666: Connection refused
NRPE v2.15
root@nagios-srv:/home/x# /usr/local/nagios/libexec/check_nrpe -H 192.168.0.106
connect to address 192.168.0.106 port 5666: No route to host
connect to host 192.168.0.106 port 5666: No route to host
root@nagios-srv:/home/x# ssh 192.168.0.106 /usr/local/nagios/libexec/check_procs
[email protected]'s password: 
PROCS OK: 93 processes | procs=93;;;0;
Tcpdump output on monitored node

Code: Select all

[root@localhost x]# tcpdump -i any tcp port 5666
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
18:29:47.231835 IP 192.168.0.154.34586 > localhost.nrpe: Flags [S], seq 2953610689, win 29200, options [mss 1460,sackOK,TS val 1946237 ecr 0,nop,wscale 6], length 0
18:37:09.533490 IP 192.168.0.154.34596 > localhost.nrpe: Flags [S], seq 2300650933, win 29200, options [mss 1460,sackOK,TS val 2056803 ecr 0,nop,wscale 6], length 0
18:59:05.558560 IP 192.168.0.154.34627 > localhost.nrpe: Flags [S], seq 294239989, win 29200, options [mss 1460,sackOK,TS val 2385780 ecr 0,nop,wscale 6], length 0
18:59:39.126467 IP6 localhost.38143 > localhost.nrpe: Flags [S], seq 3595959895, win 43690, options [mss 65476,sackOK,TS val 9583007 ecr 0,nop,wscale 6], length 0


I can also do tcp_check

From Monitored node to Nagios-sever

Code: Select all

[root@localhost x]# /usr/local/nagios/libexec/check_tcp -H localhost -p 5666
TCP OK - 0.009 second response time on localhost port 5666|time=0.008811s;;;0.000000;10.000000
[root@localhost x]# /usr/local/nagios/libexec/check_tcp -H 192.168.0.154 -p 5666
TCP OK - 0.001 second response time on 192.168.0.154 port 5666|time=0.000831s;;;0.000000;10.000000
[root@localhost x]#
From Nagios to monitored node

Code: Select all

root@nagios-serv:/home/x# /usr/local/nagios/libexec/check_tcp -H localhost -p 5666
TCP OK - 0.006 second response time on localhost port 5666|time=0.005666s;;;0.000000;10.000000
root@nagios-serv:/home/x# /usr/local/nagios/libexec/check_tcp -H 192.168.0.106 -p 5666
connect to address 192.168.0.106 and port 5666: No route to host
I have this feeling I am missing something obious but cant put my finger on it.. Any help would be much much appriciated. If any other thing is required to diagnose the problem feel free to ask. Once again thanks for your help in advance.
Top
User avatar
eloyd
Cool Title Here
Posts: 2190
Joined: Thu Sep 27, 2012 9:14 am
Location: Rochester, NY
Contact:
Contact eloyd

Re: check_nrpe : No route to host

Post by eloyd »

NRPE only needs to be running on the remote host, not the Nagios server, so that part is fine.

It looks like firewall issues on your remote host. Please do this on the remote CentOS machine:

Code: Select all

netstat -na | grep 5666
iptables -L -v -n
And paste the results into a

Code: Select all

 block.
Image
Eric Loyd • http://everwatch.global • 844.240.EVER • @EricLoyd
I'm a Nagios Fanatic! • Join our public Nagios Discord Server!
Top
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

Re: check_nrpe : No route to host

Post by nos09 »

Code: Select all

[root@localhost x]# netstat -na | grep 5666
tcp6       0      0 :::5666                 :::*                    LISTEN
IPTABLES :

Code: Select all

[root@localhost x]# iptables -L -v -n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 2039  170K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
  426 59736 INPUT_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  426 59736 INPUT_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  426 59736 INPUT_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    1    76 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
  418 59240 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
    0     0 FORWARD_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FORWARD_IN_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FORWARD_IN_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FORWARD_OUT_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FORWARD_OUT_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 1923 packets, 321K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 1923  321K OUTPUT_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD_IN_ZONES (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 FWDI_public  all  --  enp0s8 *       0.0.0.0/0            0.0.0.0/0           [goto] 
    0     0 FWDI_public  all  --  enp0s3 *       0.0.0.0/0            0.0.0.0/0           [goto] 
    0     0 FWDI_public  all  --  +      *       0.0.0.0/0            0.0.0.0/0           [goto] 

Chain FORWARD_IN_ZONES_SOURCE (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD_OUT_ZONES (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 FWDO_public  all  --  *      enp0s8  0.0.0.0/0            0.0.0.0/0           [goto] 
    0     0 FWDO_public  all  --  *      enp0s3  0.0.0.0/0            0.0.0.0/0           [goto] 
    0     0 FWDO_public  all  --  *      +       0.0.0.0/0            0.0.0.0/0           [goto] 

Chain FORWARD_OUT_ZONES_SOURCE (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDI_public (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 FWDI_public_log  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FWDI_public_deny  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FWDI_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FWDI_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDI_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDI_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDO_public (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 FWDO_public_log  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FWDO_public_deny  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 FWDO_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FWDO_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDO_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain FWDO_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT_ZONES (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  410 56791 IN_public  all  --  enp0s8 *       0.0.0.0/0            0.0.0.0/0           [goto] 
   16  2945 IN_public  all  --  enp0s3 *       0.0.0.0/0            0.0.0.0/0           [goto] 
    0     0 IN_public  all  --  +      *       0.0.0.0/0            0.0.0.0/0           [goto] 

Chain INPUT_ZONES_SOURCE (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain IN_public (3 references)
 pkts bytes target     prot opt in     out     source               destination         
  426 59736 IN_public_log  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  426 59736 IN_public_deny  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  426 59736 IN_public_allow  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain IN_public_allow (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    7   420 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 ctstate NEW

Chain IN_public_deny (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain IN_public_log (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination
Top
User avatar
eloyd
Cool Title Here
Posts: 2190
Joined: Thu Sep 27, 2012 9:14 am
Location: Rochester, NY
Contact:
Contact eloyd

Re: check_nrpe : No route to host

Post by eloyd »

Wow. You have a VERY complex iptables setup that seems to do very little actual work. But the bottom line is that I think you are rejecting your traffic to port 5666.

Try executing this command and then see if your check_nrpe is working:

Code: Select all

iptables -I INPUT -s 192.168.0.154 -p tcp -m tcp --dport 5666 -j ACCEPT
[code]
If it does,then you will want to add that to your /etc/sysconfig/iptables rules to allow traffic from your Nagios host.
Image
Eric Loyd • http://everwatch.global • 844.240.EVER • @EricLoyd
I'm a Nagios Fanatic! • Join our public Nagios Discord Server!
Top
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: check_nrpe : No route to host

Post by abrist »

eloyd wrote:Wow. You have a VERY complex iptables setup that seems to do very little actual work.
No joke. I would be curious of what the usage case is for this mess of rules.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Top
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

Re: check_nrpe : No route to host

Post by nos09 »

abrist wrote:
eloyd wrote:Wow. You have a VERY complex iptables setup that seems to do very little actual work.
No joke. I would be curious of what the usage case is for this mess of rules.
Ummm.. I know its scary but I should just say it anyway... Its the default one. I haven't touched a thing in this machine except the installation of Nrpe and plugins.
eloyd wrote:Wow. You have a VERY complex iptables setup that seems to do very little actual work. But the bottom line is that I think you are rejecting your traffic to port 5666.

Try executing this command and then see if your check_nrpe is working:

Code: Select all

iptables -I INPUT -s 192.168.0.154 -p tcp -m tcp --dport 5666 -j ACCEPT
[code]
If it does,then you will want to add that to your /etc/sysconfig/iptables rules to allow traffic from your Nagios host.[/quote]

Worked like a charm.

BTW, I am too wondering about that IPtable mess. I just installed Centos 7 Core on vm to cloned one to try nrpe. I have checked the 'untouched one' and it too contains that entire blob !
Top
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

Re: check_nrpe : No route to host

Post by nos09 »

Hi, Just found out. I dont have /etc/sysconfig/iptables file. instead there are two ip6tables-config iptables-config.
There is no system-config-firewall-tui either !
What is up with centos 7 ? have i been sleeping the whole year .. . . :( :( :(
Top
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

Re: check_nrpe : No route to host

Post by nos09 »

nos09 wrote:Hi, Just found out. I dont have /etc/sysconfig/iptables file. instead there are two ip6tables-config iptables-config.
There is no system-config-firewall-tui either !
What is up with centos 7 ? have i been sleeping the whole year .. . . :( :( :(
Centos 7 is handling the iptable rule via firewall-cmd now. reference = https://access.redhat.com/documentation ... walls.html

This should open up the port permanently :

Code: Select all

 firewall-cmd --zone=public --add-port=5666/tcp --permanent
but for lack of batter understanding, I am wondering that it would all everyone to access that port right ? Can anyone suggest modification in syntax to allow a specific host ?
Top
User avatar
eloyd
Cool Title Here
Posts: 2190
Joined: Thu Sep 27, 2012 9:14 am
Location: Rochester, NY
Contact:
Contact eloyd

Re: check_nrpe : No route to host

Post by eloyd »

I'll be honest - I'm afraid of CentOS 7. :-) As such, I don't use it. But here are two options:

Disable firewalld and start using iptables instead:

Code: Select all

systemctl disable firewalld
systemctl stop firewalld
yum install iptables-services
systemctl start iptables
systemctl enable iptables
systemctl start ip6tables (if needed)
systemctl enable ip6tables (if needed)
Then you can add the following line to /etc/sysconfig/iptables:

Code: Select all

iptables -I INPUT -s 192.168.0.154 -p tcp -m tcp --dport 5666 -j ACCEPT
Or you can use firewall-cmd itself and add the line to the appropriate zone (most likely, "work"):

Code: Select all

firewall-cmd --permanent --zone=work--add-rich-rule="rule family="ipv4" \
     source address="192.168.0.154/32" \
     port protocol="tcp" port="5666" accept"
Personally, I dislike firewalld and think manipulating iptables is easier, but that's probably because I've been doing it a long time. Your mileage may vary.
Image
Eric Loyd • http://everwatch.global • 844.240.EVER • @EricLoyd
I'm a Nagios Fanatic! • Join our public Nagios Discord Server!
Top
nos09
Posts: 9
Joined: Mon Jul 28, 2014 1:02 pm

Re: check_nrpe : No route to host

Post by nos09 »

eloyd wrote:I'll be honest - I'm afraid of CentOS 7. :-) As such, I don't use it. But here are two options:

Disable firewalld and start using iptables instead:

Code: Select all

systemctl disable firewalld
systemctl stop firewalld
yum install iptables-services
systemctl start iptables
systemctl enable iptables
systemctl start ip6tables (if needed)
systemctl enable ip6tables (if needed)
Then you can add the following line to /etc/sysconfig/iptables:

Code: Select all

iptables -I INPUT -s 192.168.0.154 -p tcp -m tcp --dport 5666 -j ACCEPT
Or you can use firewall-cmd itself and add the line to the appropriate zone (most likely, "work"):

Code: Select all

firewall-cmd --permanent --zone=work--add-rich-rule="rule family="ipv4" \
     source address="192.168.0.154/32" \
     port protocol="tcp" port="5666" accept"
Personally, I dislike firewalld and think manipulating iptables is easier, but that's probably because I've been doing it a long time. Your mileage may vary.
Just curious, why would you change the way iptables works? wasn't it good enough? :!: :!:
Top
Locked

Return to “Open Source Nagios Projects”