Page 1 of 2
Not able to monitor Apache and SSH services
Posted: Sat Feb 03, 2018 2:28 pm
by shamrozkadiwal
I have setup "check_http" "check_ssh"to monitor apache and ssh service on the server, but I have been getting
CRITICAL - Socket timeout after 10/20/30 seconds error for both plugins. I have tried to increase the timer but still getting the same error. I also check the firewall and both services have been added.
Nagios Core is running on Centos 7
NRPE is running on Ubuntu
Code: Select all
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -S
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -S -t 20
CRITICAL - Socket timeout after 20 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -S -p 443
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -S -p 80
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -S -p 8080
CRITICAL - Socket tieout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -N -p 80
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -N -p 80 -t 1
CRITICAL - Socket timeout after 1 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -N -p 80 -t 10
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43 -N -t 30
CRITICAL - Socket timeout after 30 seconds
[root@dj-mon libexec]# /usr/local/nagios/libexec/check_ssh -H 10.167.176.43
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon libexec]# /usr/local/nagios/libexec/check_ssh -H 10.167.176.43 -t 30
CRITICAL - Socket timeout after 30 seconds
Re: Not able to monitor Apache and SSH services
Posted: Mon Feb 05, 2018 12:39 pm
by npolovenko
Hello,
@shamrozkadiwal. Are you able to nmap the server for open ports? Please show us the output of the following command:
Re: Not able to monitor Apache and SSH services
Posted: Thu Feb 08, 2018 3:16 pm
by shamrozkadiwal
Code: Select all
ubuntuadmin@iusa-lin-db01:~$
ubuntuadmin@iusa-lin-db01:~$
ubuntuadmin@iusa-lin-db01:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:94:8e:ef brd ff:ff:ff:ff:ff:ff
inet 10.167.176.43/24 brd 10.167.176.255 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe94:8eef/64 scope link
valid_lft forever preferred_lft forever
ubuntuadmin@iusa-lin-db01:~$
ubuntuadmin@iusa-lin-db01:~$
ubuntuadmin@iusa-lin-db01:~$ nmap 10.167.176.43
Starting Nmap 7.01 ( https://nmap.org ) at 2018-02-08 14:15 CST
Nmap scan report for iusa-lin-db01.lan.ismailiusa.org (10.167.176.43)
Host is up (0.00032s latency).
Not shown: 997 closed ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
3306/tcp open mysql
Nmap done: 1 IP address (1 host up) scanned in 0.14 seconds
ubuntuadmin@iusa-lin-db01:~$
Re: Not able to monitor Apache and SSH services
Posted: Thu Feb 08, 2018 4:33 pm
by dwhitfield
Please take a look at
https://support.nagios.com/kb/article/n ... s-617.html and let us know if that does not help resolve the issue.
Re: Not able to monitor Apache and SSH services
Posted: Thu Feb 08, 2018 4:36 pm
by kyang
Are both servers online when running this check?
What version of check_http and check_ssh do you have?
Code: Select all
/usr/local/nagios/libexec/check_http -V
/usr/local/nagios/libexec/check_ssh -V
Can you run this towards your own Nagios server and not 10.167.176.43.
Code: Select all
/usr/local/nagios/libexec/check_http -H nagiosServerIP
Post the output thank you!
Re: Not able to monitor Apache and SSH services
Posted: Fri Feb 09, 2018 12:59 pm
by shamrozkadiwal
@kyang, Yes both servers are up all the time when these check runs.
Nagios Server (45.55.251.117)
Code: Select all
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -V
check_http v2.1.1 (nagios-plugins 2.1.1)
[root@dj-mon ~]# /usr/local/nagios/libexec/check_ssh -V
check_ssh v2.1.1 (nagios-plugins 2.1.1)
[root@dj-mon ~]#
Nagios Client (10.167.176.43)
Code: Select all
ubuntuadmin@iusa-lin-db01:~$ /usr/local/nagios/libexec/check_http -V
check_http v2.2.1.git (nagios-plugins 2.2.1)
ubuntuadmin@iusa-lin-db01:~$ /usr/local/nagios/libexec/check_ssh -V
check_ssh v2.2.1.git (nagios-plugins 2.2.1)
ubuntuadmin@iusa-lin-db01:~$
Nagios Server (45.55.251.117)
Code: Select all
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 45.55.251.117
HTTP OK: HTTP/1.1 200 OK - 664 bytes in 0.007 second response time |time=0.007303s;;;0.000000 size=664B;;;0
Re: Not able to monitor Apache and SSH services
Posted: Fri Feb 09, 2018 1:45 pm
by shamrozkadiwal
I saw this error when I tried to run "Re-schedule the next check of this service". I don't know this could be a help.
Re: Not able to monitor Apache and SSH services
Posted: Fri Feb 09, 2018 3:51 pm
by kyang
Could you create a new thread for that issue?
That way we are not troubleshooting 2 errors in one thread.
It would be less confusing for us techs as well. Thank you!
Can you run
nmap 10.167.176.43 from your Nagios Server and not from the same server itself?.
Can you also run this from your Nagios Server, please post the output.
Code: Select all
/usr/local/nagios/libexec/check_http -H 10.167.176.43
Re: Not able to monitor Apache and SSH services
Posted: Fri Feb 09, 2018 8:14 pm
by shamrozkadiwal
@kyang, nmap is saying Host is down whereas It is up.
Code: Select all
[root@dj-mon ~]# /usr/local/nagios/libexec/check_http -H 10.167.176.43
CRITICAL - Socket timeout after 10 seconds
[root@dj-mon ~]#
[root@dj-mon ~]#
[root@dj-mon ~]# nmap 10.167.176.43
Starting Nmap 6.40 ( http://nmap.org ) at 2018-02-10 01:11 UTC
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 3.05 seconds
[root@dj-mon ~]#
[root@dj-mon ~]#
Re: Not able to monitor Apache and SSH services
Posted: Fri Feb 09, 2018 8:53 pm
by shamrozkadiwal
So I checked nrpe service on
10.167.176.43 and it was not runing.
Code: Select all
ubuntuadmin@iusa-lin-db01:/usr/local/nagios/var$ sudo systemctl status nrpe
● nrpe.service - Nagios Remote Plugin Executor
Loaded: loaded (/lib/systemd/system/nrpe.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2018-02-09 19:41:40 CST; 7s ago
Docs: http://www.nagios.org/documentation
Process: 25027 ExecStopPost=/bin/rm -f /usr/local/nagios/var/nrpe.pid (code=exited, status=0/SUCCESS)
Process: 25022 ExecStart=/usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -f (code=exited, status=2)
Main PID: 25022 (code=exited, status=2)
Feb 09 19:41:40 iusa-lin-db01 systemd[1]: Started Nagios Remote Plugin Executor.
Feb 09 19:41:40 iusa-lin-db01 systemd[1]: nrpe.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 09 19:41:40 iusa-lin-db01 systemd[1]: nrpe.service: Unit entered failed state.
Feb 09 19:41:40 iusa-lin-db01 systemd[1]: nrpe.service: Failed with result 'exit-code'.
When I checked the logs (/usr/local/nagios/var/nrpe.log), it showed me this. There are three certificate errors, but not sure are those the reason to not able to check the services. One more weird thing that I am able to monitor other services on this server like Load, Partition, users and they all are working fine.
Code: Select all
[1518226046] parse_allowed_hosts: parsing the allowed host string >127.0.0.1,::1,45.55.251.117< to add to ACL list
[1518226046] parse_allowed_hosts: ADDING this record (127.0.0.1) to ACL list!
[1518226046] add_ipv4_to_acl: checking ip-address >127.0.0.1<
[1518226046] add_ipv4_to_acl: ip-address >127.0.0.1< correct, adding.
[1518226046] parse_allowed_hosts: Record added to ACL list!
[1518226046] parse_allowed_hosts: ADDING this record (::1) to ACL list!
[1518226046] parse_allowed_hosts: Record added to ACL list!
[1518226046] parse_allowed_hosts: ADDING this record (45.55.251.117) to ACL list!
[1518226046] add_ipv4_to_acl: checking ip-address >45.55.251.117<
[1518226046] add_ipv4_to_acl: ip-address >45.55.251.117< correct, adding.
[1518226046] parse_allowed_hosts: Record added to ACL list!
[1518226046] Showing ACL lists for both IP and DOMAIN acl's:
[1518226046] IP ACL: 127.0.0.1/32 16777343
[1518226046] IP ACL: 0.0.0.0/20 0
[1518226046] IP ACL: 45.55.251.117/32 1979397933
[1518226046] Added command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
[1518226046] Added command[check_load]=/usr/local/nagios/libexec/check_load -r -w .15,.10,.05 -c .30,.25,.20
[1518226046] Added command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
[1518226046] Added command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
[1518226046] Added command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
[1518226046] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:02001002:system library:fopen:No such file or directory
[1518226046] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:20074002:BIO routines:FILE_CTRL:system lib
[1518226046] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib
[1518226900] parse_allowed_hosts: parsing the allowed host string >127.0.0.1,::1,45.55.251.117< to add to ACL list
[1518226900] parse_allowed_hosts: ADDING this record (127.0.0.1) to ACL list!
[1518226900] add_ipv4_to_acl: checking ip-address >127.0.0.1<
[1518226900] add_ipv4_to_acl: ip-address >127.0.0.1< correct, adding.
[1518226900] parse_allowed_hosts: Record added to ACL list!
[1518226900] parse_allowed_hosts: ADDING this record (::1) to ACL list!
[1518226900] parse_allowed_hosts: Record added to ACL list!
[1518226900] parse_allowed_hosts: ADDING this record (45.55.251.117) to ACL list!
[1518226900] add_ipv4_to_acl: checking ip-address >45.55.251.117<
[1518226900] add_ipv4_to_acl: ip-address >45.55.251.117< correct, adding.
[1518226900] parse_allowed_hosts: Record added to ACL list!
[1518226900] Showing ACL lists for both IP and DOMAIN acl's:
[1518226900] IP ACL: 127.0.0.1/32 16777343
[1518226900] IP ACL: 0.0.0.0/20 0
[1518226900] IP ACL: 45.55.251.117/32 1979397933
[1518226900] Added command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
[1518226900] Added command[check_load]=/usr/local/nagios/libexec/check_load -r -w .15,.10,.05 -c .30,.25,.20
[1518226900] Added command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
[1518226900] Added command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
[1518226900] Added command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
[1518226900] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:02001002:system library:fopen:No such file or directory
[1518226900] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:20074002:BIO routines:FILE_CTRL:system lib
[1518226900] Error: could not use certificate file /etc/letsencrypt/live/monitor.theismailiusa.org/cert.pem : error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib
ubuntuadmin@iusa-lin-db01:/usr/local/nagios/var$ ^C