Web service timing out

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
mrkrupp
Posts: 22
Joined: Fri Apr 01, 2016 5:23 pm

Web service timing out

Post by mrkrupp »

Hello,

A few days I had Nagios Core all setup to go and everything was functioning, including the web service. Now I am getting a "this page cannot be displayed error" when I try to go the http://xxx.xx.xx.xx/nagios page. I've restarted the server, checked that the firewall on my server is disabled, restarted the nagios and httpd service and made sure they are definitely still linked. Any ideas or help would be greatly appreciated.

Thanks!
User avatar
hsmith
Agent Smith
Posts: 3539
Joined: Thu Jul 30, 2015 11:09 am
Location: 127.0.0.1
Contact:

Re: Web service timing out

Post by hsmith »

Is there anything else running on this server?

What's the output of these commands?

Code: Select all

tail /var/log/httpd/error_log
service httpd status
cat /etc/*release*
getenforce
Former Nagios Employee.
me.
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: Web service timing out

Post by rkennedy »

Can you post your logs for us to take a look at? (assuming your on CentOS)

Code: Select all

/var/log/httpd/error_log
/var/log/httpd/access_log
Also, we'll need to see /etc/httpd/conf/httpd.conf and your entire /etc/httpd/conf.d/ directory (feel free to zip it up).
Former Nagios Employee
mrkrupp
Posts: 22
Joined: Fri Apr 01, 2016 5:23 pm

Re: Web service timing out

Post by mrkrupp »

Update:

Thank you all for your help. I was able to get the web service working again. but the issue I'm having now is when I am in the web service and want to view my hosts and services I get the following error:

Forbidden

You don't have permission to access /nagios/cgi-bin/status.cgi on this server.

I am on RHEL 7. And I'm running Nagios 4.1.1 with the plugins 2.1.1 This was all working properly before and I havent changed anything manually so I'm not sure what caused the issue.

Getenforce- Disabled

error log

Code: Select all

tail /var/log/httpd/error_log
[Thu Apr 28 10:59:58.929334 2016] [autoindex:error] [pid 4731] [client 127.0.0.1:35804] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:04:58.906829 2016] [autoindex:error] [pid 4218] [client 127.0.0.1:35808] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:09:58.884736 2016] [autoindex:error] [pid 4214] [client 127.0.0.1:35812] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:14:58.862423 2016] [autoindex:error] [pid 4221] [client 127.0.0.1:35816] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[root@bauspap218 ~]# tail /var/log/httpd/error_log
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 10:59:58.929334 2016] [autoindex:error] [pid 4731] [client 127.0.0.1:35804] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:04:58.906829 2016] [autoindex:error] [pid 4218] [client 127.0.0.1:35808] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:09:58.884736 2016] [autoindex:error] [pid 4214] [client 127.0.0.1:35812] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:14:58.862423 2016] [autoindex:error] [pid 4221] [client 127.0.0.1:35816] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[root@bauspap218 ~]# [Thu Apr 28 10:59:58.929334 2016] [autoindex:error] [pid 4731] [client 127.0.0.1:35804] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# [Thu Apr 28 11:04:58.906829 2016] [autoindex:error] [pid 4218] [client 127.0.0.1:35808] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# [Thu Apr 28 11:09:58.884736 2016] [autoindex:error] [pid 4214] [client 127.0.0.1:35812] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# [Thu Apr 28 11:14:58.862423 2016] [autoindex:error] [pid 4221] [client 127.0.0.1:35816] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# [Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
-bash: [Thu: command not found
[root@bauspap218 ~]# [Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
-bash: [Thu: command not found
[root@bauspap218 ~]# [Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
-bash: [Thu: command not found
[root@bauspap218 ~]# [Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
-bash: [Thu: command not found
[root@bauspap218 ~]# [Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# [Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
-bash: syntax error near unexpected token `('
[root@bauspap218 ~]# tail /var/log/httpd/error_log
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated di[Thu Apr 28 10:59:58.929334 2016] [autoindex:error] [pid 4731] [client 127.0.0.1:35804] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directiver
[Thu Apr 28 11:04:58.906829 2016] [autoindex:error] [pid 4218] [client 127.0.0.1:35808] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:09:58.884736 2016] [autoindex:error] [pid 4214] [client 127.0.0.1:35812] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:14:58.862423 2016] [autoindex:error] [pid 4221] [client 127.0.0.1:35816] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directive
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:23.713323 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:19:58.840519 2016] [autoindex:error] [pid 4219] [client 127.0.0.1:35820] AH01276:ectory inde Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directivex
 forbid[Thu Apr 28 11:24:58.818157 2016] [autoindex:error] [pid 4730] [client 127.0.0.1:35824] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html,index.php) found, and server-generated directory index forbidden by Options directived
en by Options directive
[root@bauspap218 ~]# tail /var/log/httpd/error_log
[Thu Apr 28 11:18:20.306027 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/statusjson.cgi, referer: http://172.17.205.81/nagios/main.php
[Thu Apr 28 11:18:22.120275 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http://172.17.205.81/nagios/side.php
[Thu Apr 28 11:18:22.829417 2016] [access_compat:error] [pid 4216] [client 172.17.206.144:60699] AH01797: client denied by server configuration: /usr/local/nagios/sbin/status.cgi, referer: http:[root@bauspap218 ~]#
Access Log

Code: Select all

tail /var/log/httpd/access_log
172.17.206.144 - nagiosadmin [28/Apr/2016:11:18:20 -0700] "GET /nagios/images/sflogo.png HTTP/1.1" 200 469 "http://172.17.205.81/nagios/main.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - nagiosadmin [28/Apr/2016:11:18:20 -0700] "GET /nagios/images/passiveonly.gif HTTP/1.1" 200 882 "http://172.17.205.81/nagios/main.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - - [28/Apr/2016:11:18:20 -0700] "GET /nagios/cgi-bin/statusjson.cgi?query=programstatus HTTP/1.1" 403 231 "http://172.17.205.81/nagios/main.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - nagiosadmin [28/Apr/2016:11:18:20 -0700] "GET /nagios/images/disabled.gif HTTP/1.1" 200 100 "http://172.17.205.81/nagios/main.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - - [28/Apr/2016:11:18:22 -0700] "GET /nagios/cgi-bin/status.cgi?hostgroup=all&style=hostdetail HTTP/1.1" 403 227 "http://172.17.205.81/nagios/side.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - - [28/Apr/2016:11:18:22 -0700] "GET /nagios/cgi-bin/status.cgi?hostgroup=all&style=overview HTTP/1.1" 403 227 "http://172.17.205.81/nagios/side.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
172.17.206.144 - - [28/Apr/2016:11:18:23 -0700] "GET /nagios/cgi-bin/status.cgi?host=all HTTP/1.1" 403 227 "http://172.17.205.81/nagios/side.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
127.0.0.1 - - [28/Apr/2016:11:19:58 -0700] "GET / HTTP/1.0" 403 3985 "-" "check_http/v2.1.1 (nagios-plugins 2.1.1)"
127.0.0.1 - - [28/Apr/2016:11:24:58 -0700] "GET / HTTP/1.0" 403 3985 "-" "check_http/v2.1.1 (nagios-plugins 2.1.1)"
127.0.0.1 - - [28/Apr/2016:11:29:58 -0700] "GET / HTTP/1.0" 403 3985 "-" "check_http/v2.1.1 (nagios-plugins 2.1.1)"
Thanks for your help again.
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: Web service timing out

Post by rkennedy »

rkennedy wrote:Also, we'll need to see /etc/httpd/conf/httpd.conf and your entire /etc/httpd/conf.d/ directory (feel free to zip it up).
Please post both of these as the logs indicate it is permission related. What version of apache are you running? apachectl -v
Former Nagios Employee
mrkrupp
Posts: 22
Joined: Fri Apr 01, 2016 5:23 pm

Re: Web service timing out

Post by mrkrupp »

Server version: Apache/2.4.6 (Red Hat Enterprise Linux)

httpd.conf and conf.d zipped:

https://www.dropbox.com/s/lpwtfxclh7s6p ... F.zip?dl=0

Thanks,
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: Web service timing out

Post by rkennedy »

In your apache configuration, nagios.conf, look for -

Code: Select all

<Directory "/usr/local/nagios/sbin">
#  SSLRequireSSL
   Options ExecCGI
   AllowOverride None
   Order allow,deny
   <IfVersion >= 2.3>
Remove this -

Code: Select all

   Order allow,deny
Then, service httpd restart. Let us know if that helps.
Former Nagios Employee
mrkrupp
Posts: 22
Joined: Fri Apr 01, 2016 5:23 pm

Re: Web service timing out

Post by mrkrupp »

This fixed my issue! Do you know what could have possibly overridden that in the first place?

Thanks very much,
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: Web service timing out

Post by rkennedy »

To be honest, not a clue. If this machine was previously on apache 2.2, then that could be why, but with Core I don't have an idea at all.

Glad to see you got it working though!

Going to mark this as resolved, and close it out. If you need any assistance in the future, feel free to create a new thread.
Former Nagios Employee
Locked