Issues with external commands via Nagios Web Gui

[dortiz750]

Hello all,

I am hoping someone can help me with a rare issue I have been experiencing for the past two days. I have built a brand new CentOS 7 VM with the standard web server configs hosting Nagios 4.0.4. I was able to successfully install Nagios and get all my configs working just fine. I was able to previously submit external commands via the Web Gui, Example (Acknowledge alerts, schedule downtime) with no issues about a week ago. But now when I try to run any functions that reference cgi.cmd via the web gui I receive a generic Internal Server error. The only changes I have made from now to last week was add service dependencies, service escalations, and host escalations in to my Windows.cfg file, restarted apache and Nagios a couple of times. I believe the issue is pointing in the Apache direction. I do not see any errors on the Nagios side. I am not an Apache expert so I was just wondering if anyone can help me with this? I would greatly appreciate any input.

I have attached some logs and server information below for reference:

OS Version:

CentOS Linux release 7.2.1511 (Core)

Apache Version:

[root@njcpinfmon06 nrpe-2.15]# httpd -V
Server version: Apache/2.4.6 (CentOS)
Server built: Nov 19 2015 21:43:13
Server's Module Magic Number: 20120211:24
Server loaded: APR 1.4.8, APR-UTIL 1.5.2
Compiled using: APR 1.4.8, APR-UTIL 1.5.2
Architecture: 64-bit
Server MPM: prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/etc/httpd"
-D SUEXEC_BIN="/usr/sbin/suexec"
-D DEFAULT_PIDLOG="/run/httpd/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"

Nagios Version:

Nagios Core 4.0.4
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 03-14-2014
License: GPL

Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
Checked 5 services.
Checked 1 hosts.
Checked 2 host groups.
Checked 0 service groups.
Checked 10 contacts.
Checked 5 contact groups.
Checked 26 commands.
Checked 8 time periods.
Checked 4 host escalations.
Checked 20 service escalations.
Checking for circular paths...
Checked 1 hosts
Checked 8 service dependencies
Checked 0 host dependencies
Checked 8 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check

Web Gui error:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.


Apache error log:

[Fri Feb 19 13:25:19.019391 2016] [mpm_prefork:notice] [pid 18748] AH00170: caught SIGWINCH, shutting down gracefully
[Fri Feb 19 13:25:20.172882 2016] [suexec:notice] [pid 18989] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Feb 19 13:25:20.224897 2016] [auth_digest:notice] [pid 18989] AH01757: generating secret for digest authentication ...
[Fri Feb 19 13:25:20.226089 2016] [lbmethod_heartbeat:notice] [pid 18989] AH02282: No slotmem from mod_heartmonitor
[Fri Feb 19 13:25:20.256557 2016] [mpm_prefork:notice] [pid 18989] AH00163: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 PHP/5.4.16 configured -- resuming normal operations
[Fri Feb 19 13:25:20.256606 2016] [core:notice] [pid 18989] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Fri Feb 19 13:28:03.820323 2016] [cgi:error] [pid 19089] [client 10.137.130.174:61598] End of script output before headers: cmd.cgi, referer: http://njcpinfmon06/nagios/cgi-bin/cmd. ... ice=AD3360


Apache Access log:

10.137.130.174 - dortiz [19/Feb/2016:13:28:00 -0500] "GET /nagios/cgi-bin/cmd.cgi?cmd_typ=160&host=NJCPINFADM01&service=AD3360 HTTP/1.1" 200 3755 "http://njcpinfmon06/nagios/cgi-bin/exti ... ice=AD3360" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0E; .NET4.0C; InfoPath.3)"
10.137.130.174 - dortiz [19/Feb/2016:13:28:03 -0500] "POST /nagios/cgi-bin/cmd.cgi HTTP/1.1" 500 527 "http://njcpinfmon06/nagios/cgi-bin/cmd. ... ice=AD3360" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0E; .NET4.0C; InfoPath.3)"
::1 - - [19/Feb/2016:13:28:04 -0500] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 PHP/5.4.16 (internal dummy connection)"


I have also uploaded my httpd.conf, nagios.conf files.

Thank you ahead of time!

[hsmith]

Is it possible selinux is messing with you?

Run the following command:

Code: Select all

getenforce

[nozlaf]

The only changes I have made from now to last week was add service dependencies, service escalations, and host escalations in to my Windows.cfg file

thanks for posting a detailed request with decent amount of information in it
i'm interested if you can post the config for your host "NJCPINFADM01"
and service configuration for the service "AD3360"

have you tried external commands which dont specially deal with a host or service like for example turning off all notifications http://njcpinfmon06/nagios/cgi-bin/cmd.cgi?cmd_typ=11 (disable notifications)

[jolson]

@dortiz750 - if you could get back to us with answers to the above posts, we'd appreciate it. Thank you!