Service Commands - No longer working

[allthingstechy]

Greetings Nagios Family,

Nagios® Core™ Version 3.4.1 - Please excuse the layman's terms.

Simply put, my installation of Nagios will no longer accept multiple "Host Commands" via the gui of any sort.

On reboot, I am able to submit 3 "Host Commands".

When I issue multiple "Host Commands" such as "Re-schedule the next check of this service" or "Schedule downtime for this host" - the GUI simply does not accept.

ps aux | grep nagios reveals that i have the exact same number of submits of the following running.

Apache /usr/local/nagios/sbin/cmd.cgi
Apache /usr/local/nagios/sbin/cmd.cgi
Apache /usr/local/nagios/sbin/cmd.cgi

ps aux | grep nagios

At this point the Nagios Service is no longer running.

Nagios-Not-Running

Nagios-Not-Running.JPG (13.17 KiB) Viewed 4028 times

It feels like an apache issue. tail-f /var/log/httpd/error_log reveals the following errors

[Thu Mar 19 09:15:19 2015] [error] [client 10.1.0.230] Premature end of script headers: cmd.cgi, referer: http://10.4.0.58/nagios/cgi-bin/cmd.cgi ... orce_check
[Thu Mar 19 09:15:25 2015] [warn] [client 10.1.0.230] Timeout waiting for output from CGI script /usr/local/nagios/sbin/cmd.cgi, referer: http://10.4.0.58/nagios/cgi-bin/cmd.cgi ... orce_check
[Thu Mar 19 09:15:25 2015] [error] [client 10.1.0.230] Premature end of script headers: cmd.cgi, referer: http://10.4.0.58/nagios/cgi-bin/cmd.cgi ... orce_check
[Thu Mar 19 09:16:22 2015] [warn] [client 10.1.0.230] Timeout waiting for output from CGI script /usr/local/nagios/sbin/cmd.cgi, referer: http://10.4.0.58/nagios/cgi-bin/cmd.cgi ... ua-bv00001
[Thu Mar 19 09:16:32 2015] [warn] [client 10.1.0.230] Timeout waiting for output from CGI script /usr/local/nagios/sbin/cmd.cgi, referer: http://10.4.0.58/nagios/cgi-bin/cmd.cgi ... orce_check


In the tail -f /usr/local/nagios/var/nagios.log - I can see the first request be transmitted, but all remaining request do not occur.

tail -f /usr/local/nagios/var/nagios.log

If i restart the Nagios and Httpd services, I can again submit (less than 3) host commands, then the system stops accepting.

Service Nagios restart
Service httpd restart

[jdalrymple]

Do you have selinux enabled:

Code: Select all

[jdalrymple@localhost ~]$ sestatus
SELinux status:                 disabled

On a Nagios server you shouldn't have, if you do you're asking for all sorts of trouble.

[jolson]

Greetings!

Please check on selinux as jdalrymple has suggested. Otherwise, can you please provide the following output:

Code: Select all

cat /usr/local/nagios/etc/cgi.cfg

Thanks!

[allthingstechy]

sestatus

sestatus.JPG (12.15 KiB) Viewed 4017 times

[allthingstechy]

Attached is the requested cgi.cfg file - i deleted all referecences to end-user accounts within the file with the exception of the nagios user.

[jolson]

I checked a default cgi.cfg file, and the biggest difference I found is as follows:

My lab machine file:

Code: Select all

# SYSTEM/PROCESS COMMAND ACCESS
# This option is a comma-delimited list of all usernames that
# can issue shutdown and restart commands to Nagios via the
# command CGI (cmd.cgi).  Users in this list can also change
# the program mode to active or standby. By default, *no one*
# has access to this unless you choose to not use authorization.
# You may use an asterisk (*) to authorize any user who has
# authenticated to the web server.

authorized_for_system_commands=nagiosadmin

Your file:

Code: Select all

# SYSTEM/PROCESS COMMAND ACCESS
# This option is a comma-delimited list of all usernames that
# can issue shutdown and restart commands to Nagios via the
# command CGI (cmd.cgi).  Users in this list can also change
# the program mode to active or standby. By default, *no one*
# has access to this unless you choose to not use authorization.
# You may use an asterisk (*) to authorize any user who has
# authenticated to the web server.

authorized_for_system_commands=nagios

Are you initiating the service checks in question from the 'nagiosadmin' account in the webGUI? If so, add nagiosadmin and restart your nagios/httpd processes. Let me know if this helps!

[allthingstechy]

I'm actually logging into the homepage with an account called brad. I took out all the usernames from the cgi.cfg file - just out of safety.

[jolson]

Oh okay - and brad is listed as authorized_for_system_commands?

[allthingstechy]

absolutely ! 100%

[jdalrymple]

I suggest turning up the debugging, maybe set debug_level=448, debug_verbosity=2 and max_debug_file_size=10485760.

I don't necessarily think it's an httpd thing, but maybe.