Internal Server Error When Attempting to Schedule Downtime

[bfortman]

I'm not seeing anything in your make log either.

Please post the output of these commands:

Code: Select all

ls -l /usr/local/nagios/var
ls -l /usr/local/nagios

# ls -l /usr/local/nagios/var
total 1356
drwxrwxr-x. 2 nagios nagios 4096 May 3 23:59 archives
-rw-r--r--. 1 nagios nagios 34 May 4 15:50 nagios.configtest
-rw-r--r-- 1 nagios nagios 6 May 4 15:50 nagios.lock
-rw-r--r-- 1 nagios nagios 38187 May 4 15:50 nagios.log
-rw-r--r--. 1 nagios nagios 250170 May 4 15:50 objects.cache
-rw-r--r--. 1 nagios nagios 250170 May 4 15:50 objects.precache
-rw------- 1 nagios nagios 406281 May 4 15:50 retention.dat
drwxrwsr-x. 2 nagios nagcmd 4096 May 4 15:50 rw
drwxrwxr-x. 3 nagios nagcmd 4096 Apr 13 13:56 spool
-rw-rw-r-- 1 nagios nagios 405617 May 4 16:42 status.dat
# ls -l /usr/local/nagios
total 28
drwxrwxr-x. 2 nagios nagios 4096 Apr 13 13:56 bin
drwxrwxr-x. 3 nagios nagios 4096 Apr 27 11:06 etc
drwxr-xr-x. 2 root root 4096 Apr 13 14:26 include
drwxrwxr-x. 3 nagios nagios 4096 Apr 13 14:26 libexec
drwxrwxr-x. 2 nagios nagios 4096 May 4 13:16 sbin
drwxrwxr-x. 11 nagios nagios 4096 Apr 13 14:26 share
drwxrwxr-x. 5 nagios nagios 4096 May 4 16:42 var

[bfortman]

I assume based upon the error that this happens when you are trying to get to the schedule downtime page?


What are the results of http://myserver/nagios/cgi-bin/extinfo.cgi

How about http://myserver/nagios/cgi-bin/cmd.cgi

I expect both of those to work fine based upon what you've shared so far. Would you care to gzip up your cgis and send them my way? Unless you did any mad customization (which would no doubt be the source - no pun intended - of your problem) there should be no sensitive information. I'd love to just drop them on my 6.6 machine and see if they CGIs work properly here.

I've attached the screen shot of the extinfo.cgi command. The cmd.cgi out just says "Error: No command was specified"

I would be glad to pass along the cgis. This system is a IBM Power 8 (ppc64). Let me know if that will work for you first.

[bfortman]

The account and settings do look like they work. Lets check a few more things. Run these commands.

Code: Select all

grep nagiosadmin /usr/local/nagios/etc/cgi.cfg
cat /etc/httpd/conf.d/nagios.conf

# grep nagiosadmin /usr/local/nagios/etc/cgi.cfg
authorized_for_system_information=nagiosadmin
authorized_for_configuration_information=nagiosadmin
authorized_for_system_commands=nagiosadmin
authorized_for_all_services=nagiosadmin
authorized_for_all_hosts=nagiosadmin
authorized_for_all_service_commands=nagiosadmin
authorized_for_all_host_commands=nagiosadmin


# cat /etc/httpd/conf.d/nagios.conf

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">
# SSLRequireSSL
Options ExecCGI
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid-user
</Directory>

Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">
# SSLRequireSSL
Options None
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid-user
</Directory>

[jdalrymple]

This system is a IBM Power 8 (ppc64). Let me know if that will work for you first.

It won't, I don't have any hardware like that at my disposal

So... since this is going to be impossible for me to replicate in-house, my guess is that it is somehow related to the platform. My guess is that the CGIs probably compiled fine, but that the cgi module is wonky. Was that a binary distribution of mod_cgi provided by RHN or some such?

Either way - I'm not sure where to go or how to fix it.

[bfortman]

Ok thanks for the assistance. All of the packages used for the build were downloaded from Red Hat repositories (except nagio source of course).

The scheduled downtime links at one point were actually working. They only failed intermitantly. Suddenly they don't work at all. I think the next course of action will be to re-compile/re-install or move to different OS or x86 platform.

[jdalrymple]

I think the next course of action will be to re-compile/re-install or move to different OS or x86 platform.

I envy the hardware you have at your disposal - I think ultimately you'd be doing yourself a huge favor though by calling the folks in your VMware team and asking them for an x86_64 vm. Continuing down the road your on would be fun, but an uphill battle all the way.

[bfortman]

Based on what I just discoverd, the up hill battle may be on the Nagios Developers:

The fix to this issue appeared to be a simple "mis-step" in my *.host.cfg file.

I have two hosts defined as such:

define host{
use aix-nonprod-server ; Name of host template to use
host_name tssbox1
alias tssbox1
}

define host{
use aix-nonprod-server ; Name of host template to use
host_name tssbox2
alias tssbox2
}


I have two Services defined as:

define service{
use nonprod-service ; Name of service template to use
hostgroup_name aix-nonprod
service_description PING
check_command check_ping!100.0,20%!500.0,60%
}

define service{
use nonprod-service ; Name of service template to use
hostgroup_name aix-nonprod
service_description SSH
check_command check_ssh
notifications_enabled 1
}

Finally, I have a host group defined as "aix-nonprod"

The two hosts, tssbox1 and tssbox2 were NOT defined in the host group "aix-nonprod" therefore rendereing NO services (SSH or PING) for these two servers. Even though this is not what I wanted, it shouldn't have mattered to Nagios.

The Nagios configuration checker said:
Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check

However, when I added the two hosts to the host group, things started working just fine.

[jdalrymple]

So to clear things up...

- You did not want those hosts in the hostgroup
- You did not want those hosts to have those 2 services
- After adding those 2 hosts to the hostgroup you were then able to schedule downtime

???

Is that all right? And then the takeaway would be that if a host is void of services (I assume) you cannot schedule downtime for that host? That would definitely be a bug - I'd have to test it.

[bfortman]

- You did not want those hosts in the hostgroup
I did want them in the hostgroup, but forgot to add them.

- You did not want those hosts to have those 2 services
I did want them to have the 2 services. But the service definitions are based on host_group, and therefore since they were not in the host_group, they were void of the 2 services.

- After adding those 2 hosts to the hostgroup you were then able to schedule downtime
Yes, the "Internal Server Error" went away and I was able to schedule downtime

And then the takeaway would be that if a host is void of services (I assume) you cannot schedule downtime for that host?
Not necessarily. Even with no services, the host is still being monitored for "UP/DOWN" correct?. I may still want schedule downtime for the host since Nagios will send a notification when it goes to the DOWN state.

[jdalrymple]

And then the takeaway would be that if a host is void of services (I assume) you cannot schedule downtime for that host?
Not necessarily. Even with no services, the host is still being monitored for "UP/DOWN" correct?. I may still want schedule downtime for the host since Nagios will send a notification when it goes to the DOWN state.

Right, I understand your logic, I don't understand Nagios' and I'll build a lab and test for bugs. I'll reach back into this forum thread after I have found out more.