Whoops!, was working fine, then it lost its configs

[AcademyNag]

Our Nagios Core has been working fine for quite some time, over a year, and suddenly yesterday I get "Whoops! Error: Could not read host and service status information!"
after looking around at the errors, I am getting this
"Error: There are no services defined!
Error: There are no hosts defined!
Error: There are no contacts defined!
Total Warnings: 0
Total Errors: 3 "
I am new at this job and my team-mate is on vacation, please help.

[abrist]

Lets first check for the locations for config data:

Code: Select all

grep cfg_ /usr/local/nagios/etc/nagios.cfg

Are you using nagiosql?

[AcademyNag]

Thank You Very much for the reply, I made the command decision to restore the server back to the 23rd, before it started this behavior, and all "seems" fine, however, there are several issues with it- as in before I did have permission to view the "Event Log" in the monitoring web-interface and now I dont. and- i write all configs and verify fine, no errors- until i make some changes. This must be a permissions error- could you tell me the correct method of starting Nagios (and all other services it needs) from a reboot? I've searched the history of root and nagios and the only start/stop/restarts are from root with :

Code: Select all

/etc/init.d/nagios <start/stop/restart>

Here's the output you asked for (for future refference:

Code: Select all

 # grep cfg_ /usr/local/nagios/etc/nagios.cfg | grep -v "#"
cfg_file=/usr/local/nagios/etc/nagiosql/contacttemplates.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/hosttemplates.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/servicetemplates.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/commands.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/contactgroups.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/contacts.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/timeperiods.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/servicegroups.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/servicedependencies.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/hostdependencies.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/hostescalations.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/hostextinfo.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/serviceextinfo.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/serviceescalations.cfg
cfg_file=/usr/local/nagios/etc/nagiosql/hostgroups.cfg
cfg_dir=/usr/local/nagios/etc/nagiosql/hosts
cfg_dir=/usr/local/nagios/etc/nagiosql/services

Again, Thank you so much for the help, we have "Premium Support" for My new XI upgrade project, but until that's ready to deploy I need to keep the old servers working.

[slansing]

What is the check configuration error you are receiving..? The same one as above? Also, what do you mean you do not have permission to view the Event Log? Can you run the following, and try to access the page then post the output it generates?:

Code: Select all

tail -f /var/log/httpd/access_log

And again for:

Code: Select all

tail -f /var/log/httpd/error_log

[AcademyNag]

The attachments are from the output of your requests-

Code: Select all

# tailf /var/log/apache2/access_log | tee acc-log.txt 
# tailf /var/log/apache2/error_log | tee err-log.txt

While capturing...
On http://<hostname>.academy,com/nagios- 'Reports > "Event Log" ' >>
It appears as though you do not have permission to view the log file...

If you believe this is an error, check the HTTP server authentication requirements for accessing this CGI
and check the authorization options in your CGI configuration file.

[slansing]

Can you post your user's username and also send us a copy of:

Code: Select all

/usr/local/nagios/etc/cgi.cfg

And the output of:

Code: Select all

cat /etc/httpd/conf.d/nagios.conf

[AcademyNag]

Mine is smiller, but my boss also used to be able to view it-
Attaching requests:- I had to add a .txt extension to be allowed to attach- just fyi.
(Just a note- In SLES-11, the path is-"/etc/apache2/conf.d/nagios.conf")

[slansing]

Are you logged in as the nagiosadmin? Or another user? If you are using another user you must add it's name into the authorized_for fields:

Code: Select all

authorized_for_all_hosts=nagiosadmin,yourusernamehere

Etc.

[AcademyNag]

Hrm, then that makes me wonder- what happened- we have always authenticated via ldap with our own user/pass (except for the "/nagiosql3/index.php"), and it has always allowed, me, my teammate, and our boss, to view it. well I guess thats the minor issue- it may be a symptom of a bigger issue- with permissions, I received many errors prior to restoring- and things just dont seem quite right- Example,
I select a host that has Notifications turned off, I select 'Enable notifications for all services on this host' >commit, Repeat for each Service on that host- Done all works, but if i go to the Admin panel (/nagiosql3/index.php), > Tools>Nagios Control> write {monitoring,additional} data > write configuration files,,, lots of errors (only 1-2 but more after those are resolved, like dependencies). Key thing is- I did this same data-writing successfully, just prior to enabling the notifications on that/any server.- That happened multiple times (i just reloaded the last snapshot afterwards,) and so far it hasnt happened again, but, this perms thing with the event log is definitely related.
For now- its working again, I hope that we can find the cause of it to begin with, or at least a quicker fix if it happens again-
Again--Thank-You
Shawn Miller

[slansing]

Well let us know if it happens again, we will need more concrete information to work from, as the current info is a little vague. It does appear to have something to do with authentication. If you are not logged in as the nagios admin, your cgi.cfg file dictates you should not have access to the pages you are trying to reach that are bouncing you back. But, let us know if this resurfaces, the thread will be kept open.