Views do not accurately reflect actual CCM setting

[FLCUISIT]

After a machine reboot on 6/21/13, host/service adds/changes/deletes made to the configuration in CCM do not appear in the web view. But, the monitors set up for these hosts and services do send alerts. We've "repaired" the sql databases with the nagios provided utilities on the linux server. The permissions look to be correct. We're running Nagios XI 2012R2.2 on a linux server CentOS release 5.4 (Final) Kernel 2.6.18-164.9.1.e15 on an i686.

Additional data about the machine's (unusual) behavior (which may or may not be relevant):

(1) the five minute load average is consistently over 9 points
(2) there are 16 instances of /usr/sbin/httpd running (only two or three users are consistently access the web view of nagios)
(3) there are two (?) nagios processes running:
nagios 5686 1 3 10:19 ? 00:00:09 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 25284 5686 0 10:23 ? 00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

Thanks very much for any help.

[abrist]

(2) there are 16 instances of /usr/sbin/httpd running (only two or three users are consistently access the web view of nagios)
(3) there are two (?) nagios processes running:
nagios 5686 1 3 10:19 ? 00:00:09 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 25284 5686 0 10:23 ? 00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

2) Standard behavior.
3) Standard behavior as well (nagios is forked to run checks)

host/service adds/changes/deletes made to the configuration in CCM do not appear in the web view

Open up a terminal to the XI box, execute the following running tail. While the tail is running try to browse to one of the broken details pages:

Code: Select all

tail -f /var/log/httpd/error_log

Now browse to one of the pages not showing added hosts. Post the errors from the tail.

[FLCUISIT]

nothing new appeared in the tail as I browsed to a host that had been deleted in CCM, but was still showing up in home > hosts, but there were some other errors (not sure if they're relevant). Please see below:

[Wed Jun 26 15:14:44 2013] [error] [client 192.168.21.43] PHP Notice: Undefined index: avg_execution_time in /usr/local/nagiosxi/html/includes/utils-xmlsysstat.inc.php on line 120, referer: http://192.168.21.39/nagiosxi/index.php?
which: no gnome-session in (/sbin:/usr/sbin:/bin:/usr/bin)
[Fri Jun 28 10:19:30 2013] [error] [client 192.168.21.61] request failed: URI too long (longer than 8190)
[Fri Jun 28 11:19:21 2013] [error] [client 192.168.21.61] request failed: URI too long (longer than 8190)
[Fri Jun 28 11:47:13 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... wizard.php
[Fri Jun 28 11:47:13 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... wizard.php
[Fri Jun 28 11:47:13 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... wizard.php
[Fri Jun 28 11:58:15 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... nterserver
[Fri Jun 28 11:58:15 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... nterserver
[Fri Jun 28 11:58:15 2013] [error] [client 192.168.21.61] PHP Notice: Undefined variable: host_datastore_name in /usr/local/nagiosxi/html/includes/configwizards/vmwareesxvspherevcenterserver/vmwareesxvspherevcenterserver.inc.php on line 352, referer: http://192.168.21.39/nagiosxi/config/mo ... nterserver

[slansing]

Can you browse to one of the broken "details" pages when running the tail and share the output? It looks like you were looking at configuration wizards, unless this was prior to opening a details page.

[FLCUISIT]

Perhaps this isn't an httpd problem, while tailing the log when I click on host detail > advanced > reconfigure this host, I get no log errors. I see this message: "This host appears to make use of an advanced configuration. Use the Nagios Core Config Manager to modify the settings for this host." I see the same message when I try to delete this host. This host has already been deleted in the CCM.

Edit: Sorry to forget. There is also no configuration file for the "phantom" host in the linux directory structure.

[abrist]

If the host has already been deleted from the ccm, but is still appearing in the interface, then it must still have a config file. Try:

Go to --> Configure --> Core Config Manager --> Tools --> write config files --> Click "write" and then "verify". Do *not* click "restart or "apply configuration" just yet. Resolve any errors in the CCM and then rewite the files and reverify. Once verification is clean, go to the directories:

Code: Select all

/usr/local/nagios/etc/hosts

And

Code: Select all

/usr/local/nagios/etc/services

And remove the config files for the ghost host.
Write and verify one last time, if everything is clean, you can can then "apply configuration".

[FLCUISIT]

This helped very much. Thanks.

[slansing]

Do you have any other questions in regards to this thread?

[FLCUISIT]

No. I may post something else on a different, but related topic. Thanks.

[slansing]

Sounds good, we will lock this thread up, open a new one if you require assistance, thanks!