"Waiting for configuration verification" death trap

[capriguy84]

Hello,

I have a pilot Nagios XI installed and this issue is preventing me from adding new hosts & checks. I have seen couple of threads discusssing the same issue and after trying out all of them, I still have the problem.

OS: RHEL 6.2
NagiosXI: 2012R1.7
Pls find attached profile.zip for more info.

Steps taken so far:
1) Edit /etc/php.ini

Code: Select all

max_execution_time = 60
max_input_time = 60
memory_limit = 512M

2) /etc/hosts

Code: Select all

127.0.0.1   localhost localhost.localdomain
10.5.1.57   nagios1 nagios1.site1.domain.corp

3) Ran the rest_config_prems & reconfigure_nagios.sh scripts and it returned OK.

4) Re-installed latest copy of ndoutils-1.5.2.tar.gz

5) Stop & Start nd02db and nagios.

Code: Select all

> # /etc/init.d/ndo2db restart
Stopping ndo2db: head: cannot open [b]`/usr/local/nagios/var/ndo2db.lock'[/b] for reading: No such file or directory
done.
Starting ndo2db: done. 

Code: Select all

> # ls -l /usr/local/nagios/var/
total 244
drwxrwxr-x 2 nagios nagios  4096 Mar 28 00:00 archives
-rw-rw-r-- 1 nagios nagios   296 Mar 28 11:16 host-perfdata
-rw-r--r-- 1 nagios nagios     6 Mar 28 11:04 nagios.lock
-rw-r--r-- 1 nagios nagios 40937 Mar 28 11:12 nagios.log
-rw-r--r-- 1 nagios nagios     6 Mar 28 11:04 ndo2db.lock
-rw-rw-r-- 1 nagios nagios     0 Mar 28 11:04 ndomod.tmp
srwxr-xr-x 1 nagios nagios     0 Mar 28 11:04 ndo.sock
-rw-r--r-- 1 nagios nagios   834 Mar 27 19:19 npcd.log
-rw-r--r-- 1 nagios nagios 53043 Mar 28 11:04 objects.cache
-rw------- 1 nagios nagios 54348 Mar 28 11:04 retention.dat
drwxrwsr-x 2 nagios nagcmd  4096 Mar 28 11:04 rw
-rw-rw-r-- 1 nagios nagios   685 Mar 28 11:16 service-perfdata
drwxr-xr-x 5 nagios   nagios    4096 Mar 27 17:47 spool
drwxr-xr-x 2 nagios nagios  4096 Mar 27 17:48 stats
-rw-rw-r-- 1 nagios nagios 54888 Mar 28 11:16 status.dat

Does the perms look alright?

Code: Select all

> # ll /usr/local/nagiosxi/
total 32
drwxr-xr-x  2 root   root   4096 Mar 27 17:53 cron
drwxr-xr-x  3 root   root   4096 Mar 27 17:53 etc
drwxr-xr-x 18 apache nagios 4096 Mar 27 19:26 html
drwxr-xr-x  3 root   root   4096 Mar 27 17:53 nom
drwxr-xr-x  2 nagios nagios 4096 Mar 28 11:34 scripts
drwsrwsr-x  2 apache nagios 4096 Mar 27 17:58 tmp
drwxr-xr-x  2 root   root   4096 Mar 27 17:53 tools
drwxr-xr-x  4 root   root   4096 Mar 27 17:53 var

Any other ideas of what could be wrong? Also is there any specific log file that I could examine?

[slansing]

Does the following endlessly spin out as well?:

Code: Select all

cd /usr/local/nagiosxi/scripts

./reconfigure_nagios.sh

If not please post the output from it's return in the command line.

Also, please run the following while you try to apply configuration from the XI Interface:

Code: Select all

tail -f /var/log/httpd/error_log

[capriguy84]

Please see the attached file output of reconfigure_nagios.sh script. It seems like after I run reconfiguure it picked up all the hosts that I was trying to add. The initial thoughts seem to be permissions issue, somewhere.


> tail -f /var/log/httpd/error_log

Code: Select all

[error] [client 10.5.1.110] PHP Notice:  Undefined variable: sa in /opt/nagiosxi/html/config/configobject.php on line 951, referer: http://nagios1.nyc.laurion.corp/nagiosxi/includes/components/xicore/status.php?show=hostdetail&host=exchange-dr1.site.domain.corp

[slansing]

Oh you were importing Hosts/Services? This can take a while depending on the amount of items you are importing. Are you able to successfully apply configuration now in a timely manner? I will do some digging on that Apache error.

[capriguy84]

I am only importing like 1-3 hosts checks. Not much.

The problem still persists, the waiting for verification continues. I still cannot get the config to show in console unless I run reconfigure_nagios.sh scrip. This is annoying.

On same lines, I cannot submit a check immediately for any service. It says timed out.

Does these perms look right?

Code: Select all

ll /opt/nagiosxi/
total 32
drwxr-xr-x  2 root   root   4096 Mar 27 17:53 cron
drwxr-xr-x  3 root   root   4096 Mar 27 17:53 etc
drwxr-xr-x 18 apache nagios 4096 Mar 27 19:26 html
drwxr-xr-x  3 root   root   4096 Mar 27 17:53 nom
drwxr-xr-x  2 nagios nagios 4096 Mar 28 11:34 scripts
drwsrwsr-x  2 apache nagios 4096 Mar 27 17:58 tmp
drwxr-xr-x  2 root   root   4096 Mar 27 17:53 tools
drwxr-xr-x  4 root   root   4096 Mar 27 17:53 var

[abrist]

Your nagiosxi directory is in /opt. Is this symlinked?

When you imported your configs, did you use the prep tool and walk through all the steps in the following documents?
http://assets.nagios.com/downloads/nagi ... nto_XI.pdf
http://assets.nagios.com/downloads/nagi ... p_Tool.pdf

[capriguy84]

/usrlocal/nagios & /usr/local/nagiosxi are symlinks to /opt/nagios & /opt/nagiosxi.

I just ran full install and it succeeded with no issues.

[abrist]

/usrlocal/nagios & /usr/local/nagiosxi are symlinks to /opt/nagios & /opt/nagiosxi.

I just ran full install and it succeeded with no issues.

Sounds good. Did you follow the official documents for import prep and configuration of core files?

[capriguy84]

I did the import on he XI console. It went thru fine.

So here's what I am doing to add a new host.

Config Wizard -> Select Type (Say Linux Server) -> Go thru the wizard by selecting default checks and hit finish. Then the verification step hangs.

[scottwilkerson]

This sounds like crond might not be running

Can you send the output of the following

Code: Select all

tail /var/log/cron