New NagiosXI server apply configuration taking hours

[akepley]

I have created a new RHEL 6.8 AWS Ec2 instance with RDS backed database for Nagios 5.4.2. The only host at this point is localhost being monitored. I decided to apply configuration before importing new hosts/services and it has been spinning for about two hours now. Where can I view the progress of the apply? The ec2 instance is not topping out on CPU or Memory and neither is the RDS instance.

[avandemore]

What is the output of:

Code: Select all

# cd /usr/local/nagiosxi/scripts/
# ./reconfigure_nagios.sh

https://support.nagios.com/kb/article.php?id=34

[akepley]

I PM'd the result to you. Thanks!

[avandemore]

I think you need to follow the sudoers entry in the KB I listed.

Your config to applied fine when run as root.

[akepley]

I followed both the sudoers and the php.ini increase and still get the forever apply configuration

Waiting for configuration verification.....................................................

This will continue for hours and continue to build periods off the page.

AWS Ec2 instance, RDS for mysql, RHEL 6 build 4.5.2 NagiosXI

[avandemore]

Please show the full output for the following:

Code: Select all

# su - nagios
$ cd /usr/local/nagiosxi/scripts/
$ ./reconfigure_nagios.sh

[akepley]

Code: Select all

[email protected] scripts$ ./reconfigure_nagios.sh
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/cc                                                                                                                                                                      m/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&passw                                                                                                                                                                      ord=removed' -O nagiosql.login--2017-03-20 14:07:58--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=Tennessee/L=Nashville/O=NASBA/OU=IT/CN=nagios01.ec                                                                                                                                                                      2.prod.azw.nasba”:
  Self-signed certificate encountered.
    WARNING: certificate common name “nagios01.ec2.prod.azw.nasba” doesn't match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                         ] 32,589      --.-K/s   in 0.02s

2017-03-20 14:07:59 (1.97 MB/s) - “nagiosql.login” saved [32589]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: https://localhost/nagiosxi/includes/components/ccm/
Array
(
)
RESETTING PERMS
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/cc                                                                                                                                                                      m/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&passw                                                                                                                                                                      ord=removed' -O nagiosql.login--2017-03-20 14:08:00--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=Tennessee/L=Nashville/O=NASBA/OU=IT/CN=nagios01.ec                                                                                                                                                                      2.prod.azw.nasba”:
  Self-signed certificate encountered.
    WARNING: certificate common name “nagios01.ec2.prod.azw.nasba” doesn't match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”

    [ <=>                                                                         ] 32,589      --.-K/s   in 0.02s

2017-03-20 14:08:01 (1.82 MB/s) - “nagiosql.login” saved [32589]

LOGIN SUCCESSFUL!
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certifica                                                                                                                                                                      te --post-data 'backend=1&cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2017-03-20 14:08:01--  https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... 127.0.0.1
Connecting to localhost|127.0.0.1|:443... connected.
WARNING: cannot verify localhost’s certificate, issued by “/C=US/ST=Tennessee/L=Nashville/O=NASBA/OU=IT/CN=nagios01.ec                                                                                                                                                                      2.prod.azw.nasba”:
  Self-signed certificate encountered.
    WARNING: certificate common name “nagios01.ec2.prod.azw.nasba” doesn't match requested host name “localhost”.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.export.monitoring”

    [ <=>                                                                                                                                                                                                                                               ] 52,053      --.-K/s   in 0.02s

2017-03-20 14:08:39 (3.17 MB/s) - “nagiosql.export.monitoring” saved [52053]

WRITE CONFIGS SUCCESSFUL!
OUTPUT:
Nagios Core 4.2.4
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 12-07-2016
License: GPL

Website: https://www.nagios.org
Reading configuration data...
   Read main config file okay...
Warning: Duplicate definition found for service 'Ping' on host 'Gateway AS400 System' (config file '/usr/local/nagios/etc/services/Gateway_AS400_System.cfg', starting on line 16)
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
        Checked 1228 services.
        Checked 174 hosts.
        Checked 18 host groups.
        Checked 0 service groups.
        Checked 13 contacts.
        Checked 7 contact groups.
        Checked 130 commands.
        Checked 26 time periods.
        Checked 336 host escalations.
        Checked 1768 service escalations.
Checking for circular paths...
        Checked 174 hosts
        Checked 0 service dependencies
        Checked 0 host dependencies
        Checked 26 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...
Stopping nagios:. done.
Starting nagios: done.

[avandemore]

Please do a:

Code: Select all

tail -F /usr/local/nagiosxi/var/cmdsubsys.log

Then run an Apply Configuration from the gui. Send any output.

Also please send the apache errot log, usually located at /var/log/httpd/error_log and also send /var/log/messages as well.
What is the output of:

Code: Select all

# getenforce

[akepley]

there is no cmdsubsys.log

Code: Select all

[email protected] nagiosxi# ls -rlt
total 32
drwxr-xr-x  2 nagios nagios 4096 Mar  6 09:43 cron
drwxr-xr-x  3 nagios nagios 4096 Mar  6 09:43 nom
drwxr-xr-x  2 nagios nagios 4096 Mar  6 09:43 tools
drwxr-xr-x  3 nagios nagios 4096 Mar  6 09:43 etc
drwsrwsr-x  2 apache nagios 4096 Mar  6 09:46 tmp
drwxr-xr-x  5 nagios nagios 4096 Mar  6 09:46 var
drwxr-xr-x 19 nagios nagios 4096 Mar  7 07:55 html
drwxr-xr-x  2 nagios nagios 4096 Mar 20 14:08 scripts
[email protected] nagiosxi# cd var/
[email protected] var# ls -rlt
total 28
-rw-r--r-- 1 nagios nagios  196 Mar  6 09:43 xiversion
drwxr-xr-x 2 nagios nagios 4096 Mar  6 09:43 upgrades
-rw-r--r-- 1 nagios nagios  950 Mar  6 09:43 xi-sys.cfg
-rw-r--r-- 1 nagios nagios   37 Mar  6 09:46 xi-uuid
-rw-r--r-- 1 nagios nagios    7 Mar  6 09:46 xi-itype
drwsrwsr-x 2 apache nagios 4096 Mar 17 14:56 components
drwxr-xr-x 2 nagios nagios 4096 Mar 20 14:08 subsys

Code: Select all

[email protected] var# getenforce
Disabled

the entirety of the ssl_error_log (the only error log under /var/log/httpd/ that has any logging today:

Code: Select all

[Mon Mar 20 12:51:37 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 13:08:56 2017] [error] [client 172.17.30.224] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/xicore/status-object-detail.inc.php on line 2507, referer: https://nagios01.ec2.prod.azw.nasba/nagiosxi/includes/components/xicore/status.php?&show=hosts&hoststatustypes=4&servicestatustypes=0
[Mon Mar 20 13:26:18 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 13:34:25 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 13:44:01 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 13:46:37 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 14:45:38 2017] [error] [client 172.17.30.224] File does not exist: /usr/local/nagiosxi/html/includes/js/jquery/jquery-1.11.2.min.js, referer: https://nagios01.ec2.prod.azw.nasba/
[Mon Mar 20 15:35:19 2017] [error] [client 172.17.30.224] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/xicore/status-object-detail.inc.php on line 1155, referer: https://nagios01.ec2.prod.azw.nasba/nagiosxi/includes/components/xicore/status.php?show=services&servicestatustypes=28
[Mon Mar 20 15:36:16 2017] [error] [client 172.17.30.224] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/xicore/status-object-detail.inc.php on line 1155, referer: https://nagios01.ec2.prod.azw.nasba/nagiosxi/includes/components/xicore/status.php?show=services&servicestatustypes=28
[Mon Mar 20 15:36:17 2017] [error] [client 172.17.30.224] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/nagiosxi/html/includes/components/xicore/status-object-detail.inc.php on line 1155, referer: https://nagios01.ec2.prod.azw.nasba/nagiosxi/includes/components/xicore/status.php?show=services&servicestatustypes=28

I'll PM the message log to you

[avandemore]

I got your message.

XI > Admin > System Profile > Download Profile

Please include the zip file in your response. You can PM myself or other support personnel if you have privacy concerns.

Also please include /etc/sudoers as well.