Nagios XI - Apply Configuration Never Completes

[MOHANREDDY]

Hello,
I am attempting to Apply Configuration and i'm seeing the following output:

* Configuration submitted for processing...
* Waiting for configuration verification..................

The configuration never applies, the page may be timing out.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
When tailing the log /usr/local/nagiosxi/var/cmdsubsys.log

/usr/local/nagiosxi/scripts/export_nagiosql.sh: line 7: 6658 Killed sudo $BASEDIR/reset_config_perms.sh
RESETTING CONFIG PERMS FAILED!\n
OUTPUT=RESETTING CONFIG PERMS FAILED!\n
RETURNCODE=4

PROCESSED 1 COMMANDS
.
PROCESSED 0 COMMANDS
...........................................................
PROCESSED 0 COMMANDS

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

when i run ./reconfigure_nagios.sh

URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=S6lY6NVWpF9T' -O nagiosql.login--2018-04-17 11:31:13-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.login'

[ <=> ] 34,272 --.-K/s in 0.01s

2018-04-17 11:31:14 (3.22 MB/s) - 'nagiosql.login' saved [34272]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
)
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...

[scottwilkerson]

Can you show the output of the following

Code: Select all

grep NAGIOS /etc/sudoers

[MOHANREDDY]

grep NAGIOS /etc/sudoers
User_Alias NAGIOSXI=nagios
User_Alias NAGIOSXIWEB=apache
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios status
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios checkconfig
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/ndo2db status
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd status
NAGIOSXI ALL = NOPASSWD:/usr/bin/php /usr/local/nagiosxi/html/includes/components/autodiscovery/scripts/autodiscover_new.php *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/html/includes/components/profile/getprofile.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/upgrade_to_latest.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/change_timezone.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/reset_config_perms.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/backup_xi.sh *
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/messages
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/httpd/error_log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/mysqld.log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/php /usr/local/nagiosxi/html/includes/components/autodiscovery/scripts/autodiscover_new.php *
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/html/includes/components/profile/getprofile.sh
NAGIOSXIWEB ALL = NOPASSWD:/etc/init.d/snmptt restart
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/repair_databases.sh
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh *

[scottwilkerson]

Lets run this

Code: Select all

cd/usr/local/nagiosxi/scripts/
rm -f reconfigure_nagios.lock
./reconfigure_nagios.sh

Then also send back the results of the following

Code: Select all

chage -l nagios

[MOHANREDDY]

Lets run this

Code: Select all

cd/usr/local/nagiosxi/scripts/
rm -f reconfigure_nagios.lock
./reconfigure_nagios.sh

Then also send back the results of the following

Code: Select all

chage -l nagios

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
./reconfigure_nagios.sh
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=S6lY6NVWpF9T' -O nagiosql.login--2018-04-17 12:09:46-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.login'

[ <=> ] 34,272 --.-K/s in 0.01s

2018-04-17 12:09:47 (3.21 MB/s) - 'nagiosql.login' saved [34272]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
)
RESETTING PERMS
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=S6lY6NVWpF9T' -O nagiosql.login--2018-04-17 12:09:47-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.login'

[ <=> ] 34,272 --.-K/s in 0.01s

2018-04-17 12:09:47 (3.34 MB/s) - 'nagiosql.login' saved [34272]

LOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'backend=1&cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2018-04-17 12:09:47-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.export.monitoring'

[ <=> ] 22,008 --.-K/s in 0.008s

2018-04-17 12:09:48 (2.77 MB/s) - 'nagiosql.export.monitoring' saved [22008]

WRITE CONFIGS SUCCESSFUL!
OUTPUT:
Nagios Core 4.2.4
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 12-07-2016
License: GPL

Website: https://www.nagios.org
Reading configuration data...
Read main config file okay...
Error: Invalid max_attempts, check_interval, retry_interval, or notification_interval value for service 'Local Disk Usage' on host 'localhost'
Error: Could not register service (config file '/usr/local/nagios/etc/services/server.cfg', starting on line 63)
Error processing object config files!


***> One or more problems was encountered while processing the config files...

Check your configuration file(s) to ensure that they contain valid
directives and data defintions. If you are upgrading from a previous
version of Nagios, you should be aware that some variables/definitions
may have been removed or modified in this version. Make sure to read
the HTML documentation regarding the config files, as well as the
'Whats New' section to find out what has changed.
RET: 1
/usr/local/nagiosxi/nom/checkpoints/nagioscore/errors /usr/local/nagiosxi/scripts
tar: Removing leading `/' from member names
/usr/local/nagiosxi/scripts
LATEST NOM SNAPSHOT: /usr/local/nagiosxi/nom/checkpoints/nagioscore/1523978282.tar.gz
/ /usr/local/nagiosxi/scripts
RESTORING NOM SNAPSHOT : /usr/local/nagiosxi/nom/checkpoints/nagioscore/1523978282.tar.gz
/usr/local/nagiosxi/scripts
RESETTING PERMS
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
chage -l nagios

Last password change : Apr 13, 2018
Password expires : Jul 12, 2018
Password inactive : never
Account expires : never
Minimum number of days between password change : 7
Maximum number of days between password change : 90
Number of days of warning before password expires : 7

[MOHANREDDY]

Now i am seeing database errors
more mariadb.log
180413 11:12:38 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
180413 11:12:38 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 5397 ...
180413 11:12:38 InnoDB: The InnoDB memory heap is disabled
180413 11:12:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180413 11:12:38 InnoDB: Compressed tables use zlib 1.2.7
180413 11:12:38 InnoDB: Using Linux native AIO
180413 11:12:38 InnoDB: Initializing buffer pool, size = 128.0M
180413 11:12:38 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
180413 11:12:38 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
180413 11:12:38 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
180413 11:12:38 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
180413 11:12:38 InnoDB: Waiting for the background threads to start
180413 11:12:39 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 0
180413 11:12:39 [Note] Plugin 'FEEDBACK' is disabled.
180413 11:12:39 [Note] Server socket created on IP: '0.0.0.0'.
180413 11:12:39 [Note] Event Scheduler: Loaded 0 events
180413 11:12:39 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
180413 11:13:35 [Note] /usr/libexec/mysqld: Normal shutdown

180413 11:13:35 [Note] Event Scheduler: Purging the queue. 0 events
180413 11:13:36 InnoDB: Starting shutdown...
180413 11:13:36 InnoDB: Shutdown completed; log sequence number 1601170
180413 11:13:36 [Note] /usr/libexec/mysqld: Shutdown complete

180413 11:13:36 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended
180413 11:13:37 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
180413 11:13:37 [Note] /usr/libexec/mysqld (mysqld 5.5.52-MariaDB) starting as process 8950 ...
180413 11:13:37 InnoDB: The InnoDB memory heap is disabled
180413 11:13:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180413 11:13:37 InnoDB: Compressed tables use zlib 1.2.7
180413 11:13:37 InnoDB: Using Linux native AIO
180413 11:13:37 InnoDB: Initializing buffer pool, size = 128.0M
180413 11:13:37 InnoDB: Completed initialization of buffer pool
180413 11:13:37 InnoDB: highest supported file format is Barracuda.
180413 11:13:37 InnoDB: Waiting for the background threads to start
180413 11:13:38 Percona XtraDB (http://www.percona.com) 5.5.49-MariaDB-38.0 started; log sequence number 1601170
180413 11:13:38 [Note] Plugin 'FEEDBACK' is disabled.
180413 11:13:38 [Note] Server socket created on IP: '0.0.0.0'.
180413 11:13:38 [Note] Event Scheduler: Loaded 0 events
180413 11:13:38 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.52-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server

[scottwilkerson]

Those don't look related and are from 4 days ago

However there is an error in a config
Go to CCM -> Services
Config Template select <config form error>
click the disk icon next to any of the services

When you get the output, the service definition that begins on line 63 has an issue.

This may be the same service but also 'Local Disk Usage' on localhost has Invalid max_attempts, check_interval, retry_interval, or notification_interval

these will need to be addressed in the CCM and then the rerun

Code: Select all

./reconfigure_nagios.sh

[MOHANREDDY]

Those don't look related and are from 4 days ago

However there is an error in a config
Go to CCM -> Services
Config Template select <config form error>
click the disk icon next to any of the services

When you get the output, the service definition that begins on line 63 has an issue.

This may be the same service but also 'Local Disk Usage' on localhost has Invalid max_attempts, check_interval, retry_interval, or notification_interval

these will need to be addressed in the CCM and then the rerun

Code: Select all

./reconfigure_nagios.sh

I have corrected those errors but still the is still loading on for a long time and passing

Apply Configuration
Waiting for configuration verification......................................................................................................
Apply Configuration
Reset config permissions failed.
An error occurred while attempting to apply your configuration to Nagios Core. Monitoring engine configuration files have been rolled back to their last known good checkpoint.
Try Again View Snapshot
Show Written Configs
Error: Contact 'user' has no service notification commands defined!


I corrected the error and ran from ./reconfigure.sh from server and i didn't see any error

URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=S6lY6NVWpF9T' -O nagiosql.login--2018-04-17 16:06:27-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.login'

[ <=> ] 38,275 --.-K/s in 0.01s

2018-04-17 16:06:27 (3.27 MB/s) - 'nagiosql.login' saved [38275]

LOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
)
RESETTING PERMS
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=S6lY6NVWpF9T' -O nagiosql.login--2018-04-17 16:06:27-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.login'

[ <=> ] 38,275 --.-K/s in 0.01s

2018-04-17 16:06:28 (3.49 MB/s) - 'nagiosql.login' saved [38275]

LOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'backend=1&cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2018-04-17 16:06:28-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'nagiosql.export.monitoring'

[ <=> ] 21,806 --.-K/s in 0.008s

2018-04-17 16:06:28 (2.68 MB/s) - 'nagiosql.export.monitoring' saved [21806]

WRITE CONFIGS SUCCESSFUL!
OUTPUT:
Nagios Core 4.2.4
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 12-07-2016
License: GPL

Website: https://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
Checked 20 services.
Checked 2 hosts.
Checked 1 host groups.
Checked 0 service groups.
Checked 2 contacts.
Checked 2 contact groups.
Checked 124 commands.
Checked 8 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 2 hosts
Checked 0 service dependencies
Checked 0 host dependencies
Checked 8 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check
RET: 0


But i click apply configuration from GUI its throwing the same error as above

Apply Configuration
Waiting for configuration verification......................................................................................................
Apply Configuration
Reset config permissions failed.
An error occurred while attempting to apply your configuration to Nagios Core. Monitoring engine configuration files have been rolled back to their last known good checkpoint.
Try Again View Snapshot
Show Written Configs
Error: Contact 'user' has no service notification commands defined!

[scottwilkerson]

Please try running the following and report the results

Code: Select all

sudo su nagios -c '/usr/local/nagiosxi/scripts/reset_config_perms.sh'
echo $?

[MOHANREDDY]

it changed the permissions of all the directories you specified and the output for echo $? is "0".