Nagios XI GUI Apply Configuration Fails after Upgrade to 5.2

[mp4783]

I upgraded my Nagios XI 5.1.0 installation to 5.2.0. After the upgrade, I could not apply changes via the Nagios XI GUI. I get the following message:

Apply Configuration

Reset config permissions failed.

An error occurred while attempting to apply your configuration to Nagios Core. Monitoring engine configuration files have been rolled back to their last known good checkpoint.

Even if I have no changes to apply and then click "Apply Configuration", it fails in the GUI. Additionally, after the failure, the /usr/local/nagiosxi/scripts/reconfigure_nagios.lock is left in place and must be manually removed.

However, if I go to the command line and execute (as the nagios user) the following, it works fine:

Code: Select all

cd /usr/local/nagiosxi/scripts && ./reconfigure_nagios.sh

• - I have checked the sudo privileges for the nagios user and the user can run "sudo /usr/local/nagiosxi/scripts/reset_config_perms.sh" without any problem.
  - I did notice that the compiled version of reset_config_perms was not present, which may be normal for the 5.X release.
  - I am forcing installation of MySQL 5.6.10 because of other requirements we have.
  - I have confirmed that the "nagiosxi", "ndoutils", and "nagiosql" users can all log into MySQL using the password configured.
  - There are no errors in the nagios.log file.
  - The Apache log files show nothing suspicious.

Following is the upgrade log wherein you will note a single failed login, but successful completion of the upgrade. I also noticed that the upgrade process had to sleep quite a long time waiting for another configuration process to finish.

upgrade.log contents:

Code: Select all

Checking required prereqs...
Please wait...

OLD VERSION: 510
Saving old crontab to /tmp/root.crontab.orig...
Removing XI cron jobs...
Archive:  sourceguardian/ixed4.lin.x86-64.zip
  inflating: /usr/lib64/php/modules/ixed.5.3.lin
Sourceguardian extension found for PHP version 5.3
Sourceguardian extension already in php.ini
Copying over new XI directory...
Updating NagiosQL...
NAGIOSQL-POST
Patching NDOUtils...
NDOUtils already patched at level 103
Installing new PNP templates...
Updating init script...
Enabling large install tweaks...
Fixing config file permissions...
Fixing htpasswd permissions...
Checking group memberships...
Installing new XI templates...
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O nagiosql.loginNAGIOSQL LOGIN FAILED!
Checking required prereqs...
Please wait...

OLD VERSION: 511
Saving old crontab to /tmp/root.crontab.orig.1...
Removing XI cron jobs...
Archive:  sourceguardian/ixed4.lin.x86-64.zip
  inflating: /usr/lib64/php/modules/ixed.5.3.lin
Sourceguardian extension found for PHP version 5.3
Sourceguardian extension already in php.ini
Copying over new XI directory...
Updating NagiosQL...
NAGIOSQL-POST
Patching NDOUtils...
NDOUtils already patched at level 103
Installing new PNP templates...
Updating init script...
Enabling large install tweaks...
Fixing config file permissions...
Fixing htpasswd permissions...
Checking group memberships...
Installing new XI templates...
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O nagiosql.loginLOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
    [0] => xi_timeperiod_24x7.cfg
)
IMPORTING /usr/local/nagios/etc/import/xi_timeperiod_24x7.cfg
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=admin&type=import&importsubmitted=true&chbOverwrite=1&subForm=Import&selImportFile[]=/usr/local/nagios/etc/import/xi_timeperiod_24x7.cfg' -O nagiosql.import.monitoring
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
RESETTING PERMS
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O nagiosql.loginLOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
WRITE CONFIGS SUCCESSFUL!
OUTPUT:
Nagios Core 4.1.1
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-19-2015
License: GPL

Website: https://www.nagios.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
        Checked 14 services.
        Checked 1 hosts.
        Checked 1 host groups.
        Checked 0 service groups.
        Checked 3 contacts.
        Checked 2 contact groups.
        Checked 125 commands.
        Checked 9 time periods.
        Checked 0 host escalations.
        Checked 0 service escalations.
Checking for circular paths...
        Checked 1 hosts
        Checked 0 service dependencies
        Checked 0 host dependencies
        Checked 9 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...
Stopping nagios: done.
Starting nagios: done.
Fixing php-mcrypt bug...
Starting httpd:
==================
UPGRADE COMPLETED!
==================


You can access the Nagios XI web interface by visiting:
    http://localhost/nagiosxi/

[lmiltchev]

Can you run "reconfigure_nagios.sh" successfully as "nagios"?

Code: Select all

su nagios
cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh

[mp4783]

Yes, I thought I stated that. It runs just fine at the command line, but will not from the GUI. Typically this is a result of permissions issue (e.g. user apache permissions versus user nagios permissions). I confirmed that this was not the case by running the reconfigure_nagios.sh command as the apache user. Neither user has permission issue executing the reconfigure_nagios.sh command at the command line.

What is frustrating is that there is no easy way to see what is happening when the process is invoked from the GUI without an elaborate setup that traces processes.

I spent a considerable amount of time trying to determine if reset_config_perms.sh was the culprit, even to the point of adding an additional sudoers entry that permitted user apache to run it via sudo.

No luck.

If no one else is reporting any issues, then maybe I just need to dump the entire installation and start over (not what I want, but I'd rather this actually work).

[jdalrymple]

there is no easy way to see what is happening when the process is invoked from the GUI without an elaborate setup that traces processes.

Are we excluding tail -f /usr/local/nagiosxi/var/cmdsubsys.log?

I think it gives the exact same (level of) output as running from the CLI.

[mp4783]

I had a nice long response prepared with all sorts of "clues" as to what might be happening. However, the real clue was in the cmdsubsys.log file.

Code: Select all

sudo: sorry, you must have a tty to run sudo

Our sudo configuration imposes "requiretty". Once I inserted the following two lines, the configuration applied correctly:

Defaults:nagios !requiretty
Defaults:apache !requiretty

The audit.log also showed a failure message which "sort of" should have pointed me in the right direction. I didn't consider the fact that the Nagios XI process would require a TTY.

Thanks for helping me fix this and you may lock the thread.

PS: You might consider checking for this in your installation or upgrade scripts.

[mp4783]

The "Defaults:apache !requiretty" entry in the sudoers file is not necessary. Sorry, I should have checked.

[jdalrymple]

PS: You might consider checking for this in your installation or upgrade scripts.

We set it in install (obviously), maybe it's not part of upgrade?

But then why did it work with 5.1 and not the 5.2 upgrade. Did you fullinstall 5.1 after which your guy's config manager came through and blew it away?