Upgrade marked as Failed

[bomahony]

Hey folks.

As part of our security mitigation for the shellinabox package, I removed this prior to upgrading from 5.5.8 -> 5.5.10. It removed another package called "nagiosxi-deps-el7". The yum erase worked fine. After this I was able to see the SSH web terminal was disabled [See SSH Disabled image]

I then upgraded via the web interface. This proceeded trough. It ended with the image below "Failed Upgrade". However:
- Big red box saying 'Upgrade failed'
- Multiple lines saying: 'No entry for terminal type "unknown";using dumb terminal settings.'
- Text box has 'Nagios XI Upgrade Complete!'

I rebooted the machine, then clicked back when I got back in. I see:

Code: Select all

Your Nagios XI installation is up to date.
Latest Available Version:	5.5.10
Installed Version:	5.5.10
Last Update Check:	2019-02-26 13:24:26

However, there is no Update from today in the Update History! [Actually I havent upgraded since Dec tbh when the import bug was fixed]. I have a VM snapshot from before the upgrade that I can roll back to If you believe I should.

If I look to re-install shellinabox, nagiosxi-deps-el7 doesnt appear.

[benjaminsmith]

Hi @bomahony,

Are you able to login to the XI interface? If so, what version does it say you are running in the lower left hand corner.

Please you post the contents of the upgrade log for us to review.

Code: Select all

/usr/local/nagiosxi/tmp/upgrade.log

We planning to be able top have the ability to disable the SSH terminal in the security settings in a future release. However, you can also disable the service.

Code: Select all

systemctl disable shellinaboxd.service

[bomahony]

Log file is empty [i clicked "Go Back"]
BUT i saved it in a doc. Will pm it to you.

I should have disabled it, sure. But our GIS team I would imagine would want it removed, if possible.

Is it possible to get a copy of the dependant rpm i can install and then re-install shellinabox with it?

[benjaminsmith]

Hi @bomahony,

Shelinabox is required and is installed from the EPEL repository. Try running the following to re-install it:

Code: Select all

yum install shellinabox

If that is successful run the upgrade script again:

Code: Select all

wget -O upgrade.sh https://assets.nagios.com/downloads/nagiosxi/upgrade.sh
./upgrade.sh

Let me know if you're able to get it upgraded. We're planning to make the SSH terminal feature an option from the security settings in Nagios 5.6.

Reference:
Nagios XI Upgrade Instructions

[bomahony]

Looks the same:
#1 shellinabox reinstalled.
#2 I could not find the "nagios-deps-el7" package anywhere so I ignored it.
#3 Had enable proxy etc for the manual upgrade.
#4 Ran the wget and then ran the script, output below.
#5 Still no update history in the web interface, but i assume that is expected.

Code: Select all

Proxy request sent, awaiting response... 200 OK
Length: 54206073 (52M) [application/x-gzip]
Saving to: ‘/tmp/xi-latest.tar.gz’

100%[=========================================================================>] 54,206,073  3.35MB/s   in 28s

2019-02-27 11:15:18 (1.87 MB/s) - ‘/tmp/xi-latest.tar.gz’ saved [54206073/54206073]

Running upgrade
Nagios XI Upgrade Log
==========================
DATE: Wed 27 Feb 11:15:20 GMT 2019

DISTRO INFO:
RedHatEnterpriseServer
7.4
x86_64


Archive:  sourceguardian/ixed4.lin.x86-64.zip
  inflating: /usr/lib64/php/modules/ixed.5.4.lin
Sourceguardian extension found for PHP version 5.4
Sourceguardian extension already in php.ini
Checking required prereqs...
Please wait...

OLD VERSION: 5510
Loaded plugins: langpacks, product-id, rhnplugin, search-disabled-repos,
              : subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
No packages marked for update
Loaded plugins: langpacks, product-id, rhnplugin, search-disabled-repos,
              : subscription-manager
This system is receiving updates from RHN Classic or Red Hat Satellite.
Package perl-Test-Simple-0.98-243.el7.noarch already installed and latest version
Package perl-Class-Accessor-0.34-12.el7.noarch already installed and latest version
Package perl-Params-Validate-1.08-4.el7.x86_64 already installed and latest version
Package perl-Config-Tiny-2.14-7.el7.noarch already installed and latest version
Package perl-Math-Calc-Units-1.07-9.el7.noarch already installed and latest version
Package perl-Number-Format-1.73-14.el7.noarch already installed and latest version
Nothing to do
no crontab for nagios
no crontab for root
Copying over new XI directory...
Patching NDOUtils...
NDOUtils already patched at level 103
Installing new PNP templates...
Enabling large install tweaks...
Fixing config file permissions...
chown: cannot access ‘/var/www/html/nagiosql/config’: No such file or directory
Fixing htpasswd permissions...
Checking group memberships...
Installing new XI templates...

--- reset_config_perms.sh ------------
> Setting CCM script permissions
> Setting script permissions
> Setting special component script permissions
> Setting configuration file/directory permissions
> Setting perfdata directory and RRD permissions
> Setting NOM checkpoint user:group permissions
> + Setting CCM configuration file user:group permissions
> + Setting Recurring Downtime file user:group permissions
> + Setting BPI configuration file user:group permissions
--------------------------------------

--- ccm_import.php -------------------
> Setting import directory: /usr/local/nagios/etc/import/
> Importing config files into the CCM
  - Importing: /usr/local/nagios/etc/import/xi_timeperiod_24x7.cfg .. SUCCESS
--------------------------------------

--- ccm_export.php -------------------
> Writing CCM configuration to Nagios files
  Finished writing out configuraton
--------------------------------------

--------------------------------------
> Verifying configuration with Nagios Core
> Output:
Nagios Core 4.4.3
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 2019-01-15
License: GPL

Website: https://www.nagios.org
Reading configuration data...
   Read main config file okay...
   Read object config files okay...

Running pre-flight check on configuration data...

Checking objects...
        Checked 2762 services.
        Checked 84 hosts.
        Checked 32 host groups.
        Checked 6 service groups.
        Checked 34 contacts.
        Checked 3 contact groups.
        Checked 192 commands.
        Checked 38 time periods.
        Checked 0 host escalations.
        Checked 0 service escalations.
Checking for circular paths...
        Checked 84 hosts
        Checked 0 service dependencies
        Checked 0 host dependencies
        Checked 38 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check
> Return Code: 0
--------------------------------------

Nagios XI Upgrade Complete!
---------------------------

[bomahony]

I have actually decided to revert to yesterdays VM snapshot to avoid all the messing

I reverted the snapshot to prior to removing the packages. The SSH window still works.
I still get:
"The update did not complete successfully. Please try updating manually."
It is also still missing the "Update History" for the current work.
However it states I am running 5.5.10

I have tried this multiple time with:
1. shellinabox enabled and disabled
2. With nagios repo and proxy in yum config, and without [this was never needed before]

[bomahony]

I have actually decided to revert to yesterdays VM snapshot to avoid all the messing

I reverted the snapshot to prior to removing the packages. The SSH window still works.
I still get:
"The update did not complete successfully. Please try updating manually."
It is also still missing the "Update History" for the current work.
However it states I am running 5.5.10

I have tried this multiple time with:
1. shellinabox enabled and disabled
2. With nagios repo and proxy in yum config, and without [this was never needed before]

[benjaminsmith]

Hello @bomahony,

I have actually decided to revert to yesterdays VM snapshot to avoid all the messing

One more thing is the proxy, this can cause issues with the installation. Did you follow the documentation for using proxies with Nagios XI?
Nagios XI - Problems Using Nagios XI With Proxies

Also, please post the output of your software repositories:

Code: Select all

yum repolist -v 

Finally, if you try to run the upgrade script again, please capture the stdout and stderr to a file for us to review.

Code: Select all

sh ./upgrade.sh &> bomahony.txt

Thanks.

[bomahony]

I was on leave last week and didnt get to revert to this. FYI I have upgraded all four of our nodes, via the UI, to each point release since July last year, and not once had an issue with the proxy server, which is set in the admin section of the application.

Just to point out the upgrade was working from the CLI, after upgrading via the web interface, and this is still the case after updating yum and wgetrc configs. Unfortunately, I cannot leave the wgetrc config in place, as any user will be able to use the proxy then, which will not be allowed.

I cannot revert to the old VM snapshot as we will loose too much history.
I am instead going to move to a different node, and upgrade that one. My plan is:
#1 Take a VM snapshot.
#2 Upgrade via web interface. If this works, put down the first one as an anomaly.
#3 If this fails like previous, revert snapshot.
#4 Update the yum and wgetrc configs, and try the upgrade via the CLI.

Is there anything else I should be trying?

[bomahony]

So I tried another node that was at 5.5.8, upgrading to 5.5.11 released last week.
[Please not I havent disabled shellinabox for now, to ensure there was no issue with that]

#1 Via GUI: Got "Upgrade Complete" in the text box. However above this I get the "Update did not complete successfully, please try manually". However without doing anything else, when i refresh the page it is showing "Up to date." and version 5.5.11 says it is installed. Once again, there is no update history from today.
#2 Added entries to yum.conf and wgetrc. Same as above. [I have kept a log of this, if you want it]

Code: Select all

Things look okay - No serious problems were detected during the pre-flight check
> Return Code: 0
--------------------------------------
No entry for terminal type "unknown";
using dumb terminal settings.

Nagios XI Upgrade Complete!
---------------------------

You can access the Nagios XI web interface by visiting:

My query here is if he upgrade is successful, and it shows it is successful, why is it saying "The update did not complete successfully. Please try updating manually." in the WebUI?

I am going to try manually now, but I would imagine that will upgrade sucessfully as well.....