Page 1 of 2
Upgrade from 5.4.0 to 5.4.4 failure
Posted: Mon May 15, 2017 12:40 pm
by gwakem
Happy Monday!
Currently on 5.4.0 and trying to upgrade to 5.4.4 via the shell, but I'm getting consistent failures.
Here is the output of the log at the time of failure:
Code: Select all
make[1]: Leaving directory `/var/tmp/nagiosxi/subcomponents/nagioscore/nagios-4.2.4'
Warning: nagios.service changed on disk. Run 'systemctl daemon-reload' to reload units.
no crontab for nagios
Saving old crontab to /tmp/root.crontab.orig.5...
Removing XI cron jobs...
Copying over new XI directory...
Updating NagiosQL...
NAGIOSQL-POST
Patching NDOUtils...
NDOUtils already patched at level 103
Installing new PNP templates...
Updating init script...
Enabling large install tweaks...
Fixing config file permissions...
Fixing htpasswd permissions...
Checking group memberships...
Installing new XI templates...
URL: https://localhost/nagiosxi/includes/components/ccm/
CMDLINE
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies https://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&backend=1&username=nagiosxi&password=xxxxxxxx' -O nagiosql.login--2017-05-15 13:19:17-- https://localhost/nagiosxi/includes/components/ccm/
Resolving localhost (localhost)... 127.0.0.1, ::1
Connecting to localhost (localhost)|127.0.0.1|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2017-05-15 13:19:17 ERROR 500: Internal Server Error.
NAGIOSQL LOGIN FAILED!
This system forces SSL logins. I have attempted the upgrade with SSL redirection rules disabled, and the CCM Admin > Settings > Paths > Server Protocol set to HTTP instead of HTTPS, but it always attempts HTTPS regardless. After this point, apache only displays a blank page and I can't do anything but restore from backup.
The "/usr/bin/wget --save-cookies nagiosql.cookies" that fails during the upgrade process works properly before the upgrade with no errors.
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Mon May 15, 2017 1:26 pm
by tgriep
Is the server setup use the LDAP / Active Directory integration component?
Try this KB article and see if it fixes the error when you try the upgrade.
https://support.nagios.com/kb/article/n ... login.html
Can you look in the Apache Error logs in the /var/log/httpd folder when the upgrade was done, search them for any errors and post them here so we can review them?
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Mon May 15, 2017 2:51 pm
by gwakem
This does use AD Integration, but the only thing I see in /usr/local/nagiosxi/html/includes/components/ is ldap_ad_integration. I attempted removing that after the upgrade failed (and then again before an upgrade attempt,) but that didn't fix the 500 errors in the logs and white screen. As far as logs, I removed all apache logs, restarted apache, ran the upgrade, and reviewed the logs, but there is nothing that stands out. It simply goes from showing 200's to 500's. I gzipped the logs in case you want to review them.
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Mon May 15, 2017 4:05 pm
by tgriep
Yes, we would need to see the log files and also your System Profile from the server as well.
Can you PM them to me?
To send us your system profile. Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and PM it back to me.
Before doing the upgrade, can you go in to the Core Config Manager and Apply the Config without any errors?
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 8:24 am
by gwakem
The profile has been PM'd. and yes, I can apply with no issues.
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 8:58 am
by tgriep
Thanks for the profile, I didn't find any settings or configurations that could cause the upgrade to fail.
If you still have the Apache Log files as well as the upgrade.log file for when you ran the upgrade, PM them to me so I can check them for errors.
The only thing you should do is to allocate more memory to the system, it is very tight and it looks like the system swapped in the past which slows the server down.
Try increasing the memory to 8 Gig.
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 10:50 am
by gwakem
I increased the memory, although the fact that I have pretty much only been non stop testing the upgrade from different angles may have contributed to that.
Also, I sent you the upgrade log and apache logs via PM.
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 11:38 am
by tgriep
I received the file and shared it.
I saw one permission error in the logs so can you run the following commands as root and post the output?
Code: Select all
ls -l /usr/local/nagiosxi/html/includes/
ls -l /usr/local/nagiosxi/html/includes/phpmailer
Can you attach the following files here so we can check the servers ssl settings?
Code: Select all
/etc/hosts
/etc/httpd/conf/httpd.conf
/etc/httpd/conf.d/nagiosxi.conf
/etc/httpd/conf.d/ssl.conf
Thanks
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 12:00 pm
by gwakem
ls -l /usr/local/nagiosxi/html/includes/
Code: Select all
total 2776
-rwxr-x--- 1 nagios nagios 5980 Jan 11 09:35 auth.inc.php
-rwxr-x--- 1 nagios nagios 807 Jan 11 09:35 common.inc.php
drwxr-x--- 63 nagios nagios 4096 Apr 11 15:17 components
-rwxr-x--- 1 nagios nagios 1060 Jan 11 09:35 components.inc.php
drwxr-x--- 65 nagios nagios 4096 Mar 17 2016 configwizards
-rwxr-x--- 1 nagios nagios 619 Jan 11 09:35 configwizards.inc.php
-rwxr-x--- 1 nagios nagios 21886 Jan 11 09:35 constants.inc.php
-rwxr-x--- 1 nagios nagios 24962 Jan 11 09:35 constants-nagioscore.inc.php
drwxr-x--- 3 nagios nagios 4096 Nov 30 09:57 css
drwxr-x--- 15 nagios nagios 4096 Apr 5 2016 dashlets
-rwxr-x--- 1 nagios nagios 680 Jan 11 09:35 dashlets.inc.php
-rwxr-x--- 1 nagios nagios 245009 Jan 11 09:35 dbauth.inc.php
-rwxr-x--- 1 nagios nagios 16251 Jan 11 09:35 db.inc.php
-rwxr-x--- 1 nagios nagios 104785 Jan 11 09:35 dbl.inc.php
-rwxr-x--- 1 nagios nagios 763 Jan 11 09:35 errors.inc.php
drwxr-x--- 2 nagios nagios 4096 Feb 18 2016 fonts
-rwxr-x--- 1 nagios nagios 56 Jan 11 09:35 footer-child.inc.php
-rwxr-x--- 1 nagios nagios 7349 Jan 11 09:35 footer.inc.php
drwxr-x--- 5 nagios nagios 112 Feb 18 2016 fpdf
-rwxr-x--- 1 nagios nagios 564 Jan 11 09:35 header-child.inc.php
-rwxr-x--- 1 nagios nagios 20209 Jan 11 09:35 header.inc.php
drwxr-x--- 6 nagios nagios 4096 Nov 30 09:57 js
drwxr-x--- 3 nagios nagios 64 Feb 18 2016 lang
-rwxr-x--- 1 nagios nagios 163 Jan 11 09:35 notificationmethods.inc.php
-rw-r----- 1 nagios nagios 17692 Jan 11 09:35 page-default-splash.php
-rwxr-x--- 1 nagios nagios 2428 Jan 11 09:35 page-home-main.php
-rwxr-x--- 1 nagios nagios 789 Jan 11 09:35 page-home.php
-rwxr-x--- 1 nagios nagios 835 Jan 11 09:35 page-missing.php
-rwxr-x--- 1 nagios nagios 30657 Jan 11 09:35 pageparts.inc.php
drwxr-x--- 2 nagios nagios 74 Feb 18 2016 phpmailer
-rwxr-x--- 1 nagios nagios 4054 Jan 11 09:35 utils-auditlog.inc.php
-rwxr-x--- 1 nagios nagios 62681 Jan 11 09:35 utils-backend.inc.php
-rwxr-x--- 1 nagios nagios 13188 Jan 11 09:35 utils-ccm.inc.php
-rwxr-x--- 1 nagios nagios 31557 Jan 11 09:35 utils-commands.inc.php
-rwxr-x--- 1 nagios nagios 3797 Jan 11 09:35 utils-components.inc.php
-rwxr-x--- 1 nagios nagios 12064 Jan 11 09:35 utils-configwizards.inc.php
-rwxr-x--- 1 nagios nagios 133425 Jan 11 09:35 utils-dashboards.inc.php
-rwxr-x--- 1 nagios nagios 110053 Jan 11 09:35 utils-dashlets.inc.php
-rwxr-x--- 1 nagios nagios 6502 Jan 11 09:35 utils-email.inc.php
-rwxr-x--- 1 nagios nagios 21745 Jan 11 09:35 utils-events.inc.php
-rwxr-x--- 1 nagios nagios 2036 Jan 11 09:35 utils-graphtemplates.inc.php
-rwxr-x--- 1 nagios nagios 38305 Jan 11 09:35 utils.inc.php
-rwxr-x--- 1 nagios nagios 822297 Jan 11 09:35 utilsl.inc.php
-rwxr-x--- 1 nagios nagios 9328 Jan 11 09:35 utils-links.inc.php
-rwxr-x--- 1 nagios nagios 75902 Jan 11 09:35 utils-menu.inc.php
-rwxr-x--- 1 nagios nagios 27445 Jan 11 09:35 utils-metrics.inc.php
-rwxr-x--- 1 nagios nagios 1998 Jan 11 09:35 utils-mibs.inc.php
-rwxr-x--- 1 nagios nagios 246321 Jan 11 09:35 utils-nagioscore.inc.php
-rwxr-x--- 1 nagios nagios 2080 Jan 11 09:35 utils-notificationmethods.inc.php
-rwxr-x--- 1 nagios nagios 115801 Jan 11 09:35 utils-notifications.inc.php
-rwxr-x--- 1 nagios nagios 38660 Jan 11 09:35 utils-objects.inc.php
-rwxr-x--- 1 nagios nagios 2442 Jan 11 09:35 utils-perms.inc.php
-rw-r----- 1 nagios nagios 4103 Jan 11 09:35 utils-reports-export.inc.php
-rwxr-x--- 1 nagios nagios 45445 Jan 11 09:35 utils-reports.inc.php
-rw-r----- 1 nagios nagios 6099 Jan 11 09:35 utils-rrdexport.inc.php
-rw-r----- 1 nagios nagios 1392 Jan 11 09:35 utils-rss.inc.php
-rwxr-x--- 1 nagios nagios 5473 Jan 11 09:35 utils-status.inc.php
-rwxr-x--- 1 nagios nagios 2379 Jan 11 09:35 utils-systat.inc.php
-rwxr-x--- 1 nagios nagios 14349 Jan 11 09:35 utils-tables.inc.php
-rwxr-x--- 1 nagios nagios 726 Jan 11 09:35 utils-themes.inc.php
-rw-r----- 1 nagios nagios 9530 Jan 11 09:35 utils-time.inc.php
-rw-r----- 1 nagios nagios 7751 Jun 21 2016 utils-timezones.inc.php
-rwxr-x--- 1 nagios nagios 4299 Jan 11 09:35 utils-tools.inc.php
-rwxr-x--- 1 nagios nagios 3738 Jan 11 09:35 utils-updatecheck.inc.php
-rwxr-x--- 1 nagios nagios 31615 Jan 11 09:35 utils-users.inc.php
-rwxr-x--- 1 nagios nagios 38085 Jan 11 09:35 utils-views.inc.php
-rwxr-x--- 1 nagios nagios 6705 Jan 11 09:35 utils-wizards.inc.php
-rwxr-x--- 1 nagios nagios 18071 Jan 11 09:35 utilsx.inc.php
-rwxr-x--- 1 nagios nagios 34326 Jan 11 09:35 utilsx-json.php
-rwxr-x--- 1 nagios nagios 4364 Jan 11 09:35 utils-xmlauditlog.inc.php
-rwxr-x--- 1 nagios nagios 37057 Jan 11 09:35 utils-xmlobjects.inc.php
-rwxr-x--- 1 nagios nagios 33960 Jan 11 09:35 utils-xmlreports.inc.php
-rwxr-x--- 1 nagios nagios 62752 Jan 11 09:35 utils-xmlstatus.inc.php
-rwxr-x--- 1 nagios nagios 7753 Jan 11 09:35 utils-xmlsysstat.inc.php
-rwxr-x--- 1 nagios nagios 1947 Jan 11 09:35 utils-xmlusers.inc.php
ls -l /usr/local/nagiosxi/html/includes/phpmailer
Code: Select all
total 116
-rwxr-x--- 1 nagios nagios 74783 Jan 11 09:35 class.phpmailer.php
-rwxr-x--- 1 nagios nagios 11833 Jan 11 09:35 class.pop3.php
-rwxr-x--- 1 nagios nagios 28024 Jan 11 09:35 class.smtp.php
Hosts:
Code: Select all
127.0.0.1 localhost.localdomain localhost
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.20.12.198 xi-test.ingles-markets.com xi-test
Re: Upgrade from 5.4.0 to 5.4.4 failure
Posted: Tue May 16, 2017 4:20 pm
by tgriep
Those config files look like they are configured correctly and the permission error I saw looks to be a false alarm.
Can you login to the XI GUI and go to the Admin > System Config > System Settings menu and verify that the Program URL is setup to use https and is using the FQDN of the server
Next navigate to Configure > Core Config Manager > CCM Admin > Settings menu and click on the Paths tab, verify that the server protocol is https.
Go to the Admin > Reset Security Credentials menu, Click on the Update Credentials to reset passwords Nagiosxi uses for the backend, etc.
Then login to the XI server, run the following and post the output.
Code: Select all
cd /usr/local/nagiosxi/scripts
su nagios
./reconfigure_nagios.sh
I took another look at the Apache log files you sent earlier and I do not see the log change from 200 to 500 errors so they didn't help much.