Upgrading Nagios XI 5.2.5 to 5.6.7

[moirey]

Hello,

We are trying to upgrade our Nagios XI server from version 5.2.5 to 5.6.7. I wanted to know if there are any important details I should know before making the jump to 5.6.7, ex. Should I be upgrading to another version as middlestep before going to 5.6.7? Any configurations I should keep note of? Should i also be upgrading the nagios client on our host before doing so?

We attempted to upgrade earlier this year by cloning the current nagios xi server vm. Then, upgrading the cloned server to 5.6.7. When the upgrade was complete, we changed the ip of the cloned server to that of the original vm after powering down the original.

After the upgrade, all of our host were returning UNKNOWN status when running checks and we had to rollback to the old 5.2.5 server.

Any help would be greatly appreciated!

Thank you.

[ssax]

First, I'd go to XI 5.6.8. You can go straight from 5.2.5 to XI latest (or 5.6.7 if you really need to go to that specific version).

https://www.nagios.com/downloads/nagios ... -releases/

Do you have an RPM install?

Code: Select all

rpm -qa | grep nagiosxi

If so, you'll likely need to do this method of upgrading:

https://repo.nagios.com/

See this one if you have an offline system:

https://repo.nagios.com/?repo=offline

My general protocol for upgrades that makes me pretty successful:

1. Make sure you have enough free space:

Code: Select all

df -h
df -i

2. Make sure your configs verify (no errors is what we want, don't care about warnings)

Code: Select all

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

3. Make sure you don't have any crashed tables:

First, run it to see the sizes (to make sure it outputs valid data):

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -uroot -pnagiosxi --table

Then you can run it like this to check for crashed tables:

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -uroot -pnagiosxi --table | grep NULL

If no NULL are found, no crashed tables. (NULL means it was unable to calculate that tables size so it means the table is crashed)

4. Backup XI/VM Snap (just in case):

https://assets.nagios.com/downloads/nag ... ios-XI.pdf

5. Reboot server, wait 15 minutes, this will ensure everything is fresh and up and running.

6. Follow the manual upgrade process from the CLI, if it fails, send us the upgrade.log file from the installation directory (generally /tmp/nagiosxi/upgrade.log) so that we can help you work through any issues.

https://assets.nagios.com/downloads/nag ... ctions.pdf

Let us know if you have any questions or if I can clarify anything.

Thank you!

[moirey]

Hi ssax,

Thank you very much for the info. I was able to successfully upgrade to 5.6.8. Everything seems to be working fine for the most part. Only issue I'm having post upgrade is hosts are returning (No output on stdout) stderr: for the check_http command.

[lmiltchev]

Can you show us the actual check that you are running from the command line along with the output of it? Add "-v" at the end of your command for increased verbosity - it may help us troubleshoot the issue. You are using check_http version 2.2.1, correct? What is the output of the following command?

Code: Select all

/usr/local/nagios/libexec/check_http -V

[moirey]

Hi lmiltchev,

Output for:

Code: Select all

/usr/local/nagios/libexec/check_http -V

check_http v2.2.1 (nagios-plugins 2.2.1)

Command I am running is :

Code: Select all

$USER1$/check_http -H $HOSTADDRESS$ -p 8080 -u '/StoreFront/ImageCompositionServlet?files=jsp/builderimages/BaseFiles/TR1300.png,jsp/builderimages/Figurine/FIGHOST.png,,&width=378&height=378&trim=true' -w 2.5 -c 5

Output for this when ran:

Code: Select all

GET /StoreFront/ImageCompositionServlet?files=jsp/builderimages/BaseFiles/TR1300.png,jsp/builderimages/Figurine/FIGHOST.png,,&width=378&height=378&trim=true HTTP/1.1
User-Agent: check_http/v2.2.1 (nagios-plugins 2.2.1)
Connection: close
Host: 172.XXX.XXX.XXX:8080
Accept: */*


http://172.XX.XXX.XXX:8080/StoreFront/ImageCompositionServlet?files=jsp/builderimages/BaseFiles/TR1300.png,jsp/builderimages/Figurine/FIGHOST.png,,&width=378&height=378&trim=true is 8360 characters
STATUS: HTTP/1.1 200 OK
**** HEADER ****
Server: Apache-Coyote/1.1
Content-Type: image/jpeg
Transfer-Encoding: chunked
Date: Fri, 22 Nov 2019 19:29:51 GMT
Connection: close
**** CONTENT ****
\
Segmentation fault

[swolf]

Hi @moirey,

As a stopgap measure, you can re-install the previous version of nagios-plugins that came with 5.2.5. If I'm doing my math right, this should be v2.1.1. See here for how to manually re-install an older version of the plugins.

As far as actually resolving the issue, is this a public-facing host that we'd be able to run against (and if so, can we have the IP/hostname)? If not, we have a couple of things that we can try, but the process will become more involved on your end.

(If you can supply the hostname, feel free to skip the next step.)

The next thing I'd like to try is compiling the plugins from the latest pre-release. There were quite a few changes introduced for the check_http plugin, so there's a good chance this will fix your issues. You can use the same guide as was mentioned in the first paragraph, but make sure you're using the "maint" branch of our github repository, rather than the release listed in the article.

If this still produces the same error, let us know in the thread and we'll figure out next steps.