Monitoring Engine not running after attempted upgrade.

[sawdusty]

I attempted an offline upgrade from 5.11.3 to 2024R1.1.2-1 and after running the upgrade script everything appears to have installed correctly. When I bring up the url for our nagiosxi, the monitoring engine service is showing red and will not start. I've tried to restart the service via the gui as well as cli. When I run 'systemctl status nagios' it's showing failed. When I look at my system profile, it still shows I'm running version 5.11.3. For history I built this server from the 5.10.0-64 ova file downloaded from the nagios site and completed an upgrade to 5.11.3 in November.


× nagios.service - Nagios Core 4.4.13
Loaded: loaded (/usr/lib/systemd/system/nagios.service; enabled; preset: disabled)
Active: failed (Result: exit-code) since Wed 2024-05-08 18:27:47 UTC; 313ms ago
Duration: 4ms
Docs: https://www.nagios.org/documentation
Process: 12697 ExecStartPre=/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg (code=exited, status=0/SUCCESS)
Process: 12698 ExecStart=/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg (code=exited, status=0/SUCCESS)
Process: 12708 ExecStopPost=/usr/bin/rm -f /usr/local/nagios/var/rw/nagios.cmd (code=exited, status=0/SUCCESS)
Main PID: 12699 (code=exited, status=1/FAILURE)
CPU: 18ms

May 08 18:27:45 localhost.localdomain nagios[12699]: wproc: Successfully registered manager as @wproc with query handler
May 08 18:27:45 localhost.localdomain nagios[12699]: wproc: Registry request: name=Core Worker 12700;pid=12700
May 08 18:27:45 localhost.localdomain nagios[12699]: wproc: Registry request: name=Core Worker 12702;pid=12702
May 08 18:27:45 localhost.localdomain nagios[12699]: wproc: Registry request: name=Core Worker 12703;pid=12703
May 08 18:27:45 localhost.localdomain nagios[12699]: wproc: Registry request: name=Core Worker 12701;pid=12701
May 08 18:27:45 localhost.localdomain nagios[12699]: Error: Could not load module '/usr/local/nagios/bin/ndo.so' -> libmysqlclient.so.21: cannot open shared object file: No such file or directory
May 08 18:27:45 localhost.localdomain nagios[12699]: Error: Failed to load module '/usr/local/nagios/bin/ndo.so'.
May 08 18:27:45 localhost.localdomain nagios[12699]: Error: Module loading failed. Aborting.
May 08 18:27:45 localhost.localdomain systemd[1]: nagios.service: Main process exited, code=exited, status=1/FAILURE
May 08 18:27:47 localhost.localdomain systemd[1]: nagios.service: Failed with result 'exit-code'.
~

[danderson]

Thanks for reaching out @sawdusty,

Is there anything interesting in /tmp/xi-upgrade.log? I believe that's where the RPM upgrades put their logs.

[sawdusty]

I get this....

[root@localhost ~]# cat /tmp/xi-install.log
MySQL not yet installed - that's okay.
cp: cannot stat '/etc/my.cnf.d/mysql-server.cnf': No such file or directory

[swolf]

Hi @sawdusty - thanks for reaching out.

I don't recommend mixing and matching installers/upgrade methods. If you used the OVA to do the original installation, I recommend that you:

1) Revert your server to the previous version,
2) Get the source distribution for upgrade,
3) take the system online briefly,
4) use `./upgrade` as you would for the offline distribution

The offline RPM is only supported from a system that originally used the offline RPM to install.

If you want to try to patch this system as-is, I would check for the missing library. My CentOS stream 9 system has this output:

Code: Select all

[root@localhost ~]# locate libmysqlclient
/usr/lib64/mysql/libmysqlclient.so
/usr/lib64/mysql/libmysqlclient.so.21
/usr/lib64/mysql/libmysqlclient.so.21.2.32

If libmysqlclient.so is missing, here are the mysql-related packages I have installed:

Code: Select all

[root@localhost ~]# yum list installed | grep mysql
mysql.x86_64                                     8.0.32-1.el9                      @appstream       
mysql-common.x86_64                              8.0.32-1.el9                      @appstream       
mysql-devel.x86_64                               8.0.32-1.el9                      @crb             
mysql-errmsg.x86_64                              8.0.32-1.el9                      @appstream       
mysql-libs.x86_64                                8.0.32-1.el9                      @crb             
mysql-selinux.noarch                             1.0.5-1.el9                       @appstream       
mysql-server.x86_64                              8.0.32-1.el9                      @appstream       
php-mysqlnd.x86_64                               8.0.30-1.el9                      @appstream       

My guess is that you don't have access to the CodeReady Builder repository - you'll either need an offline clone of that repository, or you'll need to configure access via the internet. For an online system, you can do this via:

Code: Select all

dnf config-manager --set-enabled crb

Hopefully all of that makes sense - please let me know if anything needs clarification or if you have further questions.
-Sebastian

[sawdusty]

I rebuilt our Nagios server with the ova and successfully performed an online update. When I attempt to load my license onto the server I get an error stating that "Max activations exceeded" How do I rectify this? The other VM's I tried rebuilds with have all been terminated/deleted. Once I repopulate all the devices into Nagios, I will terminate the original box that has the monitoring engine that will not start.

[swolf]

Hi @sawdusty,

Reach out to csm@nagios.com with the last 6 characters of your license key, an explanation of what's going on, and a link to this thread. They should be able to increase the number of activations.

-Sebastian