Issues after upgrading to 5.4.0

[kcarlson]

Seeing multiple issues with NagiosXI gui after an upgrade to 5.4 on CentOS 7. Attached a screen grab of the database backend being in alarm. When I click the "start" action it will come back as successful but will never run. It appears that mariadb is running but postgresql is not. Also noticed when I check the status of ndo2db I receive
"ndo2db is not running but subsystem locked"

Thanks!

[dwhitfield]

All you may need to do is rm -f /usr/local/nagios/var/ndo2db.lock.

If that doesn't fix it, in /usr/local/nagios/etc/ndo2db.cfg change debug_level=0 to a 1. Also, change debug_verbosity=1 to 2. Save the file and run service ndo2db start.

If it fails, please post the output of the following

Code: Select all

tail -25 /usr/local/nagios/var/ndo2db.debug
tail -25 /var/log/messages
tail -25 /usr/local/nagios/var/nagios.log

Once we have things resolved, do not forget to decrease the debug level in ndo2db.cfg back to defaults.

[kcarlson]

Removing ndo2db.lock did not resolve the issue. Changed the debug level and verbosity, saved, then tried to start the service. Once I start it comes back Ok, but when I check status I get the same subsystem locked message. Tail on the ndo2db.debug returns nothing. I have attached the results from messages and nagios.log.
Thanks!

[dwhitfield]

Did you go through the rest of the suggestions at https://support.nagios.com/forum/viewto ... 0&start=20 ? If not, give those a shot. I still think it's good to separate the possible RHEL/Cent 6 versus RHEL/Cent 7 issues, but there might be some relevant stuff there too.

I was looking back at that thread to see if you had said from which version were you upgrading. If you were not on 5.3.4, could you roll back to a previously working state, and then upgrade to 5.3.4. Then, you can try the upgrade to 5.4.0. We tested older versions going back to 2014, but there are a lot of changes, so maybe smaller increments will work.

[kcarlson]

Looked through the thread posted. I see differences in that their ndo2db as well as postgresql are running, mine are not. I went through /etc/init.d/ndo2db and see that mine has "Ndo2dbRunFile=$Ndo2dbVarDir/nod2db.pid" while your example shows "Ndo2dbRunFile=$Ndo2dbVarDir/ndo2db.lock".

Sorry for not clarifying earlier, I upgraded from 5.3.4 to 5.4.0

Thanks!

[dwhitfield]

Where did your upgrade process start originally? If you started with something 5.x, then the postgres stuff is not of concern.

I don't mean this upgrade, I mean your first ever install. Thanks!

[kcarlson]

I believe we started on 5.3.0 or 5.3.1 then upgraded as new releases were available until 5.4.0.

[avandemore]

Please do this:
# tail -F /usr/local/nagiosxi/var/cmdsubsys.log

and then restart the process in the web gui. After a bit of time you some info this sudo /usr/local/nagiosxi/scripts/manage_services.sh restart ndo2db followed later by this tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated After you see that last one, you can ctrl-C out of the tail. Please post the full prompt, command, and output in a code block.

[jomann]

Can you do the following:

Code: Select all

ps aux | grep ndo2db

If there are any running, kill them, and then try to restart the ndo2db service.

[kcarlson]

@avandemore Here is the output from the command you wanted me to run.

Code: Select all

[root@nagios ~]# tail -F /usr/local/nagiosxi/var/cmdsubsys.log
......
PROCESSED 0 COMMANDS
:"";}
CMDLINE=sudo /usr/local/nagiosxi/scripts/manage_services.sh start ndo2db
OUTPUT=
RETURNCODE=0
..................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated

PROCESSED 1 COMMANDS

@jomann Here is the output from ps with the kill on that pid.

Code: Select all

[root@nagios ~]# ps aux | grep ndo2db
root      7928  0.0  0.0 112648   960 pts/0    S+   09:46   0:00 grep --color=auto ndo2db
[root@nagios ~]# kill 7928
-bash: kill: (7928) - No such process