Page 1 of 2
Issues after upgrading to 5.4.0
Posted: Wed Jan 04, 2017 4:26 pm
by kcarlson
Seeing multiple issues with NagiosXI gui after an upgrade to 5.4 on CentOS 7. Attached a screen grab of the database backend being in alarm. When I click the "start" action it will come back as successful but will never run. It appears that mariadb is running but postgresql is not. Also noticed when I check the status of ndo2db I receive
"ndo2db is not running but subsystem locked"
Thanks!
Re: Issues after upgrading to 5.4.0
Posted: Wed Jan 04, 2017 4:29 pm
by dwhitfield
All you may need to do is
rm -f /usr/local/nagios/var/ndo2db.lock.
If that doesn't fix it, in
/usr/local/nagios/etc/ndo2db.cfg change
debug_level=0 to a 1. Also, change
debug_verbosity=1 to 2. Save the file and run
service ndo2db start.
If it fails, please post the output of the following
Code: Select all
tail -25 /usr/local/nagios/var/ndo2db.debug
tail -25 /var/log/messages
tail -25 /usr/local/nagios/var/nagios.log
Once we have things resolved, do not forget to decrease the debug level in ndo2db.cfg back to defaults.
Re: Issues after upgrading to 5.4.0
Posted: Wed Jan 04, 2017 4:54 pm
by kcarlson
Removing ndo2db.lock did not resolve the issue. Changed the debug level and verbosity, saved, then tried to start the service. Once I start it comes back Ok, but when I check status I get the same subsystem locked message. Tail on the ndo2db.debug returns nothing. I have attached the results from messages and nagios.log.
Thanks!
Re: Issues after upgrading to 5.4.0
Posted: Wed Jan 04, 2017 5:17 pm
by dwhitfield
Did you go through the rest of the suggestions at
https://support.nagios.com/forum/viewto ... 0&start=20 ? If not, give those a shot. I still think it's good to separate the possible RHEL/Cent 6 versus RHEL/Cent 7 issues, but there might be some relevant stuff there too.
I was looking back at that thread to see if you had said from which version were you upgrading. If you were not on 5.3.4, could you roll back to a previously working state, and then upgrade to 5.3.4. Then, you can try the upgrade to 5.4.0. We tested older versions going back to 2014, but there are a lot of changes, so maybe smaller increments will work.
Re: Issues after upgrading to 5.4.0
Posted: Thu Jan 05, 2017 9:53 am
by kcarlson
Looked through the thread posted. I see differences in that their ndo2db as well as postgresql are running, mine are not. I went through /etc/init.d/ndo2db and see that mine has "Ndo2dbRunFile=$Ndo2dbVarDir/nod2db.pid" while your example shows "Ndo2dbRunFile=$Ndo2dbVarDir/ndo2db.lock".
Sorry for not clarifying earlier, I upgraded from 5.3.4 to 5.4.0
Thanks!
Re: Issues after upgrading to 5.4.0
Posted: Thu Jan 05, 2017 11:41 am
by dwhitfield
Where did your upgrade process start originally? If you started with something 5.x, then the postgres stuff is not of concern.
I don't mean this upgrade, I mean your first ever install. Thanks!
Re: Issues after upgrading to 5.4.0
Posted: Thu Jan 05, 2017 12:40 pm
by kcarlson
I believe we started on 5.3.0 or 5.3.1 then upgraded as new releases were available until 5.4.0.
Re: Issues after upgrading to 5.4.0
Posted: Thu Jan 05, 2017 4:49 pm
by avandemore
Please do this:
# tail -F /usr/local/nagiosxi/var/cmdsubsys.log
and then restart the process in the web gui. After a bit of time you some info this sudo /usr/local/nagiosxi/scripts/manage_services.sh restart ndo2db followed later by this tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated After you see that last one, you can ctrl-C out of the tail. Please post the full prompt, command, and output in a code block.
Re: Issues after upgrading to 5.4.0
Posted: Thu Jan 05, 2017 4:51 pm
by jomann
Can you do the following:
If there are any running, kill them, and then try to restart the ndo2db service.
Re: Issues after upgrading to 5.4.0
Posted: Fri Jan 06, 2017 11:03 am
by kcarlson
@avandemore Here is the output from the command you wanted me to run.
Code: Select all
[root@nagios ~]# tail -F /usr/local/nagiosxi/var/cmdsubsys.log
......
PROCESSED 0 COMMANDS
:"";}
CMDLINE=sudo /usr/local/nagiosxi/scripts/manage_services.sh start ndo2db
OUTPUT=
RETURNCODE=0
..................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
PROCESSED 1 COMMANDS
@jomann Here is the output from ps with the kill on that pid.
Code: Select all
[root@nagios ~]# ps aux | grep ndo2db
root 7928 0.0 0.0 112648 960 pts/0 S+ 09:46 0:00 grep --color=auto ndo2db
[root@nagios ~]# kill 7928
-bash: kill: (7928) - No such process