Page 1 of 2
nagios db issue
Posted: Tue Apr 29, 2014 9:39 am
by ravish78
Hi team,
We are migrating our nagios xi environment to new OS.When we did restore it happened successfully but it is showing db error in GUI.
Attached screenshot for reference.We have production live this weekend.So any help would be appreciated.
Also ndo is showing as locked
/etc/init.d/ndo2db status
ndo2db is not running but subsystem locked
Thanks
Ravi
Re: nagios db issue
Posted: Tue Apr 29, 2014 9:42 am
by slansing
That is showing that it is not running, it is not precisely an error.... Click the cog wheel and then click start. Or issue the following from an ssh session:
Re: nagios db issue
Posted: Tue Apr 29, 2014 9:53 am
by ravish78
We tried that but it still remains same.
Restarted mysql but still no luck.We suspect it is ndo issue.
i could not see ndo.sock in /usr/local/nagios/var
Re: nagios db issue
Posted: Tue Apr 29, 2014 10:10 am
by slansing
What was the exact output you got when you ran:
On the command line? You may have a stuck mysql.sock file, in which case make sure mysqld is stopped (it should be) and run:
Re: nagios db issue
Posted: Tue Apr 29, 2014 10:20 am
by ravish78
I tried steps as suggested by you.
mysql in running state and found mysql.sock
stopped mysqld,could not find mysql.sock after stopping.
Tried removing command.
started mysql
Attached complete output.
Re: nagios db issue
Posted: Tue Apr 29, 2014 2:01 pm
by slansing
Are you running an offloaded database? If so, just as the document says you don't need it running locally. If you are running a local database, what does the red icon on the System Status page say when you mouse over it?
Re: nagios db issue
Posted: Tue Apr 29, 2014 2:24 pm
by ravish78
It is local database and when mouse over it is showing as database backend not running.
Re: nagios db issue
Posted: Tue Apr 29, 2014 2:50 pm
by sreinhardt
OK, let's get some basics out of the way then.
Code: Select all
ps -ef | grep bin/nag
ps -ef | grep bin/ndo
ps -ef | grep mysql
service ndo2db status
grep 'output=' /usr/local/nagios/etc/ndomod.cfg
grep 'socket_name' /usr/local/nagios/etc/ndo2db.cfg
grep 'db_' /usr/local/nagios/etc/ndo2db.cfg | grep -v 'user\|pass'
Re: nagios db issue
Posted: Tue Apr 29, 2014 3:32 pm
by ravish78
Attached output.
Can we have online session tommorow around 10:00am CST to fix this.
We have production luanch this weekend and we need to fix this ASAP.
Re: nagios db issue
Posted: Tue Apr 29, 2014 4:39 pm
by abrist
Lets check your mysqld logs for crashed tables and then restart ndo:
Code: Select all
tail -50 /var/log/mysqld.log | grep "crashed"
If you have any crashed tables, run the repair script:
Code: Select all
/usr/local/nagiosxi/scripts/repairmysql.sh nagios
/usr/local/nagiosxi/scripts/repairmysql.sh nagiosql
Then restart ndo2db:
Code: Select all
service ndo2db restart
service ndo2db status
service mysqld status