nagios db crashed tables =(

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

nagios db crashed tables =(

Post by benhank »

hey guys
imm getting this error on my production server:

Code: Select all

] /usr/libexec/mysqld: Table './nagios/nagios_statehistory' is marked as crashed and last (automatic?) repair failed
because its production im hesitant to do any repairs.
one thing I did do was a manual restore from the daily backups, but i think the daily was messed up as well. can we log a ticket and have someone look at it pls?
I normally would not come out the gate with this request but its production.
thanks
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: nagios db crashed tables =(

Post by ssax »

You will need to repair the DB, please run these commands:

Code: Select all

service nagios stop
service ndo2db stop
service mysqld restart
cd /usr/local/nagiosxi/scripts
./repair_databases.sh
This will automatically start the services back up so go ahead and test after that.
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: nagios db crashed tables =(

Post by ssax »

If your DB is offloaded you will need to run the repair on the mysql server.
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: nagios db crashed tables =(

Post by benhank »

rgr rgr
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
rkennedy
Posts: 6579
Joined: Mon Oct 05, 2015 11:45 am

Re: nagios db crashed tables =(

Post by rkennedy »

@benhank - report back after repairing. Over. Kssssht.
Former Nagios Employee
User avatar
lmiltchev
Bugs find me
Posts: 13589
Joined: Mon May 23, 2011 12:15 pm

Re: nagios db crashed tables =(

Post by lmiltchev »

You can also try running the following command on the remote (offloaded) box:

Code: Select all

mysqlcheck -r -f -u root -pnagiosxi --databases nagios nagiosql nagiosxi
Let us know if this fixed your issue.
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: nagios db crashed tables =(

Post by benhank »

before I run said commands, any chance that my production machine will go down? I don't have a good backup because of this issue.
I forgot to mention, this issue is on a 2014 version of xi.
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: nagios db crashed tables =(

Post by ssax »

I've never seen an issue caused from running a repair. That being said, it will go down when you stop the nagios/ndo2db/mysqld services.

Is it a VM, can you take a snapshot?
User avatar
benhank
Posts: 1264
Joined: Tue Apr 12, 2011 12:29 pm

Re: nagios db crashed tables =(

Post by benhank »

i should not have doubted the team...
That did it running the repair script fied the problem. Thanks team!
Proudly running:
NagiosXI 5.4.12 2 node Prod Env 2500 hosts, 13,000 services
Nagiosxi 5.5.7(test env) 2500 hosts, 13,000 services
Nagios Logserver 2 node Prod Env 500 objects sending
Nagios Network Analyser
Nagios Fusion
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: nagios db crashed tables =(

Post by mcapra »

Is it alright if we lock this thread and mark the issue as resolved?
Former Nagios employee
https://www.mcapra.com/
Locked