mysql tables marked as crashed

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
TSCAdmin
Posts: 155
Joined: Wed Apr 14, 2010 3:00 pm
Location: India

mysql tables marked as crashed

Post by TSCAdmin »

We are receiving errors in /var/log/messages every minute:

Eg.
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='15296''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='15285''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='1019''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'

Have you seen this before? Is there a way I can resolve this?
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: mysql tables marked as crashed

Post by scottwilkerson »

Yep we need to fix that table...

Please run

Code: Select all

# /usr/local/nagiosxi/scripts/repairmysql.sh nagios
If you continue to have problems with the table lets run

Code: Select all

mysql -u root -pnagiosxi nagios
REPAIR TABLE nagios_timedeventqueue USE_FRM;
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
TSCAdmin
Posts: 155
Joined: Wed Apr 14, 2010 3:00 pm
Location: India

Re: mysql tables marked as crashed

Post by TSCAdmin »

Hello Scott,

We were able to fix the problem running mysql table repair commands found on open source information.

Unfortunately we could not locate the script repairmysql.sh in /usr/local/nagiosxi/scripts/ or on whole /usr/local/nagios* directories. We are using Nagios XI 2009R1.3 if that makes any difference. Should I be looking somewhere else?

Could you please also help us finding the root cause analysis for this problem so we have a formal internal documentation to understand and fix this problem?

Many thanks
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: mysql tables marked as crashed

Post by scottwilkerson »

I'm glad you got the tables fixed.

With that old of a version of XI it is possible you don't have the repairmysql.sh script.

As for the root cause, it could be many, but usually tables get marked as crashed when mysql goes down unexpectedly, such as a hard reboot or power failure. But there are other possibilities.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked