Page 1 of 1

mysql tables marked as crashed

Posted: Wed Feb 22, 2012 11:47 am
by TSCAdmin
We are receiving errors in /var/log/messages every minute:

Eg.
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='15296''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='15285''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND event_type='0' AND scheduled_time=FROM_UNIXTIME(1329925410) AND recurring_event='0' AND object_id='1019''
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedeventqueue WHERE instance_id='1' AND scheduled_time<FROM_UNIXTIME(1329925410)'
Feb 22 10:43:30 us-ham-svb-0224 ndo2db: mysql_error: 'Table './nagios/nagios_timedeventqueue' is marked as crashed and should be repaired'

Have you seen this before? Is there a way I can resolve this?

Re: mysql tables marked as crashed

Posted: Wed Feb 22, 2012 12:08 pm
by scottwilkerson
Yep we need to fix that table...

Please run

Code: Select all

# /usr/local/nagiosxi/scripts/repairmysql.sh nagios
If you continue to have problems with the table lets run

Code: Select all

mysql -u root -pnagiosxi nagios
REPAIR TABLE nagios_timedeventqueue USE_FRM;

Re: mysql tables marked as crashed

Posted: Wed Mar 07, 2012 1:21 am
by TSCAdmin
Hello Scott,

We were able to fix the problem running mysql table repair commands found on open source information.

Unfortunately we could not locate the script repairmysql.sh in /usr/local/nagiosxi/scripts/ or on whole /usr/local/nagios* directories. We are using Nagios XI 2009R1.3 if that makes any difference. Should I be looking somewhere else?

Could you please also help us finding the root cause analysis for this problem so we have a formal internal documentation to understand and fix this problem?

Many thanks

Re: mysql tables marked as crashed

Posted: Wed Mar 07, 2012 9:47 am
by scottwilkerson
I'm glad you got the tables fixed.

With that old of a version of XI it is possible you don't have the repairmysql.sh script.

As for the root cause, it could be many, but usually tables get marked as crashed when mysql goes down unexpectedly, such as a hard reboot or power failure. But there are other possibilities.