Page 1 of 1

SQL Error, repair script help only for a short time

Posted: Wed Apr 18, 2018 2:27 am
by snakekick
Hello,
i get in irregular intervals sql error like this
SQL Error [ndoutils] : Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed

after a repair with repair_databases.sh it´s work for a while but the error come back (no reboot, disk is free)
i hope someone can help me

Re: SQL Error, repair script help only for a short time

Posted: Wed Apr 18, 2018 1:08 pm
by cdienger
What does the typical load look like on the machine? Something likely performance related is causing the tables to become corrupted and https://assets.nagios.com/downloads/nag ... ios-XI.pdf can help tune performance. The RAM disk option it covers is a quick way to improve things.

Re: SQL Error, repair script help only for a short time

Posted: Thu Apr 19, 2018 1:18 am
by snakekick
now only 16 host with 112 services are monitored with a quadcore system with 4gb ram



80419 7:07:22 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 7:07:22 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 7:07:24 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 7:07:24 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 7:07:26 [Note] /usr/libexec/mysqld: Normal shutdown

180419 7:07:26 [Note] Event Scheduler: Purging the queue. 0 events
180419 7:07:26 InnoDB: Starting shutdown...
180419 7:07:30 InnoDB: Shutdown completed; log sequence number 1600492
180419 7:07:30 [Note] /usr/libexec/mysqld: Shutdown complete

180419 07:07:30 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended
180419 07:07:31 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
180419 7:07:31 [Note] /usr/libexec/mysqld (mysqld 5.5.56-MariaDB) starting as process 9297 ...
180419 7:07:31 InnoDB: The InnoDB memory heap is disabled
180419 7:07:31 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180419 7:07:31 InnoDB: Compressed tables use zlib 1.2.7
180419 7:07:31 InnoDB: Using Linux native AIO
180419 7:07:31 InnoDB: Initializing buffer pool, size = 128.0M
180419 7:07:31 InnoDB: Completed initialization of buffer pool
180419 7:07:31 InnoDB: highest supported file format is Barracuda.
180419 7:07:31 InnoDB: Waiting for the background threads to start
180419 7:07:32 Percona XtraDB (http://www.percona.com) 5.5.52-MariaDB-38.3 started; log sequence number 1600492
180419 7:07:32 [Note] Plugin 'FEEDBACK' is disabled.
180419 7:07:32 [Note] Server socket created on IP: '0.0.0.0'.
180419 7:07:32 [Note] Event Scheduler: Loaded 0 events
180419 7:07:32 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.56-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
180419 7:07:33 [Note] /usr/libexec/mysqld: Normal shutdown

180419 7:07:33 [Note] Event Scheduler: Purging the queue. 0 events
180419 7:07:33 InnoDB: Starting shutdown...
180419 7:07:36 InnoDB: Shutdown completed; log sequence number 1600492
180419 7:07:36 [Note] /usr/libexec/mysqld: Shutdown complete

180419 07:07:36 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended
180419 07:07:37 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
180419 7:07:37 [Note] /usr/libexec/mysqld (mysqld 5.5.56-MariaDB) starting as process 9657 ...
180419 7:07:37 InnoDB: The InnoDB memory heap is disabled
180419 7:07:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180419 7:07:37 InnoDB: Compressed tables use zlib 1.2.7
180419 7:07:37 InnoDB: Using Linux native AIO
180419 7:07:37 InnoDB: Initializing buffer pool, size = 128.0M
180419 7:07:37 InnoDB: Completed initialization of buffer pool
180419 7:07:37 InnoDB: highest supported file format is Barracuda.
180419 7:07:37 InnoDB: Waiting for the background threads to start
180419 7:07:38 Percona XtraDB (http://www.percona.com) 5.5.52-MariaDB-38.3 started; log sequence number 1600492
180419 7:07:38 [Note] Plugin 'FEEDBACK' is disabled.
180419 7:07:38 [Note] Server socket created on IP: '0.0.0.0'.
180419 7:07:38 [Note] Event Scheduler: Loaded 0 events
180419 7:07:38 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.56-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
180419 7:07:39 [Note] /usr/libexec/mysqld: Normal shutdown

180419 7:07:39 [Note] Event Scheduler: Purging the queue. 0 events
180419 7:07:39 InnoDB: Starting shutdown...
180419 7:07:42 InnoDB: Shutdown completed; log sequence number 1600492
180419 7:07:42 [Note] /usr/libexec/mysqld: Shutdown complete

180419 07:07:42 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended
180419 07:07:42 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
180419 7:07:42 [Note] /usr/libexec/mysqld (mysqld 5.5.56-MariaDB) starting as process 10036 ...
180419 7:07:42 InnoDB: The InnoDB memory heap is disabled
180419 7:07:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins
180419 7:07:42 InnoDB: Compressed tables use zlib 1.2.7
180419 7:07:42 InnoDB: Using Linux native AIO
180419 7:07:42 InnoDB: Initializing buffer pool, size = 128.0M
180419 7:07:42 InnoDB: Completed initialization of buffer pool
180419 7:07:42 InnoDB: highest supported file format is Barracuda.
180419 7:07:42 InnoDB: Waiting for the background threads to start
180419 7:07:43 Percona XtraDB (http://www.percona.com) 5.5.52-MariaDB-38.3 started; log sequence number 1600492
180419 7:07:43 [Note] Plugin 'FEEDBACK' is disabled.
180419 7:07:43 [Note] Server socket created on IP: '0.0.0.0'.
180419 7:07:43 [Note] Event Scheduler: Loaded 0 events
180419 7:07:43 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.56-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server
180419 8:00:02 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 8:00:02 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 8:00:02 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed
180419 8:00:02 [ERROR] mysqld: Table './nagios/nagios_servicestatus' is marked as crashed and last (automatic?) repair failed

Re: SQL Error, repair script help only for a short time

Posted: Thu Apr 19, 2018 5:36 am
by snakekick
next thing that i try was truncate the damaged table but this solves the problem only for minutes ;(

Re: SQL Error, repair script help only for a short time

Posted: Thu Apr 19, 2018 9:49 am
by scottwilkerson
There are only a few scenarios I've seen that cause this, 2 of which you ruled out in your first post.

The 3rd is hardware failure whereas the physical disk you are using could have bad block on it.

What is the possibility of taking a full backup and restoring on a new XI server with different physical hardware?

https://assets.nagios.com/downloads/nag ... ios-XI.pdf