MySQL Fails to repair

[isadmin]

The Mysql DB is taking excessive amounts of disk space and keeps crashing the box...
the error during repair is
recovering (with sort) MyISAM-table 'nagios_logentries.MYI'
Data records: 1077266
- Fixing index 1
- Fixing index 2
- Fixing index 3
/usr/local/nagiosxi/scripts/repairmysql.sh: line 51: 13932 Segmentation fault (core dumped) $cmd $t
Starting mysqld: [ OK ]

has anyone seen this? Is there a fix?

[abrist]

I have never seen a seg fault on mysqld, though anything is technically "possible", though not always "probable".
What version of mysqld are you running?

Code: Select all

yum list installed | grep mysql

[isadmin]

yum list installed | grep mysql
mysql.x86_64 5.1.69-1.el6_4 @updates
mysql-devel.x86_64 5.1.69-1.el6_4 @updates
mysql-libs.x86_64 5.1.69-1.el6_4 @updates
mysql-server.x86_64 5.1.69-1.el6_4 @updates
php-mysql.x86_64 5.3.3-23.el6_4 @updates

[abrist]

1. What shell are you using?

Code: Select all

env | grep SHELL

2. Did you run the script as root?
3. Try updating mysql:

Code: Select all

yum update mysql

[isadmin]

Bash
I ran the script as root
here is the update results...seems all up to date

yum update mysql
Loaded plugins: fastestmirror, security
Determining fastest mirrors
epel/metalink | 14 kB 00:00
* base: mirrors.usinternet.com
* epel: mirror.nexcess.net
* extras: mirror.oss.ou.edu
* rpmforge: mirror.team-cymru.org
* updates: mirror.serversurgeon.com
base | 3.7 kB 00:00
cr | 2.9 kB 00:00
epel | 4.2 kB 00:00
epel/primary_db | 5.6 MB 00:06
extras | 3.4 kB 00:00
rpmforge | 1.9 kB 00:00
rpmforge/primary_db | 2.6 MB 00:03
updates | 3.4 kB 00:00
updates/primary_db | 4.5 MB 00:03
Setting up Update Process
No Packages marked for Update

[isadmin]

updated to newest nagios xi version 2012R2.5 with no help

[slansing]

Can you run the following and show the output?:

Code: Select all

ls -lS /var/lib/mysql/nagios/ | grep MYD

[isadmin]

ls -lS /var/lib/mysql/nagios/ | grep MYD
-rw-rw---- 1 mysql mysql 1798567200 Oct 11 10:35 nagios_logentries.MYD
-rw-rw---- 1 mysql mysql 75002896 Oct 15 09:51 nagios_statehistory.MYD
-rw-rw---- 1 mysql mysql 2540840 Oct 15 09:50 nagios_notifications.MYD
-rw-rw---- 1 mysql mysql 1363116 Oct 15 09:50 nagios_commenthistory.MYD
-rw-rw---- 1 mysql mysql 693088 Oct 15 09:50 nagios_contactnotificationmethods.MYD
-rw-rw---- 1 mysql mysql 614328 Oct 15 09:50 nagios_contactnotifications.MYD
-rw-rw---- 1 mysql mysql 320552 Oct 15 09:51 nagios_servicestatus.MYD
-rw-rw---- 1 mysql mysql 232072 Oct 15 09:51 nagios_hoststatus.MYD
-rw-rw---- 1 mysql mysql 218660 Oct 15 08:45 nagios_processevents.MYD
-rw-rw---- 1 mysql mysql 178508 Oct 15 07:40 nagios_services.MYD
-rw-rw---- 1 mysql mysql 141852 Oct 15 06:48 nagios_objects.MYD
-rw-rw---- 1 mysql mysql 127530 Oct 15 09:35 nagios_flappinghistory.MYD
-rw-rw---- 1 mysql mysql 122700 Oct 15 06:55 nagios_hosts.MYD
-rw-rw---- 1 mysql mysql 87484 Oct 15 09:18 nagios_acknowledgements.MYD
-rw-rw---- 1 mysql mysql 77828 Oct 15 09:51 nagios_conninfo.MYD
-rw-rw---- 1 mysql mysql 65476 Oct 15 06:55 nagios_downtimehistory.MYD
-rw-rw---- 1 mysql mysql 39765 Oct 15 06:48 nagios_service_contactgroups.MYD
-rw-rw---- 1 mysql mysql 27180 Oct 15 06:48 nagios_service_contacts.MYD
-rw-rw---- 1 mysql mysql 16492 Oct 15 06:55 nagios_hostdependencies.MYD
-rw-rw---- 1 mysql mysql 16424 Oct 15 06:55 nagios_commands.MYD
-rw-rw---- 1 mysql mysql 15988 Oct 15 09:51 nagios_customvariablestatus.MYD
-rw-rw---- 1 mysql mysql 15988 Oct 15 09:51 nagios_systemcommands.MYD
-rw-rw---- 1 mysql mysql 13484 Oct 15 06:55 nagios_customvariables.MYD
-rw-rw---- 1 mysql mysql 12840 Oct 15 06:55 nagios_host_contactgroups.MYD
-rw-rw---- 1 mysql mysql 11445 Oct 15 06:55 nagios_hostgroup_members.MYD
-rw-rw---- 1 mysql mysql 10815 Oct 15 06:55 nagios_host_parenthosts.MYD
-rw-rw---- 1 mysql mysql 7644 Oct 15 09:50 nagios_comments.MYD
-rw-rw---- 1 mysql mysql 7280 Oct 15 06:55 nagios_configfilevariables.MYD
-rw-rw---- 1 mysql mysql 5808 Oct 15 06:55 nagios_contact_notificationcommands.MYD
-rw-rw---- 1 mysql mysql 4011 Oct 15 06:48 nagios_timeperiod_timeranges.MYD
-rw-rw---- 1 mysql mysql 2835 Oct 15 06:55 nagios_host_contacts.MYD
-rw-rw---- 1 mysql mysql 2452 Oct 15 09:50 nagios_externalcommands.MYD
-rw-rw---- 1 mysql mysql 1784 Oct 15 06:55 nagios_contacts.MYD
-rw-rw---- 1 mysql mysql 1180 Oct 15 07:40 nagios_timeperiods.MYD
-rw-rw---- 1 mysql mysql 1122 Oct 15 06:55 nagios_contactstatus.MYD
-rw-rw---- 1 mysql mysql 960 Oct 15 06:55 nagios_contactgroup_members.MYD
-rw-rw---- 1 mysql mysql 808 Oct 15 07:40 nagios_runtimevariables.MYD
-rw-rw---- 1 mysql mysql 552 Oct 15 06:55 nagios_contactgroups.MYD
-rw-rw---- 1 mysql mysql 508 Oct 15 06:55 nagios_hostgroups.MYD
-rw-rw---- 1 mysql mysql 360 Oct 15 06:48 nagios_servicegroup_members.MYD
-rw-rw---- 1 mysql mysql 316 Oct 15 09:50 nagios_eventhandlers.MYD
-rw-rw---- 1 mysql mysql 120 Oct 15 09:51 nagios_programstatus.MYD
-rw-rw---- 1 mysql mysql 44 Oct 15 06:55 nagios_configfiles.MYD
-rw-rw---- 1 mysql mysql 40 Oct 15 06:48 nagios_servicegroups.MYD
-rw-rw---- 1 mysql mysql 20 Oct 15 06:55 nagios_dbversion.MYD
-rw-rw---- 1 mysql mysql 20 Oct 15 06:55 nagios_instances.MYD
-rw-rw---- 1 mysql mysql 0 Oct 15 06:55 nagios_contact_addresses.MYD
-rw-rw---- 1 mysql mysql 0 Oct 15 06:55 nagios_hostchecks.MYD
-rw-rw---- 1 mysql mysql 0 Oct 15 06:55 nagios_hostescalation_contactgroups.MYD
-rw-rw---- 1 mysql mysql 0 Oct 15 06:55 nagios_hostescalation_contacts.MYD
-rw-rw---- 1 mysql mysql 0 Oct 15 06:55 nagios_hostescalations.MYD
-rw-rw---- 1 mysql mysql 0 Oct 3 13:00 nagios_scheduleddowntime.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_servicechecks.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_servicedependencies.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_serviceescalation_contactgroups.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_serviceescalation_contacts.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_serviceescalations.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_timedeventqueue.MYD
-rw-rw---- 1 mysql mysql 0 Sep 6 10:22 nagios_timedevents.MYD

[abrist]

Try repairing the db once again, and then, once it segfaults, get a tail of the mysql log and post it here:

Code: Select all

tail -25 /var/log/mysql.log

[isadmin]

- recovering (with sort) MyISAM-table 'nagios_logentries.MYI'
Data records: 1077266
- Fixing index 1
- Fixing index 2
- Fixing index 3
/usr/local/nagiosxi/scripts/repairmysql.sh: line 51: 12271 Segmentation fault (core dumped) $cmd $t
Starting mysqld: [ OK ]
/tmp/nagiosxi

===============
REPAIR COMPLETE
===============
You have new mail in /var/spool/mail/root
[root@machine1 ]# tail -25 /var/log/mysql.log
tail: cannot open `/var/log/mysql.log' for reading: No such file or directory
[root@machine 1]# tail -25 /var/log/mysqld.log
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:19 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
131016 7:19:41 [Note] /usr/libexec/mysqld: Normal shutdown

131016 7:19:41 [Note] Event Scheduler: Purging the queue. 0 events
131016 7:19:43 InnoDB: Starting shutdown...
131016 7:19:45 InnoDB: Shutdown completed; log sequence number 0 44233
131016 7:19:45 [Note] /usr/libexec/mysqld: Shutdown complete

131016 07:19:45 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
131016 07:24:34 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
131016 7:24:34 InnoDB: Initializing buffer pool, size = 8.0M
131016 7:24:34 InnoDB: Completed initialization of buffer pool
131016 7:24:35 InnoDB: Started; log sequence number 0 44233
131016 7:24:35 [Note] Event Scheduler: Loaded 0 events
131016 7:24:35 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.69' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution