Crash kernel (LVM_LV=vg_00/lv_swap)

[morabanc]

Hi guys,

Yesterday we found crash tables in mysql database.

The tables are:

- nagios.nagios_eventhandlers
- nagios.nagios_hoststatus
- nagios.nagios_logentries


Today, watching log file we see....

Dec 5 23:48:33 nagiosxi kernel: Command line: ro root=/dev/mapper/vg_00-lv_root rd_NO_LUKS rd_LVM_LV=vg_00/lv_root KEYBOARDTYPE=pc KEYTABLE=es rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto LANG=es_ES.UTF-8 rd_LVM_LV=vg_00/lv_swap rd_NO_DM

Dec 5 23:48:33 nagiosxi kernel: Kernel command line: ro root=/dev/mapper/vg_00-lv_root rd_NO_LUKS rd_LVM_LV=vg_00/lv_root KEYBOARDTYPE=pc KEYTABLE=es rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=129M@0M LANG=es_ES.UTF-8 rd_LVM_LV=vg_00/lv_swap rd_NO_DM

Our lv swap:

LV Name /dev/vg_00/lv_swap
VG Name vg_00
LV UUID HgMy3Y-sIUs-x9ul-ffdz-ilFy-oooz-fMsUnG
LV Write Access read/write
LV Status available
# open 1
LV Size 2.00 GiB
Current LE 512
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1


Some more logs:

131205 23:49:16 [ERROR] /usr/libexec/mysqld: Incorrect key file for table './nagios/nagios_hoststatus.MYI'; try
to repair it
131205 23:49:16 [ERROR] /usr/libexec/mysqld: Incorrect key file for table './nagios/nagios_hoststatus.MYI'; try
to repair it
131205 23:49:20 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:49:24 [ERROR] Got error 127 when reading table './nagios/nagios_hoststatus'
131205 23:50:02 [ERROR] /usr/libexec/mysqld: Incorrect key file for table './nagios/nagios_logentries.MYI'; try
to repair it
131205 23:55:03 [ERROR] /usr/libexec/mysqld: Incorrect key file for table './nagios/nagios_eventhandlers.MYI'; t
ry to repair it

To solve the problem we had to repair the tables.

Why happen this problem ?? What we can do to prevent future problems like this ??


Thanks in advance !!!

[sreinhardt]

Most times this happens due to an improper shutdown or drives filling up. Have you checked the spaced left on those lvm partitions? As for shutting down, always use the shutdown command if possible. Disk\memory corruption is always a possibility, but ideally that is not the case here.

[morabanc]

Thanks sreinhardt, sorry for the late.

We found that the machine restart automatically so we're studying the case with other department.

I'll contact you when we recive a response.

Thanks !!!

[slansing]

Excellent, in addition I would check to make sure your drives are not full, or dangerously close. If they are not you are good to go through with the repair method here:

http://assets.nagios.com/downloads/nagi ... tabase.pdf