Page 1 of 1
Nagios XI logentries table crashes frequently
Posted: Fri Nov 06, 2015 2:09 pm
by Jithin
We have NagiosXI setup ready on our staging/production environment.
But We are getting logentries table crashed error frequently, every time we’ll repair or sometimes it turns to truncate table as well. Could you suggest any permanent solution for this.
Code: Select all
Log
Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
Re: Nagios XI logentries table crashes frequently
Posted: Fri Nov 06, 2015 2:18 pm
by hsmith
I would like some more information:
- What version of XI are you running?
- How much free space do you have? df -h
- How do your inodes look? df -ih
- How are you doing on memory usage? free -m
- How much of a load is your CPU under? top | head -n5
- Is your DB offloaded?
Re: Nagios XI logentries table crashes frequently
Posted: Wed Nov 11, 2015 4:46 am
by Jithin
(1)I'm using XI version of Nagios XI 2014R2.7
(2)We have above 60% on disk space .but once we get this issue all the logentries will fail and mysqld.log and /var/log/messages increase in GBs
(3)root 3.8M 186K 3.6M 5% /
(4,5) memory and CPU seems to be OK even during this issue.
(6) We have not done any customized off loading DB.
Re: Nagios XI logentries table crashes frequently
Posted: Wed Nov 11, 2015 11:25 am
by scottwilkerson
Does this happen after powerloss?
The most common reason for crashed tables in MySQL is power failure, or in the case of a VM, the host machine shutting down abruptly without shutting down the VM properly first such as
Re: Nagios XI logentries table crashes frequently
Posted: Thu Nov 12, 2015 4:25 am
by Jithin
Our servers are running on cloud and since last 2-3 months servers are up and running
Re: Nagios XI logentries table crashes frequently
Posted: Thu Nov 12, 2015 11:10 am
by jdalrymple
It's not uncommon for the backup process to fill the disk (on a nightly basis) which then causes the databases to crash. Is your free disk space adequate to absorb your nightly uncompressed backups prior to compression/shipping?
Re: Nagios XI logentries table crashes frequently
Posted: Thu Dec 17, 2015 1:52 am
by Jithin
Hi ,
We had almost 60% free disk space.
Re: Nagios XI logentries table crashes frequently
Posted: Thu Dec 17, 2015 10:36 am
by hsmith
Maybe if we rotate the MySQL log and wait for this to happen we can figure out a cause.
Stop your mysql service
Rename /var/log/mysqld.log
Code: Select all
mv /var/log/mysqld.log /var/log/mysqld.log.old
Start mysql
Once it happens again, please attach the log to this post for us to look at.
Meanwhile, it would be nice to see the
top | head -n5 command to see what that usage is at. I've seen high CPU load cause this before.
Re: Nagios XI logentries table crashes frequently
Posted: Thu Dec 17, 2015 10:40 am
by rkennedy
I'd like to gather a bit more information about your environment.
How many host / service checks are you running?
How much resources are assigned to this machine?