Nagios XI logentries table crashes frequently

[Jithin]

We have NagiosXI setup ready on our staging/production environment.

But We are getting logentries table crashed error frequently, every time we’ll repair or sometimes it turns to truncate table as well. Could you suggest any permanent solution for this.

Code: Select all

 Log
 Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed

[hsmith]

I would like some more information:

1. What version of XI are you running?
2. How much free space do you have? df -h
3. How do your inodes look? df -ih
4. How are you doing on memory usage? free -m
5. How much of a load is your CPU under? top | head -n5
6. Is your DB offloaded?

[Jithin]

(1)I'm using XI version of Nagios XI 2014R2.7
(2)We have above 60% on disk space .but once we get this issue all the logentries will fail and mysqld.log and /var/log/messages increase in GBs
(3)root 3.8M 186K 3.6M 5% /
(4,5) memory and CPU seems to be OK even during this issue.
(6) We have not done any customized off loading DB.

[scottwilkerson]

Does this happen after powerloss?

The most common reason for crashed tables in MySQL is power failure, or in the case of a VM, the host machine shutting down abruptly without shutting down the VM properly first such as

Code: Select all

shutdown -h now

[Jithin]

Our servers are running on cloud and since last 2-3 months servers are up and running

[jdalrymple]

It's not uncommon for the backup process to fill the disk (on a nightly basis) which then causes the databases to crash. Is your free disk space adequate to absorb your nightly uncompressed backups prior to compression/shipping?

[Jithin]

Hi ,


We had almost 60% free disk space.

[hsmith]

Maybe if we rotate the MySQL log and wait for this to happen we can figure out a cause.

Stop your mysql service

Code: Select all

service mysqld stop

Rename /var/log/mysqld.log

Code: Select all

mv /var/log/mysqld.log /var/log/mysqld.log.old

Start mysql

Code: Select all

service mysqld start

Once it happens again, please attach the log to this post for us to look at.

Meanwhile, it would be nice to see the top | head -n5 command to see what that usage is at. I've seen high CPU load cause this before.

[rkennedy]

I'd like to gather a bit more information about your environment.
How many host / service checks are you running?
How much resources are assigned to this machine?