Page 1 of 1

Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 12:34 pm
by brucej543
NagiosXI 5.6.9 Around 11:24 AM, the var file system jumped almost 9 GB of space used the max out to 100% used. After adding additional space the file system usage dropped about 9 GB. I was trying to research the reason using the event log, but it does not display any information at all.
The event log has a 0 records to display. How do I get this corrected?
A profile is attached.
below is a display of the file size of the nagios_logentries
[root@bcnagios01 nagios]# find /var -type f -size +2000000 -exec ls -al {} \;
-rw-rw---- 1 mysql mysql 1071376384 Dec 19 06:46 /var/lib/mysql/nagios/nagios_logentries.TMM
-rw-rw---- 1 mysql mysql 3075026944 Jan 21 12:03 /var/lib/mysql/nagios/nagios_logentries.MYI
-rw-rw---- 1 mysql mysql 6631765696 Jan 21 11:26 /var/lib/mysql/nagios/nagios_logentries.MYD

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 12:38 pm
by scottwilkerson
Looking at the profile it looks like you have some crashed DB tables (probably from the full /var path)

Here is a document outlining how to repair that
https://assets.nagios.com/downloads/nag ... tabase.pdf

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:01 pm
by brucej543
Scott, Ran database repair and successfully completed, now the log is showing. Attached is the profile after the DB repair.
Thanks for your help

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:10 pm
by scottwilkerson
Database looks better now..

Anything else we can help with today?

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:17 pm
by brucej543
My biggest concern, is what caused a 9 GB in space usage over just a few minutes in the /var file system. Does the app perform some sort of file reorg and log rotation that would have causes this big jump.

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:38 pm
by scottwilkerson
There are several things that can make it jump, the size of the jump would depend on the on the size of the data being accessed.

The /var directory holds all of the mysql data, and will also hold any temporary tables that mysql need to create depending on actions taken.

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:47 pm
by brucej543
Scott, thank you for your help and the information. This can be closed

Re: Event Log not displaying after /var hit 100%

Posted: Tue Jan 21, 2020 1:57 pm
by scottwilkerson
brucej543 wrote:Scott, thank you for your help and the information. This can be closed
Great!

Locking thread