Root File system getting full NagiosXI

[mejokj]

Hi Team,

We are facing an issue with Nagiosxi.

The root disk is getting full every time and not able to login because of that.

We have 45GB of / filesystem with 80 hosts and 176 services. Initially, we have only 35GB / space, which got full and increased to 45GB. Now the 45GB also got full.

While looking into the folder, the MySQL folder getting full always.

Logs using very low space.

Can you suggest why the / filesystem is getting full very frequently? Is it because of any issue with the database (done database repair script, doesn't find any issue) because size of MySQL folder increasing every time we increase / filesystem.

[gsmith]

Hi

1. Take a look at /usr/local/nagios/etc/nagios.cfg and let me know the value for the setting:
log_rotation_method=

2. Which mysql directory are you referring too? For example I have:
./var/lib/mysql
./var/lib/mysql/mysql
./usr/lib/python2.7/site-packages/ansible/modules/database/mysql
./usr/lib64/perl5/vendor_perl/auto/DBD/mysql
./usr/lib64/perl5/vendor_perl/DBD/mysql
./usr/lib64/mysql
./usr/share/mysql
./usr/include/mysql
./store/backups/mysql
./store/backups/mysql/daily/mysql
./store/backups/mysql/weekly/mysql

3. Please run this command and let me know the output:
awk '/"nagiosxi" => array\(/{getline;print}' /usr/local/nagiosxi/html/config.inc.php


Thanks

[mejokj]

Hi gsmith,

Shared the screenshots as a PM with you.

Please have a look.

Thanks

[mejokj]

Attached some more screenshots

[mejokj]

Database is repaired successfully.

Want to understand the disk getting full due to /var/lib/mysql folder size increasing. Due to / file system database tables getting crashing and nagios getting down


1. Take a look at /usr/local/nagios/etc/nagios.cfg and let me know the value for the setting:
log_rotation_method=

================================
log_rotation_method=d

[gsmith]

Hi

Could you please run:
find / -path ./proc -prune -o -path ./sys -prune -o -iname *.ibd

and post the output?

Thanks!

[mejokj]

Send you as PM.

[gsmith]

Hi,

So I think the #sql*.ibd files are from mysql crashes. For now please leave them there while I investigate.

Please run:
ls -l /var/lib/mysql/nagio* >> /tmp/dbfiles.txt

And send me the resulting /tmp/dbfiles.txt file.

Thanks!

[mejokj]

PM the details requested

[mejokj]

Any update?