Database repair and /var/tmp filling up

[DFaught]

Hi,
I am a Nagios admin entering this on behalf of our platform support group that maintains the OS and software for Nagios XI.

Attached are files that describe the problem and the system profile.

Thank you for your help.

Regards,
Dave

[rkennedy]

What files appear to be filling up the /var/tmp directory? I looked at your profile, and things seem to be OK for disk:

Code: Select all

Filesystem                           Size  Used Avail Use% Mounted on
/dev/mapper/rhel-root                 15G  2.4G   13G  16% /
devtmpfs                              16G     0   16G   0% /dev
tmpfs                                 16G     0   16G   0% /dev/shm
tmpfs                                 16G     0   16G   0% /sys/fs/cgroup
tmpfs                                 16G  401M   16G   3% /run
/dev/sda1                           1014M  270M  745M  27% /boot
/dev/mapper/rhel-var                 7.0G  5.4G  1.7G  77% /var
/dev/mapper/datavg-var_lib_mysql     150G  9.2G  141G   7% /var/lib/mysql
/dev/mapper/rhel-var_tmp              13G   34M   13G   1% /var/tmp
/dev/mapper/rhel-var_log             4.5G  2.6G  2.0G  57% /var/log
/dev/mapper/rhel-var_log_audit      1014M   74M  941M   8% /var/log/audit
/dev/mapper/rhel-home                2.0G  109M  1.9G   6% /home
/dev/mapper/datavg-store              75G  9.9G   66G  14% /store
/dev/mapper/datavg-data_trans        253M   14M  240M   6% /data/trans
/dev/mapper/datavg-usr_local_nagios   25G   20G  5.4G  79% /usr/local/nagios
/dev/mapper/datavg-opt               509M  262M  248M  52% /opt
/dev/mapper/datavg-opt_bit9         1014M   74M  941M   8% /opt/bit9
/dev/mapper/datavg-opt_chef         1014M  228M  787M  23% /opt/chef
/dev/mapper/datavg-opt_tivoli       1014M  762M  253M  76% /opt/tivoli
/dev/mapper/datavg-opt_ftl           125M  8.0M  117M   7% /opt/ftl
/dev/mapper/datavg-opt_ctmagent     1014M  358M  657M  36% /opt/ctmagent
/dev/mapper/datavg-srv                29M  1.8M   27M   6% /srv
/dev/mapper/datavg-srv_bit9          5.0G  603M  4.4G  12% /srv/bit9
/dev/mapper/rhel-tmp                 4.0G   42M  4.0G   2% /tmp
tmpfs                                3.2G     0  3.2G   0% /run/user/4847

The SQL repair can be needed if things become corrupt - usually not very often though. If it's happening consistently, then you may have a further performance issue going on somewhere. With the amount of services you have, it may be worth segregating out to another instance, and look at Nagios Fusion.

Code: Select all

Total Hosts: 2153 
Total Services: 32242 

[mcapra]

Certain MySQL storage engines (I believe MyISAM is included) store temporary files in the TMPDIR if there is a pending transaction that is taking a particularly long time, and MySQL 8 natively handles grouping/ordering requests by storing results to TMPDIR before joining/sorting them. More info on that here:
https://dev.mysql.com/doc/refman/8.0/en ... files.html

In a nut-shell, with an environment this large certain object operations within Nagios XI could be creating rather large temp files.

As this is a particularly large environment, a good first step might be to offload MySQL to a separate machine if you haven't already done that:
https://assets.nagios.com/downloads/nag ... Server.pdf

[cdienger]

@DFaught can you confirm if the directory is having issues with sql temp files? If so then offloading a @mcpra suggested would be something to investigate for the environment. Additionally, there are other performance improvements that can be made covered in https://assets.nagios.com/downloads/nag ... ios-XI.pdf