/usr/local/nagios/var/spool/xidpe filled inodes

[kinnema]

Hello,

I'm having the same issue [referenced here: http://support.nagios.com/forum/viewtop ... 107#p28107] - 500,000+ files using up all the inodes in /usr/local/nagios/var/spool/xidpe. Although, this is happening on three DNX nodes and not the master NagiosXI server. Permissions look fine for the directory. What is supposed to actually remove these files?

I only see the following references in commands.cfg to move perfdata into the directory, but don't see anything about removing the files:

define command {
command_name process-host-perfdata-file-bulk
command_line /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

define command {
command_name process-service-perfdata-file-bulk
command_line /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

I'm getting about 8 of these flies moved there per second, although it seems like it can't be keeping that up indefinitely since we only noticed the problem today and at that rate we'd hit well over 600,000 over a day.

Any ideas?

[scottwilkerson]

Does this DNX node have Nagios XI installed on it as well?

[kinnema]

Yes, the DNX nodes do have XI installed.

[scottwilkerson]

Is crond running on the DNX nodes?

Code: Select all

service crond status

These directories usually get reaped by the perfdataproc.php cron job

Code: Select all

ps aux|grep cron/perfdataproc.php

[kinnema]

crond is running, but I'm not seeing the perfdataproc.php cron job you refer to.

Would I just need to add that job to crontab and how often should it run?

[scottwilkerson]

/etc/cron.d/nagiosxi should contain the following:

Code: Select all

# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi

# Backup MySQL & PostgreSQL Databases
0   7 * * * root   /root/scripts/automysqlbackup
0   8 * * * root   /root/scripts/autopostgresqlbackup

*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01  * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1

Note that when you add the perfdataproc.php cron with a directory of 600000 items, it is going to make this server VERY busy for a while