Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

find process flooding my system

https://support.nagios.com/forum/viewtopic.php?t=8531

Page 1 of 2

find process flooding my system

Posted: Wed Dec 05, 2012 10:50 am
by ytlcomm
Hi,

My system got flooded with this 'find' process. I have no idea how to solve this problem. Load average is ridiculously high. Please help.
12-5-2012 11-47-51 PM.png
- Faiz

Re: find process flooding my system

Posted: Wed Dec 05, 2012 10:53 am
by scottwilkerson
I would start by killing the find processes.

Code: Select all

killall -9 find
It is possible because the load is so high it cannot complete this standard maintenance task

Re: find process flooding my system

Posted: Wed Dec 05, 2012 11:07 am
by ytlcomm
I have kill it several time and it coming back again.

Re: find process flooding my system

Posted: Wed Dec 05, 2012 11:56 am
by mguthrie
Can you show the output from:

Code: Select all

ps aux | grep php

Re: find process flooding my system

Posted: Wed Dec 05, 2012 8:22 pm
by ytlcomm

Code: Select all

# ps aux | grep php
nagios     635  0.0  0.0   9228   608 ?        Ss   10:38   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios     643  0.0  0.0 222708  5176 ?        S    10:38   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    1201  0.0  0.0   9228   592 ?        Ss   10:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    1206  0.0  0.0 222708  3016 ?        S    10:29   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    1361  0.0  0.0   9228   616 ?        Ss   10:43   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    1362  0.0  0.0 222708  2948 ?        S    10:43   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    2457  0.0  0.0   9228   580 ?        Ss   10:10   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    2465  0.0  0.0 222708  1360 ?        S    10:10   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    3006  0.0  0.0   9228   580 ?        Ss   10:11   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    3011  0.0  0.0 222708  1340 ?        S    10:11   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    5073  0.0  0.0   9228   580 ?        Ss   10:12   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    5081  0.0  0.0 222708  1332 ?        S    10:12   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    5446  0.0  0.0   9228   624 ?        Ss   10:16   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    5454  0.0  0.0 222708  5704 ?        S    10:16   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    5706  0.0  0.0   9228   600 ?        Ss   10:49   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    5711  0.0  0.1 222708 10260 ?        S    10:49   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    5904  0.0  0.0   9228   580 ?        Ss   10:18   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    5910  0.0  0.0 222708  2184 ?        S    10:18   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    6458  0.0  0.0   9228   600 ?        Ss   10:44   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    6469  0.0  0.0 222708  6132 ?        S    10:44   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    8413  0.0  0.0   9228   692 ?        Ss   10:24   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    8425  0.0  0.0 222708  7248 ?        S    10:24   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    8535  0.0  0.0   9228   580 ?        Ss   10:13   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    8542  0.0  0.0 222708  1284 ?        S    10:13   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios    9070  0.0  0.0   9228   640 ?        Ss   10:20   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios    9078  0.0  0.1 222708 12688 ?        S    10:20   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   10423  0.0  0.0   9228   580 ?        Ss   10:21   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   10429  0.0  0.0 222708  1812 ?        S    10:21   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   10818  0.0  0.0   9228   588 ?        Ss   10:30   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   10823  0.0  0.0 222708  2748 ?        S    10:30   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   10906  0.0  0.0   9228   584 ?        Ss   10:35   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   10921  0.0  0.0 222708  4644 ?        S    10:35   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   12065  0.0  0.0   9228   596 ?        Ss   10:31   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   12072  0.0  0.0 222708  1568 ?        S    10:31   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   12458  0.0  0.0   9228   596 ?        Ss   10:45   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   12467  0.0  0.0 222668  3400 ?        S    10:45   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   13797  0.0  0.0   9228   584 ?        Ss   10:32   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   13801  0.0  0.1 222708 15168 ?        S    10:32   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   14387  0.0  0.0   9228   600 ?        Ss   10:46   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   14394  0.0  0.0 222708  3160 ?        S    10:46   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   15174  0.0  0.0   9228   596 ?        Ss   10:36   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   15178  0.0  0.0 222708  2852 ?        S    10:36   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   15578  0.0  0.0   9228   700 ?        Ss   10:22   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   15589  0.0  0.1 222708 13328 ?        S    10:22   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   15643  0.0  0.0   9228   580 ?        Ss   10:04   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   15648  0.0  0.0 222708  1284 ?        S    10:04   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   15896  0.0  0.0   9228   604 ?        Ss   10:25   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   15904  0.0  0.0 222708  2236 ?        S    10:25   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   16701  0.0  0.0   9228   596 ?        Ss   10:26   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   16709  0.0  0.1 222708 10556 ?        S    10:26   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   17518  0.0  0.0   9228   580 ?        Ss   10:08   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   17524  0.0  0.0 222708  1284 ?        S    10:08   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   17983  0.0  0.0   9228   628 ?        Ss   10:39   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   17992  0.0  0.0 222708  6056 ?        S    10:39   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
root     18354  0.0  0.0 103248   860 pts/4    S+   12:07   0:00 grep php
nagios   19545  0.0  0.0   9228   580 ?        Ss   10:05   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   19553  0.0  0.0 222708  1288 ?        S    10:05   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   19892  0.0  0.0   9228   580 ?        Ss   10:14   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   19896  0.0  0.0 222708  1284 ?        S    10:14   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   20142  0.0  0.0   9228   580 ?        Ss   10:06   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   20149  0.0  0.0 222708  1284 ?        S    10:06   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   20450  0.0  0.0   9228   596 ?        Ss   10:33   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   20453  0.0  0.0 222708  3080 ?        S    10:33   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   20562  0.0  0.0   9228   580 ?        Ss   10:17   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   20573  0.0  0.0 222708  1284 ?        S    10:17   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   20922  0.0  0.0   9228   668 ?        Ss   10:54   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   20930  0.0  0.1 222708 14912 ?        S    10:54   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   21064  0.0  0.0   9228   596 ?        Ss   10:27   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   21066  0.0  0.0 222708  3632 ?        S    10:27   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   21132  0.0  0.0   9228   684 ?        Ss   10:37   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   21139  0.0  0.1 222708 11896 ?        S    10:37   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   21794  0.0  0.0   9228   584 ?        Ss   10:19   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   21800  0.0  0.0 222708  4764 ?        S    10:19   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   24515  0.0  0.0   9228   592 ?        Ss   10:48   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   24527  0.0  0.0 222708  4076 ?        S    10:48   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   27377  0.0  0.0   9228   604 ?        Ss   10:28   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   27381  0.0  0.0 222708  3176 ?        S    10:28   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   27619  0.0  0.0   9228   580 ?        Ss   10:07   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   27624  0.0  0.0 222708  1300 ?        S    10:07   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   28940  0.0  0.0   9228   580 ?        Ss   10:15   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   28948  0.0  0.0 222708  1284 ?        S    10:15   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   29246  0.0  0.0   9228   620 ?        Ss   10:40   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   29255  0.0  0.0 222708  4924 ?        S    10:40   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   30531  0.0  0.0   9228   604 ?        Ss   10:34   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   30539  0.0  0.0 222708  4492 ?        S    10:34   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   31151  0.0  0.0   9228   616 ?        Ss   10:23   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   31154  0.0  0.2 222708 19224 ?        S    10:23   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   31317  0.0  0.0   9228   608 ?        Ss   10:41   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   31322  0.0  0.0 222708  5188 ?        S    10:41   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   32506  0.0  0.0   9228   608 ?        Ss   10:42   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   32517  0.0  0.0 222708  5388 ?        S    10:42   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
nagios   32548  0.0  0.0   9228   580 ?        Ss   10:09   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
nagios   32553  0.0  0.0 222708  1392 ?        S    10:09   0:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php
Please help... :cry:

Re: find process flooding my system

Posted: Thu Dec 06, 2012 9:40 am
by mguthrie
Yikes. Yeah that's the problem. For the moment, run the following:

Code: Select all

service crond stop
killall php
service crond start
Can you manually run the following cron job and see what kind of output you get?

Code: Select all

php /usr/local/nagiosxi/cron/cleaner.php

Re: find process flooding my system

Posted: Thu Dec 06, 2012 9:16 pm
by ytlcomm
Its showing some out put of 'No such file or directory'. Should I run it endlessly? Find process accumulating... I've commented cleaner.php in cron file for temporary.
12-7-2012 10-15-43 AM.png

Re: find process flooding my system

Posted: Fri Dec 07, 2012 10:45 am
by mguthrie
How many files do you have in the following two directories?

Code: Select all

ls -f /etc/nagiosql/backup/hosts | wc -l
ls -f /etc/nagiosql/backup/hosts | wc -l
I'm wondering if there's just too many files in there where it gets stuck on an endless loop...

Re: find process flooding my system

Posted: Sat Dec 08, 2012 8:26 pm
by ytlcomm

Code: Select all

# ls -f /etc/nagiosql/backup/hosts | wc -l
310975
Yeah, it's quite a lot of files there. But I guess we don't have that many hosts.

Re: find process flooding my system

Posted: Mon Dec 10, 2012 10:20 am
by mguthrie
I would go ahead and remove those backup config files in that directory. We added a feature recently where you can restore a database snapshot of the configs, which is far more effective than keeping the old .cfg files around. I'm betting once that directory is cleaned up that issue will disappear.
All times are UTC-05:00
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited