Help with NRDS?

[scottwilkerson]

Lets grab the check_results_dir you specified in this post
http://support.nagios.com/forum/viewtop ... =10#p50070

and list the directory, such as

Code: Select all

ls -l /var/nagiosramdisk/spool/checkresults

Do you get results?

[jbennett]

I'm assuming I am to run this on the nagios server, correct?

If so:

Code: Select all

-rw------- 1 nagios nagios   0 Apr 17 15:28 ce5gsNZ.ok
-rw------- 1 nagios nagios 474 Apr 17 15:28 cFGyfnq
-rw------- 1 nagios nagios   0 Apr 17 15:28 cFGyfnq.ok
-rw------- 1 nagios nagios 463 Apr 17 15:28 cGsUg7W
-rw------- 1 nagios nagios   0 Apr 17 15:28 cGsUg7W.ok
-rw------- 1 nagios nagios 473 Apr 17 15:28 chHh7sx
-rw------- 1 nagios nagios   0 Apr 17 15:28 chHh7sx.ok
-rw------- 1 nagios nagios 474 Apr 17 15:28 cI9e5hZ
-rw------- 1 nagios nagios   0 Apr 17 15:28 cI9e5hZ.ok
-rw------- 1 nagios nagios 473 Apr 17 15:28 cIaMZ6s
-rw------- 1 nagios nagios   0 Apr 17 15:28 cIaMZ6s.ok
-rw------- 1 nagios nagios 472 Apr 17 15:28 cKdGVxu
-rw------- 1 nagios nagios   0 Apr 17 15:28 cKdGVxu.ok
-rw------- 1 nagios nagios 471 Apr 17 15:28 cKUx4KY
-rw------- 1 nagios nagios   0 Apr 17 15:28 cKUx4KY.ok
-rw------- 1 nagios nagios 476 Apr 17 15:28 cLfeNZq
-rw------- 1 nagios nagios   0 Apr 17 15:28 cLfeNZq.ok
-rw------- 1 nagios nagios 473 Apr 17 15:28 cLQvs91
-rw------- 1 nagios nagios   0 Apr 17 15:28 cLQvs91.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 cMDu7Vr
-rw------- 1 nagios nagios   0 Apr 17 15:28 cMDu7Vr.ok
-rw------- 1 nagios nagios 472 Apr 17 15:28 cob9V7V
-rw------- 1 nagios nagios   0 Apr 17 15:28 cob9V7V.ok
-rw------- 1 nagios nagios 472 Apr 17 15:28 cohpvcr
-rw------- 1 nagios nagios   0 Apr 17 15:28 cohpvcr.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 cozdAzV
-rw------- 1 nagios nagios   0 Apr 17 15:28 cozdAzV.ok
-rw------- 1 nagios nagios 474 Apr 17 15:28 cPDWCl0
-rw------- 1 nagios nagios   0 Apr 17 15:28 cPDWCl0.ok
-rw------- 1 nagios nagios 476 Apr 17 15:28 cqmaYw1
-rw------- 1 nagios nagios   0 Apr 17 15:28 cqmaYw1.ok
-rw------- 1 nagios nagios 471 Apr 17 15:28 cqPiwBU
-rw------- 1 nagios nagios   0 Apr 17 15:28 cqPiwBU.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 cQtf9MV
-rw------- 1 nagios nagios   0 Apr 17 15:28 cQtf9MV.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 crOtf4u
-rw------- 1 nagios nagios   0 Apr 17 15:28 crOtf4u.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 cryhKDv
-rw------- 1 nagios nagios   0 Apr 17 15:28 cryhKDv.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 cSSABdw
-rw------- 1 nagios nagios   0 Apr 17 15:28 cSSABdw.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 cSTdIT0
-rw------- 1 nagios nagios   0 Apr 17 15:28 cSTdIT0.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 ct8fWKW
-rw------- 1 nagios nagios   0 Apr 17 15:28 ct8fWKW.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 cu1pQsX
-rw------- 1 nagios nagios   0 Apr 17 15:28 cu1pQsX.ok
-rw------- 1 nagios nagios 474 Apr 17 15:28 cUt6fxt
-rw------- 1 nagios nagios   0 Apr 17 15:28 cUt6fxt.ok
-rw------- 1 nagios nagios 464 Apr 17 15:28 cVQ70iV
-rw------- 1 nagios nagios   0 Apr 17 15:28 cVQ70iV.ok
-rw------- 1 nagios nagios 473 Apr 17 15:28 cVtAlPw
-rw------- 1 nagios nagios   0 Apr 17 15:28 cVtAlPw.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 cw13EsW
-rw------- 1 nagios nagios   0 Apr 17 15:28 cw13EsW.ok
-rw------- 1 nagios nagios 465 Apr 17 15:28 cXAtfks
-rw------- 1 nagios nagios   0 Apr 17 15:28 cXAtfks.ok
-rw------- 1 nagios nagios 463 Apr 17 15:28 cxryK1t
-rw------- 1 nagios nagios   0 Apr 17 15:28 cxryK1t.ok
-rw------- 1 nagios nagios 473 Apr 17 15:28 cZRK9Eq
-rw------- 1 nagios nagios   0 Apr 17 15:28 cZRK9Eq.ok

When I try to view these files I can't as they are apparently being processed too quickly?

[scottwilkerson]

I'm assuming I am to run this on the nagios server, correct?

Correct

When I try to view these files I can't as they are apparently being processed too quickly?

This is normal.

Can you tail the following for a minute to see if you are getting any errors

Code: Select all

tail -f /usr/local/nagiosxi/var/feedproc.log

Also, please run the following the verify the feedproc cron is running

Code: Select all

ps -ef |grep -v grep|grep feedproc

[jbennett]

Code: Select all

# tail -f /usr/local/nagiosxi/var/feedproc.log
.PHP Warning:  exec(): Unable to fork [grep 'Warning: Check result queue contained results for ' /usr/local/nagiosxi/var/corelog.diff] in /usr/local/nagiosxi/scripts/parse_core_eventlog.php on line 92
PHP Warning:  exec(): Unable to fork [grep 'Passive check result was received for ' /usr/local/nagiosxi/var/corelog.diff] in /usr/local/nagiosxi/scripts/parse_core_eventlog.php on line 92
.PHP Warning:  exec(): Unable to fork [php -q /usr/local/nagiosxi/scripts/parse_core_eventlog.php] in /usr/local/nagiosxi/cron/feedproc.php on line 96
.

It appears that the feedproc cron is NOT running.

If I try and run the feedproc script manually, these are the results:

Code: Select all

# /usr/local/nagiosxi/cron/feedproc.php
...
PROCESSED 0 COMMANDS

[scottwilkerson]

It appears that the feedproc cron is NOT running.

This could be the problem.

Can you run the following and post the output

Code: Select all

cat /etc/cron.d/nagiosxi
tail -50 /var/log/cron

[jbennett]

Can you run the following and post the output

Code: Select all

cat /etc/cron.d/nagiosxi
tail -50 /var/log/cron

Code: Select all

cat /etc/cron.d/nagiosxi
# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi

# Backup MySQL & PostgreSQL Databases
0   7 * * * root   /root/scripts/automysqlbackup
0   8 * * * root   /root/scripts/autopostgresqlbackup

*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
*   * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01  * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php > /usr/local/nagiosxi/var/deadpool.log 2>&1
# tail -50 /var/log/cron
Apr 18 13:58:01 nagiosxivm crond[7567]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7567]: (nagios) ERROR (failed to change user)
Apr 18 13:58:01 nagiosxivm crond[7563]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7563]: (nagios) ERROR (failed to change user)
Apr 18 13:58:01 nagiosxivm crond[7562]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7562]: (nagios) ERROR (failed to change user)
Apr 18 13:58:01 nagiosxivm crond[7568]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7568]: (nagios) ERROR (failed to change user)
Apr 18 13:58:01 nagiosxivm crond[7566]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7566]: (nagios) ERROR (failed to change user)
Apr 18 13:58:01 nagiosxivm crond[7565]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:58:01 nagiosxivm crond[7565]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8209]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8208]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8208]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8209]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8210]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8210]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8212]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8212]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8211]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8211]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8214]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8214]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8213]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8213]: (nagios) ERROR (failed to change user)
Apr 18 13:59:01 nagiosxivm crond[8215]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 13:59:01 nagiosxivm crond[8215]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10976]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10976]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10975]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10983]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10983]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10975]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10979]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10979]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10980]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10980]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10977]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10977]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10982]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10982]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm CROND[10986]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lock/mrtg/mrtg_l --confcache-file /var/lib/mrtg/mrtg.ok &> /dev/null)
Apr 18 14:00:01 nagiosxivm CROND[10987]: (root) CMD (/usr/lib/sa/sa1 -S DISK 1 1)
Apr 18 14:00:01 nagiosxivm crond[10981]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10981]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10978]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10978]: (nagios) ERROR (failed to change user)
Apr 18 14:00:01 nagiosxivm crond[10984]: (CRON) ERROR (setreuid failed): Resource temporarily unavailable
Apr 18 14:00:01 nagiosxivm crond[10984]: (nagios) ERROR (failed to change user)

[scottwilkerson]

Hmm, not able to run any of the crons...

Is your nagios user expired?

Code: Select all

chage -l nagios

[jbennett]

Hmm, not able to run any of the crons...

Is your nagios user expired?

Code: Select all

chage -l nagios

Code: Select all

# chage -l nagios
Last password change                                    : Aug 29, 2011
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7

[scottwilkerson]

Please post the output of

Code: Select all

ps -ef|wc -l
ulimit -u
cat /etc/security/limits.conf |grep -v '#'

[jbennett]

Please post the output of

Code: Select all

ps -ef|wc -l
ulimit -u
cat /etc/security/limits.conf |grep -v '#'

Code: Select all

# ps -ef|wc -l
1525
# ulimit -u
1024
# cat /etc/security/limits.conf |grep -v '#'
(returned a blank line)