Graphs not updating

[adam.sage]

Our postgres database was uninstalled by accident a week or so ago. I was able to get that sorted out easily enough, however, the performance graphs are not updating since the database was reinstalled. Here is some output from commands I've seen in other threads:

[root@nagios perfdata]# ls -l /usr/local/nagios/var/spool
total 4992
drwxrwxr-x 2 nagios nagios 4096 Jun 4 17:32 checkresults
drwxr-xr-x 2 nagios nagios 466944 May 27 16:26 perfdata
drwxr-xr-x 2 nagios nagios 4632576 Jun 4 18:05 xidpe

[root@nagios perfdata]# service npcd status
NPCD running (pid 25398).

All the files in perfdata have the same last modified date +/- 3 minutes.

Also, I have an active maintenance contract but I cant seem to post in the Customers section. What do I have to do to post there?

[tmcdonald]

Can we get a count of the files in the perfdata directory?

Code: Select all

ls -l /usr/local/nagios/var/spool/perfdata/ | wc -l
ls -l /usr/local/nagios/var/spool/perfdata/ | tail

Also if you need access to our Customer forum, please email [email protected] with your customer ID number, email, and associated forum username.

[adam.sage]

[root@nagios log]# ls -l /usr/local/nagios/var/spool/perfdata/ | wc -l
1
[root@nagios log]# ls -l /usr/local/nagios/var/spool/perfdata/ | tail
total 0

edit:

I just noticed I should have edited the file path a bit

[root@nagios perfdata]# ls -l /usr/local/nagios/share/perfdata/ | wc -l
35
[root@nagios perfdata]# ls -l /usr/local/nagios/share/perfdata/ | tail
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 Refton_House_3448P_Switch
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 Refton_House_6224P_Switch
drwxrwxr-x 2 nagios nagios 4096 May 27 16:25 Refton_Router
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 Schnecksville_Router
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 Thompsontown_Router
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 Tower_City_Router
drwxrwxr-x 2 nagios nagios 4096 May 27 16:25 ts1.ad-qsv.qualitystoneveneer.com
drwxrwxr-x 2 nagios nagios 4096 May 27 16:26 ts2.ad-qsv.qualitystoneveneer.com
drwxrwxr-x 2 nagios nagios 4096 May 27 16:23 VCenter
drwxrwxr-x 2 nagios nagios 4096 May 27 16:25 West_Columbia_Router

[scottwilkerson]

And how about the following

Code: Select all

ll /usr/local/nagios/var/spool/xidpe|wc -l

Also, is crond running?

Code: Select all

service crond status

[adam.sage]

[root@nagios perfdata]# ll /usr/local/nagios/var/spool/xidpe|wc -l
101551
[root@nagios perfdata]# service crond status
crond (pid 1451) is running...

[slansing]

There are a massive number of files in there that have to be crunched through by NPCD before they will be displayed, it looks like NPCD may have not been running for a period of time which caused this to happen. You will either need to wait until NPCD can reap these results "if it can even catch up" or remove them once we verify npcd is running properly, and wait for new perfdata to come through.

Code: Select all

service npcd restart

Code: Select all

service npcd status

Code: Select all

tail -30 /usr/local/nagios/var/npcd.log

[adam.sage]

Like I said in the first post, there was a week or so when the database was not working. I'm not sure if it was collecting data during that time or not, but judging by the amount of files I would say yes.

[root@nagios ~]# service npcd restart
NPCD Stopped.
NPCD started.
You have new mail in /var/spool/mail/root
[root@nagios ~]# service npcd status
NPCD running (pid 22122).
[root@nagios ~]# tail -30 /usr/local/nagios/var/npcd.log
[04-15-2014 12:49:51] NPCD: WARN: MAX load reached: load 10.160000/10.000000 at i=4722
[04-15-2014 12:50:06] NPCD: WARN: MAX load reached: load 10.290000/10.000000 at i=4722
[05-15-2014 13:43:44] NPCD: Caught Termination Signal - Hasta la vista... baby
[05-15-2014 13:44:13] NPCD: npcd Daemon (0.4.14) started with PID=1426
[05-15-2014 13:44:13] NPCD: Please have a look at 'npcd -V' to get license information
[05-15-2014 13:44:13] NPCD: HINT: load_threshold is enabled - ('10.000000')
[05-15-2014 15:04:31] NPCD: Caught Termination Signal - Hasta la vista... baby
[05-15-2014 15:05:04] NPCD: npcd Daemon (0.4.14) started with PID=1406
[05-15-2014 15:05:04] NPCD: Please have a look at 'npcd -V' to get license information
[05-15-2014 15:05:04] NPCD: HINT: load_threshold is enabled - ('10.000000')
[05-21-2014 14:21:10] NPCD: Caught Termination Signal - Hasta la vista... baby
[05-21-2014 14:23:06] NPCD: npcd Daemon (0.4.14) started with PID=1449
[05-21-2014 14:23:06] NPCD: Please have a look at 'npcd -V' to get license information
[05-21-2014 14:23:06] NPCD: HINT: load_threshold is enabled - ('10.000000')
[05-28-2014 12:27:33] NPCD: Caught Termination Signal - Hasta la vista... baby
[05-28-2014 12:28:24] NPCD: npcd Daemon (0.4.14) started with PID=1446
[05-28-2014 12:28:24] NPCD: Please have a look at 'npcd -V' to get license information
[05-28-2014 12:28:24] NPCD: HINT: load_threshold is enabled - ('10.000000')
[06-04-2014 15:13:32] NPCD: Caught Termination Signal - Hasta la vista... baby
[06-04-2014 17:32:48] NPCD: npcd Daemon (0.4.14) started with PID=1461
[06-04-2014 17:32:48] NPCD: Please have a look at 'npcd -V' to get license information
[06-04-2014 17:32:48] NPCD: HINT: load_threshold is enabled - ('10.000000')
[06-04-2014 17:50:09] NPCD: Caught Termination Signal - Hasta la vista... baby
[06-04-2014 17:50:09] NPCD: npcd Daemon (0.4.14) started with PID=25398
[06-04-2014 17:50:09] NPCD: Please have a look at 'npcd -V' to get license information
[06-04-2014 17:50:09] NPCD: HINT: load_threshold is enabled - ('10.000000')
[06-06-2014 11:59:13] NPCD: Caught Termination Signal - Hasta la vista... baby
[06-06-2014 11:59:13] NPCD: npcd Daemon (0.4.14) started with PID=22122
[06-06-2014 11:59:13] NPCD: Please have a look at 'npcd -V' to get license information
[06-06-2014 11:59:13] NPCD: HINT: load_threshold is enabled - ('10.000000')
[root@nagios ~]#

[scottwilkerson]

Actually these files are supposed to get processed by crond and moved to /usr/local/nagios/var/spool/perfdata where NPCD picks them up.

Can you show the following output

Code: Select all

tail -f /var/log/cron
chage -l nagios
cat /usr/local/nagios/etc/nagios.cfg |grep _perfdata_file_processing_command

[adam.sage]

[root@nagios ~]# tail -f /var/log/cron
Jun 6 18:33:01 nagios CROND[2448]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 6 18:33:01 nagios CROND[2449]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 6 18:34:01 nagios CROND[3462]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 6 18:34:01 nagios CROND[3463]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 6 18:34:01 nagios CROND[3466]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 6 18:34:01 nagios CROND[3467]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 6 18:34:01 nagios CROND[3469]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 6 18:34:01 nagios CROND[3465]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 6 18:34:01 nagios CROND[3471]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 6 18:34:01 nagios CROND[3464]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1)

^C
[root@nagios ~]# chage -l nagios
Last password change : Feb 12, 2014
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7
[root@nagios ~]# cat /usr/local/nagios/etc/nagios.cfg |grep _perfdata_file_processing_command
service_perfdata_file_processing_command=process-service-perfdata-file-bulk
host_perfdata_file_processing_command=process-host-perfdata-file-bulk

[scottwilkerson]

Ok, so the perfdataproc cron is running, now we are going to need to see the output in

Code: Select all

tail -200 /usr/local/nagiosxi/var/perfdataproc.log