Performance Graph Issues on New Install

[Juddnick]

Hi

I have a new installation of Nagios Xi which I cannot seem to get any performance graphs working on. Even for localhost there is nothing. The initial install had issues due to an incorrectly set up nagios account (there was no /var/spool/nagios directory) - this prevented the setup and running of the cron jobs. This was corrected and is now ok but could this explain why performance collection is not working?

The system status panel shows "performance grapher" is running and Monitoring Engine Status shows performance data as green.

Is there a clear guide describing how performance graphing works and how to set up basic performance graphs? I looked through the documentation but cannot find what I am looking for.

I have followed the info in the FAQs for performance graph issues -

- npcd service is running
- process_perfdata.pl has correct permissions

I have also clicked the "process perf data" radio button on each of the service templates - but still no graphs or even collected rrd.

Are there any other basic checks I can run? There are NO logs being generated (/usr/local/nagios/var/perfdata.log does not exist) so I suspect there is either a process not running which should be or I have not set performance graphing up correctly to begin with.

The following directory is also empty:
/usr/local/nagios/share/perfdata/

[abrist]

Please follow the faq steps to increase your log level for npcd and process_perfdata.pl:
http://support.nagios.com/wiki/index.ph ... leshooting
Afterwards, wait 15 minutes and then post a long tail of both logs:

Code: Select all

tail -25 /usr/local/nagios/var/perfdata.log
tail -25 /usr/local/nagios/var/npcd.log

Remember to decrease the log levels back to default when you are all done here!

[Juddnick]

Thanks abrist:

Code: Select all

[root@ms00556 perfdata]# tail -25 /usr/local/nagios/var/perfdata.log
tail: cannot open `/usr/local/nagios/var/perfdata.log' for reading: No such file or directory

[root@ms00556 perfdata]# tail -25 /usr/local/nagios/var/npcd.log
[11-26-2014 13:35:58] NPCD: No more files to process... waiting for 15 seconds
[11-26-2014 13:36:13] NPCD: Found 2 files in /usr/local/nagios/var/spool/perfdata/
[11-26-2014 13:36:13] NPCD: DEBUG: load 0.370000/10.000000
[11-26-2014 13:36:13] NPCD: ThreadCounter 0/5 File is .
[11-26-2014 13:36:13] NPCD: DEBUG: load 0.370000/10.000000
[11-26-2014 13:36:13] NPCD: ThreadCounter 0/5 File is ..
[11-26-2014 13:36:13] NPCD: No more files to process... waiting for 15 seconds
[11-26-2014 13:36:28] NPCD: Found 2 files in /usr/local/nagios/var/spool/perfdata/
[11-26-2014 13:36:28] NPCD: DEBUG: load 0.360000/10.000000
[11-26-2014 13:36:28] NPCD: ThreadCounter 0/5 File is .
[11-26-2014 13:36:28] NPCD: DEBUG: load 0.360000/10.000000
[11-26-2014 13:36:28] NPCD: ThreadCounter 0/5 File is ..
[11-26-2014 13:36:28] NPCD: No more files to process... waiting for 15 seconds
[11-26-2014 13:36:43] NPCD: Found 2 files in /usr/local/nagios/var/spool/perfdata/
[11-26-2014 13:36:43] NPCD: DEBUG: load 0.280000/10.000000
[11-26-2014 13:36:43] NPCD: ThreadCounter 0/5 File is .
[11-26-2014 13:36:43] NPCD: DEBUG: load 0.280000/10.000000
[11-26-2014 13:36:43] NPCD: ThreadCounter 0/5 File is ..
[11-26-2014 13:36:43] NPCD: No more files to process... waiting for 15 seconds
[11-26-2014 13:36:58] NPCD: Found 2 files in /usr/local/nagios/var/spool/perfdata/
[11-26-2014 13:36:58] NPCD: DEBUG: load 0.220000/10.000000
[11-26-2014 13:36:58] NPCD: ThreadCounter 0/5 File is .
[11-26-2014 13:36:58] NPCD: DEBUG: load 0.220000/10.000000
[11-26-2014 13:36:58] NPCD: ThreadCounter 0/5 File is ..
[11-26-2014 13:36:58] NPCD: No more files to process... waiting for 15 seconds

[abrist]

[root@ms00556 perfdata]# tail -25 /usr/local/nagios/var/perfdata.log
tail: cannot open `/usr/local/nagios/var/perfdata.log' for reading: No such file or directory

You most likely did not increase the log levels as this file was never populated. Could you refer to the FAQ once again and make sure to increase the log level for perfdata?

[Juddnick]

I did change this value as you suggested:

Code: Select all

#
#
LOG_FILE = /usr/local/nagios/var/perfdata.log
#
# Loglevel 0=silent 1=normal 2=debug
#
#LOG_LEVEL = 0
LOG_LEVEL = 2
#

[Box293]

Can you please post the output of this command:

Code: Select all

grep perfdata /usr/local/nagios/etc/nagios.cfg

Also attach the file /usr/local/nagios/etc/commands.cfg

[Juddnick]

Thanks:

Code: Select all

[root@ms00556 perfdata]# grep perfdata /usr/local/nagios/etc/nagios.cfg
service_perfdata_file=/usr/local/nagios/var/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tSERVICEOUTPUT::$SERVICEOUTPUT$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=process-service-perfdata-file-bulk
host_perfdata_file=/usr/local/nagios/var/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tHOSTOUTPUT::$HOSTOUTPUT$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=process-host-perfdata-file-bulk
perfdata_timeout=5

[Box293]

Well all of that seems configured correctly.

Can you give the output of:

Code: Select all

ls -al /usr/local/nagios/var/spool/perfdata/
ls -al /usr/local/nagios/share/

If /usr/local/nagios/var/perfdata.log is not being populated then let's turn on nagios debugging.

Try setting the debug level on and then restart Nagios.

Code: Select all

sed -i 's/.*debug_level=.*/debug_level=-1/g' /usr/local/nagios/etc/nagios.cfg
service nagios restart

Wait about 5 minutes and then send through the file /usr/local/nagios/var/nagios.debug

[Juddnick]

Thanks Box293:

Code: Select all

[root@ms00556 ~]# ls -al /usr/local/nagios/var/spool/perfdata/
total 8
drwxr-xr-x. 2 nagios nagios 4096 Nov 14 10:24 .
drwxr-xr-x. 5 nagios nagios 4096 Nov 14 10:24 ..
[root@ms00556 ~]#

Code: Select all

[root@ms00556 ~]# ls -al /usr/local/nagios/share/
total 112
drwxrwxr-x. 14 nagios nagios 4096 Nov 14 11:41 .
drwxr-xr-x.  9 root   root   4096 Nov 14 10:24 ..
drwxr-xr-x.  2 root   root   4096 Nov 14 10:25 ajaxterm
-rw-rw-r--.  1 nagios nagios  576 Nov 14 11:41 config.inc.php
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 11:41 contexthelp
drwxrwxr-x.  3 nagios nagios 4096 Nov 14 10:23 docs
drwxrwxr-x.  3 nagios nagios 4096 Nov 14 11:41 images
drwxrwxr-x.  3 nagios nagios 4096 Nov 14 11:41 includes
-rw-rw-r--.  1 nagios nagios 1628 Nov 14 11:41 index.php
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 11:41 js
-rw-rw-r--.  1 nagios nagios 1176 Nov 14 11:41 jsonquery.html
drwxr-xr-x.  4 root   root   4096 Nov 14 10:24 locale
-rw-rw-r--.  1 nagios nagios 8697 Nov 14 11:41 main.php
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 10:23 media
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 10:25 perfdata
drwxr-xr-x.  8 nagios nagios 4096 Nov 14 11:42 pnp
-rw-rw-r--.  1 nagios nagios   26 Nov 14 11:41 robots.txt
-rw-rw-r--.  1 nagios nagios  656 Nov 14 11:41 rss-corebanner.php
-rw-rw-r--.  1 nagios nagios 1064 Nov 14 11:41 rss-corefeed.html
-rw-rw-r--.  1 nagios nagios  895 Nov 14 11:41 rss-corefeed.php
-rw-rw-r--.  1 nagios nagios  540 Nov 14 11:41 rss-newsfeed.html
-rw-rw-r--.  1 nagios nagios 1064 Nov 14 11:41 rss-newsfeed.php
-rw-rw-r--.  1 nagios nagios 6009 Nov 14 11:41 side.php
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 10:23 ssi
drwxrwxr-x.  2 nagios nagios 4096 Nov 14 11:41 stylesheets
[root@ms00556 ~]#

So turned on debugging:

Code: Select all

[root@ms00556 ~]# sed -i 's/.*debug_level=.*/debug_level=-1/g' /usr/local/nagios/etc/nagios.cfg
[root@ms00556 ~]# service nagios restart
Running configuration check...done.
Stopping nagios: done.
Starting nagios: done.
[root@ms00556 ~]#

Attached the output. I should point out as per my earlier comments - I may not have even set up performance collection properly (there may be no technical issue here at all other than my lack of experience with this tool). I expected from what little documentation on this I found that the performance counters returned from the scripts should be plottable - I was expecting at least the basics from localhost to be covered?

[lmiltchev]

I may not have even set up performance collection properly...

Performance graphs *should* work "out of the box" on a localhost. There's nothing that needs to be modified on a new install to make them work...

Can you run the following commands and show us the output?

Code: Select all

ls /usr/local/nagios/var/spool/xidpe | wc -l
ls /usr/local/nagios/var/spool/perfdata | wc -l
ls /usr/local/nagios/var/spool/checkresults | wc -l
service crond status
service npcd status
ps -ef | grep perf