Page 1 of 1
No performance graphs on installation. Data is being sent.
Posted: Wed Jan 22, 2014 9:09 am
by dflick
NagiosXI 2012R2.8 installed via VMware image
/usr/local/nagios/var/perfdata.log does not exist.
If I look at the host on the advanced tab I see data collected:
Performance Data: rta=8.995ms;3000.000;5000.000;0; pl=0%;80;100;;
All of my hosts have data collected but none have graphs. I have hit a dead end and don't know where to go next.
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 10:35 am
by slansing
What is the output of the following:
Code: Select all
tail -50 /usr/local/nagios/var/npcd.log
Code: Select all
ls -1 /usr/local/nagios/var/spool/checkresults/ | wc -l
ls -1 /usr/local/nagios/var/spool/perfdata/ | wc -l
ls -1 /usr/local/nagios/var/spool/xidpe | wc -l
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 10:42 am
by dflick
[root@caecd-buc-xi ~]# tail -50 /usr/local/nagios/var/npcd.log
[01-03-2014 22:25:56] NPCD: npcd Daemon (0.4.14) started with PID=1495
[01-03-2014 22:25:56] NPCD: Please have a look at 'npcd -V' to get license information
[01-03-2014 22:25:56] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-03-2014 22:40:15] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-03-2014 23:03:54] NPCD: npcd Daemon (0.4.14) started with PID=1494
[01-03-2014 23:03:54] NPCD: Please have a look at 'npcd -V' to get license information
[01-03-2014 23:03:54] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-03-2014 23:14:28] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-03-2014 23:14:55] NPCD: npcd Daemon (0.4.14) started with PID=1480
[01-03-2014 23:14:55] NPCD: Please have a look at 'npcd -V' to get license information
[01-03-2014 23:14:55] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-04-2014 13:20:40] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-04-2014 13:38:12] NPCD: npcd Daemon (0.4.14) started with PID=1481
[01-04-2014 13:38:12] NPCD: Please have a look at 'npcd -V' to get license information
[01-04-2014 13:38:12] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-04-2014 13:55:47] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-04-2014 13:57:48] NPCD: npcd Daemon (0.4.14) started with PID=1535
[01-04-2014 13:57:48] NPCD: Please have a look at 'npcd -V' to get license information
[01-04-2014 13:57:48] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-04-2014 14:02:53] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-04-2014 14:03:14] NPCD: npcd Daemon (0.4.14) started with PID=1398
[01-04-2014 14:03:14] NPCD: Please have a look at 'npcd -V' to get license information
[01-04-2014 14:03:14] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-04-2014 14:17:54] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-04-2014 14:18:39] NPCD: npcd Daemon (0.4.14) started with PID=1495
[01-04-2014 14:18:39] NPCD: Please have a look at 'npcd -V' to get license information
[01-04-2014 14:18:39] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-04-2014 14:23:13] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-09-2014 12:30:48] NPCD: npcd Daemon (0.4.14) started with PID=1494
[01-09-2014 12:30:48] NPCD: Please have a look at 'npcd -V' to get license information
[01-09-2014 12:30:48] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-09-2014 12:42:52] NPCD: npcd Daemon (0.4.14) started with PID=1392
[01-09-2014 12:42:52] NPCD: Please have a look at 'npcd -V' to get license information
[01-09-2014 12:42:52] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-13-2014 21:03:38] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-13-2014 21:04:07] NPCD: npcd Daemon (0.4.14) started with PID=1467
[01-13-2014 21:04:07] NPCD: Please have a look at 'npcd -V' to get license information
[01-13-2014 21:04:07] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-13-2014 21:22:13] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-13-2014 21:22:37] NPCD: npcd Daemon (0.4.14) started with PID=1470
[01-13-2014 21:22:37] NPCD: Please have a look at 'npcd -V' to get license information
[01-13-2014 21:22:37] NPCD: HINT: load_threshold is enabled - ('10.000000')
[01-16-2014 21:50:21] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-16-2014 21:50:21] NPCD: npcd Daemon (0.4.14) started with PID=25567
[01-16-2014 21:50:21] NPCD: Please have a look at 'npcd -V' to get license information
[01-16-2014 21:50:21] NPCD: HINT: load_threshold is enabled - ('20.000000')
[01-16-2014 21:55:34] NPCD: Caught Termination Signal - Hasta la vista... baby
[01-16-2014 21:56:00] NPCD: npcd Daemon (0.4.14) started with PID=1582
[01-16-2014 21:56:00] NPCD: Please have a look at 'npcd -V' to get license information
[01-16-2014 21:56:00] NPCD: HINT: load_threshold is enabled - ('20.000000')
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/checkresults/ | wc -l
6
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/perfdata/ | wc -l
0
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/xidpe | wc -l
148724
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 11:00 am
by slansing
What is the current load on your XI server? You can view this with:
It looks like you might be hitting the NPCD max load threshold, or your were doing so recently. To increase this threshold edit:
Code: Select all
/usr/local/nagios/etc/pnp/npcd.cfg
And change:
To:
Save, and quit the file, then stop npcd:
Now, this next step will require that you remove the trapped performance data from the XIDPE directory, once you do this you won't be able to recover it, but it is really of no use to you now anyways as it cannot be processed by Nagios XI and it is clogging the pipe.:
Code: Select all
find /usr/local/nagios/var/spool/xidpe/ . -type d -print -delete
Start NPCD again:
And after about 10 or so minutes run these commands again and share the output, you should start seeing graphs if there are not a large number of files piling up "100+ usually.":
Code: Select all
ls -1 /usr/local/nagios/var/spool/checkresults/ | wc -l
ls -1 /usr/local/nagios/var/spool/perfdata/ | wc -l
ls -1 /usr/local/nagios/var/spool/xidpe | wc -l
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 11:17 am
by dflick
Still nothing but I got an error on that find command.
[root@caecd-buc-xi ~]# top
top - 10:04:50 up 5 days, 12:09, 2 users, load average: 0.44, 0.79, 1.12
Tasks: 134 total, 2 running, 132 sleeping, 0 stopped, 0 zombie
Cpu(s): 47.3%us, 3.3%sy, 0.0%ni, 47.3%id, 1.7%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 3923756k total, 1436100k used, 2487656k free, 206248k buffers
Swap: 2064376k total, 0k used, 2064376k free, 492712k cached
I already set the load threshold to 20 per another post and it unfortunately did not help. I have not seen any "threshold reached" messages in the logs.
[root@caecd-buc-xi ~]# find /usr/local/nagios/var/spool/xidpe . -type d -print -delete
/usr/local/nagios/var/spool/xidpe
find: cannot delete `/usr/local/nagios/var/spool/xidpe': Directory not empty
./.pki/nssdb
./.pki
./scripts
find: cannot delete `./scripts': Directory not empty
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/checkresults/ | wc -l
16
You have new mail in /var/spool/mail/root
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/perfdata/ | wc -l
0
[root@caecd-buc-xi ~]# ls -1 /usr/local/nagios/var/spool/xidpe | wc -l
148994
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 12:12 pm
by slansing
Please run:
Code: Select all
ls /usr/local/nagios/var/spool/xidpe | wc -l
service npcd stop
cd /usr/local/nagios/var/spool/xidpe
find . -type f -delete
service npcd start
After a few minutes:
Code: Select all
ls /usr/local/nagios/var/spool/xidpe | wc -l
Also, is cron running?
If it is not:
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 12:17 pm
by dflick
Completed your request. Here is the needed output.
[root@caecd-buc-xi ~]# ls /usr/local/nagios/var/spool/xidpe | wc -l
149458
[root@caecd-buc-xi xidpe]# service crond status
crond (pid 1571) is running...
[root@caecd-buc-xi xidpe]# ls /usr/local/nagios/var/spool/xidpe | wc -l
2
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 12:18 pm
by dflick
That last set of commands seems to have fixed it! I am getting the graphs now. I will wait to confirm that data is coming into those. Can you tell me what that last set of commands did to fix it?
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 12:31 pm
by slansing
It looks like at some point you must have hit the load threshold on the system, this likely caused the scraped performance data to pile up in XIDPE. The reason I had you check cron is because cron is responsible for moving the files from the xidpe directory to the perfdata directory where perfdataproc processes the data, and dumps it to the RRD's that your graphs on the front end pull their data from.
Re: No performance graphs on installation. Data is being se
Posted: Wed Jan 22, 2014 12:35 pm
by dflick
Thanks for the info and the help to resolve this issue!