Page 1 of 1
Perfdata Stop Working!
Posted: Mon Aug 24, 2015 7:36 am
by lb2cons
Yesterday my Perfdata stop working.
When I checked some graphs I've discovered every graph stopped plotting after yesterday (2015-23-08) 05:00 PM. So I simple restarted the Performance Grapher and everything worked again. Another interesting fact is that every XI System Component was with status OK.
Screen Shot 2015-08-24 at 09.29.30.png
Screen Shot 2015-08-24 at 09.34.21.png
My question is how to further investigate what happen. What logs should I check?
Performance graphs working it's crucial for me because I've send graphs reports for customers and I don't want to find that my Perf. isn't working after 15 days or more.
My second question is how can I prevent this?
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 9:28 am
by tgriep
We would have to look at these logs to determine the failure. Can you run the following and post back the results?
Code: Select all
tail -50 /usr/local/nagios/var/perfdata.log
tail -50 /usr/local/nagios/var/npcd.log
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 10:38 am
by lb2cons
I've attached the log from the time performance data stopped working.
Note that on both logs there's a gap of information from day 23 17:27 to day 24 08:56 that is the time I've restarted monitoring engine.
perfdata.log
npcd.log
The log doesn't tell nothing for me. Please report if you need more information.
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 11:19 am
by jolson
One thing to take a look at is the load of your Nagios XI server versus the threshold that you have set up for performance data processing:
Load:
Threshold:
Code: Select all
grep threshold /usr/local/nagios/etc/pnp/npcd.cfg
By default, performance data will not be processed if your load is sustained at greater than 10 percent. Is it possible that your load was higher than the threshold during the period that your perfdata stopped processing?
You will want to take a look at this FAQ entry:
https://support.nagios.com/wiki/index.p ... h_Problems
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 11:21 am
by ssax
If you look at this FAQ entry here, what is your TIMEOUT and load_threshold set to?
https://support.nagios.com/wiki/index.p ... ta_Timeout
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 11:56 am
by lb2cons
Jolson, the load was above "3" and the threshold was 10.
I've read the docs that both of you sent and updated load_threshold and timeout just in case.
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 2:00 pm
by jolson
Are your perfdata graphs working now? It will be worth watching your perfdata.log for any additional timeouts that might occur.
You will want to see how many files have piled up in the xidpe directory:
Code: Select all
ls -l /usr/local/nagios/var/spool/xidpe | wc -l
Is the number of files in that directory decreasing?
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 3:39 pm
by lb2cons
Jolson, yes, it's decreasing!
Re: Perfdata Stop Working!
Posted: Mon Aug 24, 2015 3:41 pm
by jolson
That's good news! You should see your performance graphs populating appropriately at this point. Let us know if you experience any other difficulties!