Page 1 of 2
missing perfdata -- again
Posted: Mon Jul 17, 2017 6:30 am
by clauskc
Hi,
We have several hosts where some counters, all related to disk, on performance graphs and capacity planning it stopped showing data after 20 June.
Have had same issue before:
https://support.nagios.com/forum/viewto ... 16&t=43171
Now its a bigger problem since its on many servers the problem is.
Checks are running as they should and data returned, an rrd files are updated.
I checked the perfdata.log and see it has not had a entry in the log since 17-05-2017??
Looked at npcd.log are getting entries but no problems
Looked at nagios.log there are no problems with executing the checks with missing data.
/Claus
Re: missing perfdata -- again
Posted: Mon Jul 17, 2017 11:40 am
by tgriep
Can you edit the following file
Code: Select all
/usr/local/nagios/etc/pnp/process_perfdata.cfg
Change the following line to the following fro enabling debugging
Save the file and restart these processes
Code: Select all
service npcd restart
service nagios restart
Wait for about 30 minutes and email in the log file
Code: Select all
/usr/local/nagios/var/perfdata.log
Thanks
Re: missing perfdata -- again
Posted: Tue Jul 18, 2017 1:17 am
by clauskc
Have PM the requested log to you.
Re: missing perfdata -- again
Posted: Tue Jul 18, 2017 9:23 am
by tgriep
For some reason, the log file did not get attached to the PM, can you send it again or post it here?
Re: missing perfdata -- again
Posted: Wed Jul 19, 2017 12:37 am
by clauskc
Have send it again, did not notice the 1mb limit in the first pm.
Re: missing perfdata -- again
Posted: Wed Jul 19, 2017 8:50 am
by tgriep
I received the log file but I didn't see and errors or issues for when the performance data is updated in to the RRD files.
It looks like it they are getting updated.
I would have to know the names of the host and service that is exhibiting the issue and also the rrd and xml files for the checks.
It might be an issue with viewing them and not updating them.
Run the following as root on the Nagios server.
Then in the GUI, view one of the graphs that is not updating and post the output of the tail command so we can check it for any errors.
Re: missing perfdata -- again
Posted: Thu Jul 20, 2017 2:24 am
by clauskc
The files Changed stamp is 20/6, so does not seem to be updated with data.
Are there a max file size or number of records for the RRD files, can see all the files not being updated are over 4mb?
Host: pfanpbisql01 / service: PFA Free Diskspace
The requested files and logs are attached.
Re: missing perfdata -- again
Posted: Thu Jul 20, 2017 9:15 am
by tgriep
Thanks for the files, I found the reason they are not updating.
2017-07-18 07:59:33 [18446] [2] RRDs::update /usr/local/nagios/share/perfdata/pfanpbisql01/PFA_Free_Diskspace.rrd 1500357545:79:39:11:67:14:99:6:40:19:24
2017-07-18 07:59:33 [18446] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/pfanpbisql01/PFA_Free_Diskspace.rrd: expected 15 data source readings (got 11) from 1500357545
It looks like the check was changed which caused the number of data sources to change. When this happens, the RRD file would have to be deleted so they will be recreated with the correct number of data sources or you can follow this KB article to update the existing files.
https://support.nagios.com/kb/article/n ... phing.html
If that KB article doesn't work, you could try this one.
https://support.nagios.com/kb/article/n ... -file.html
Re: missing perfdata -- again
Posted: Fri Jul 21, 2017 1:08 am
by clauskc
Ok, that make sense why all the same disk counters on some host went bad at the same time.
We are running on RHEL 6.9 now, as I read the articles we should be over this problem with RRDs not updating if we go to RHEL 7.+
Is that correct?
Re: missing perfdata -- again
Posted: Fri Jul 21, 2017 8:38 am
by tgriep
If the expected data sources are incorrect, it doesn't matter if the OS is Centos6 or Centos7.
One thing you need to note, if the Centos6 system is 32 bit and Centos7 system is 64 bit, then all of the RRD files would have to be converted.
See this KB article.
https://support.nagios.com/kb/article/n ... -data.html