Nagios XI Performance Graph No Longer Showing Data

[hamija2017]

Nagios XI is no longer showing performance graph data.
Following the following topic: https://support.nagios.com/kb/article.php?id=9.

I checked the log file using the following command. tail -f /usr/local/nagios/var/npcd.log

I did not see any errors.
This is part of the log file details:

Code: Select all

[root@nagiosxi pnp]# tail -f /usr/local/nagios/var/perfdata.log
2017-05-25 09:33:04 [101361] [2] RRDs Perl Modules are not installed. Falling back to rrdtool system call.
2017-05-25 09:33:04 [101361] [2] /usr/bin/rrdtool update --daemon=unix:/var/rrdtool/rrdcached/rrdcached.sock /usr/local/nagios/share/perfdata/Merrick/Check_All_Drives.rrd 1495726378:49.26982:82:2.79339:13
2017-05-25 09:33:04 [101362] [1] rrdtool update returns 256
2017-05-25 09:33:04 [101362] [1] 34 lines processed
2017-05-25 09:33:04 [101362] [1] /var/nagiosramdisk/spool/perfdata//1495726363.perfdata.service-PID-101362 deleted
2017-05-25 09:33:04 [101362] [1] PNP exiting (runtime 0.123073s) ...
2017-05-25 09:33:04 [101361] [1] rrdtool update returns 256
2017-05-25 09:33:04 [101361] [1] 35 lines processed
2017-05-25 09:33:04 [101361] [1] /var/nagiosramdisk/spool/perfdata//1495726378.perfdata.service-PID-101361 deleted
2017-05-25 09:33:04 [101361] [1] PNP exiting (runtime 0.124477s) ...
2017-05-25 09:33:34 [101612] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2017-05-25 09:33:34 [101611] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2017-05-25 09:33:34 [101612] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2017-05-25 09:33:34 [101612] [2] searching for /var/nagiosramdisk/spool/perfdata//1495726408.perfdata.service
2017-05-25 09:33:34 [101611] [1] process_perfdata.pl-0.6.11 starting in BULK Mode called by NPCD
2017-05-25 09:33:34 [101612] [2] renaming /var/nagiosramdisk/spool/perfdata//1495726408.perfdata.service to /var/nagiosramdisk/spool/perfdata//1495726408.perfdata.service-PID-101612 for bulk update
2017-05-25 09:33:34 [101611] [2] searching for /var/nagiosramdisk/spool/perfdata//1495726393.perfdata.service
2017-05-25 09:33:34 [101612] [2] reading /var/nagiosramdisk/spool/perfdata//1495726408.perfdata.service-PID-101612 for bulk update
2017-05-25 09:33:34 [101611] [2] renaming /var/nagiosramdisk/spool/perfdata//1495726393.perfdata.service to /var/nagiosramdisk/spool/perfdata//1495726393.perfdata.service-PID-101611 for bulk update
2017-05-25 09:33:34 [101611] [2] reading /var/nagiosramdisk/spool/perfdata//1495726393.perfdata.service-PID-101611 for bulk update
2017-05-25 09:33:34 [101612] [2] Processing Line 1
2017-05-25 09:33:34 [101611] [2] Processing Line 1
2017-05-25 09:33:34 [101612] [2] Datatype set to 'SERVICEPERFDATA'
2017-05-25 09:33:34 [101612] [1] Found Performance Data for XXXXXXXX / Exchange_Server_Backend_Services ('Microsoft Exchange Active Directory Topology'=4;0;4 'Microsoft Exchange Information Store'=4;0;4 'Microsoft Exchange Mail Submission'=4;0;4 'Microsoft Exchange Mailbox Assistants'=4;0;4 'Microsoft Exchange Replication'=4;0;4 'Microsoft Exchange RPC Client Access'=4;0;4 'Microsoft Exchange Search Indexer'=4;0;4 'Microsoft Exchange Service Host'=4;0;4 'Microsoft Exchange System Attendant'=4;0;4 'Microsoft Exchange Throttling'=4;0;4 'Microsoft Exchange Transport Log Search'=4;0;4)
2017-05-25 09:33:34 [101611] [2] Datatype set to 'SERVICEPERFDATA'
2017-05-25 09:33:34 [101611] [1] Found Performance Data for xxxxxxxx.extraspace.com / Memory_for_VMHost (mem_usagemb=119955.19MB;; mem_usage=30.51%;; mem_overhead=0.00MB;; mem_swap=0.00MB;;)

Checked the nagios account age, it is not expired.
The last time the Nagios XI showed any data was on April 26, 2017. This happened to be the same time the software was upgraded to version: 5.4.4

Attached a screenshot of what the performance graph is showing.

[dwhitfield]

When you upgraded, did you also migrate, or was it an update on the same system?

Can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.

After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.

[hamija2017]

The upgrade was performed under the "Admin -> System Information -> Check for Updates Now button". As I know, this is how it was installed.

I am currently going through the profile that was zipped and removing some sensitive info from it and will then PM you the file.

[dwhitfield]

Awesome.

Please be aware that Monday is a US Holiday and we close early on Fridays, so the sooner your can get me the profile the sooner we'll be able to get things resolved.

[hamija2017]

dwhitefield,

I have sent you a PM with the Nagios Profile information. Please review and let me know if you need any additional information.

Thank you,

hamija2017

[tgriep]

Your Nagios server is setup to use rrdcached for caching the performance data and that seems to be causing the problem.
Check the /var/log/messages log to see if if you find any errors.
Also, run the following as root and post the output.

Code: Select all

cat /etc/sysconfig/rrdcached
ls -la /tmp/rrd.journal*

[hamija2017]

I ran the following as root, this is the output.

[root@nagiosxi ~]# cat /etc/sysconfig/rrdcached
OPTIONS="-s nagios -m 0660 -l unix:/var/rrdtool/rrdcached/rrdcached.sock -F -w 900 -z 90 -j /tmp/ -b /var/rrdtool/rrdcached -P FLUSH,PENDING"
RRDC_USER=nagios
[root@nagiosxi ~]# ls -la /tmp/rrd.journal*
-rw-r--r-- 1 nagios users 0 May 26 11:54 /tmp/rrd.journal.1495821280.571867
-rw-r--r-- 1 nagios users 0 May 26 12:54 /tmp/rrd.journal.1495824880.571853

[tgriep]

Can you edit this file

Code: Select all

/usr/local/nagios/etc/pnp/process_perfdata.cfg

And comment out the following line at the bottom to the file

Code: Select all

RRD_DAEMON_OPTS = unix:/var/rrdtool/rrdcached/rrdcached.sock

Then restart the npcd daemon by running

Code: Select all

service npcd restart

Wait for 15 minutes and see if the graphs start to populate.

[hamija2017]

I commented out the following line in the file: /usr/local/nagios/etc/pnp/process_perfdata.cfg
#RRD_DAEMON_OPTS = unix:/var/rrdtool/rrdcached/rrdcached.sock

Restarted the NPCD service using the command indicated. Waiting to see if the data will start populating.

[hamija2017]

I can now confirm the performance graph data is now starting to populate data again.