memory performance graphs stop generating on some servers

[rnjie]

memory performance graphs do not generate or stop generating, i have gone through this issue in the past and the solution that worked for me was removing the Memory_usage.rrd file for the host from /usr/local/nagios/share/perfdata/ and then it will generate a new performance report after trying all the other solutions like restarting, changing admin passwords and the rest.


i was asked to give performance graph reports for the month of august today but there was no data to display, on gong to this file /usr/local/nagios/share/perfdata/xxserver/Memory_usage.rrd i noticed that it had stop generating graphs since april 4th, i did apply the solution mentioned above but i want to know how this can be avoided or prevent from happening, what the root cause is and all that, i do not want this repeating itself in the future, it mostly occurs on memory graphs and we use snmp and also ncpa to grab reports but its the same case for both. please advise on what to do to prevent this from happening.

[benjaminsmith]

Hi @rnjie,

Let's see if you we can get this worked out for you. Let's make sure the nagios user account has not expired, please post the output of the following command:

Code: Select all

chage -l nagios

Also, send over your system profile so we can review the logs and settings on your server? Thanks.

To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message and then reply to this post to bring it up in the queue.

[rnjie]

please see command output and profile attached

dximonp1 ~]# chage -l nagios
Last password change : Apr 01, 2019
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7


Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

[benjaminsmith]

Hi @rnjie,

Thank you for sending over the system profile, nothing really standing in the profile. However, you may want to change the Log_Level in the process_perfdata.cfg back to 0 as it will just consume more resources ( unless you're actively troubleshooting).

I discussed this with the support team and what will often cause an RRD graph to stop processing is any change to the output of the performance data. For example, if the plugin was returning 3 data points and now it is returning 4, this can cause issues with processing.

Going forward, if you haven't done so already, I would recommend running the Nagios XI Server wizard to monitor the system and make sure all daemons and jobs are running without interruption.

[rnjie]

thank you, i did change the log level to 0, am not sure if that changed anything because i just noticed this morning that one of servers performance graphs stopped generating on sept 4th, how do i get it to start generating without having to delete all the previous data? i have attached the system profile again for your review and the server name is dxacapp1

Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

[benjaminsmith]

Hello,

Most likely the performance data changed during that time. Can you post a screenshot of the historical data. For example,here's a shot of the performance graph.

perf-graph.png

The click on the host or service and go to the Advanced tab and take a screen shot of the Advanced Status Details. I would like compare the historical graph and the incoming performance data.

host-detal.png

As an example, in your current perdata log, there was a change for a host (from 4 sources to 2).

2019-09-10 08:44:20 [10385] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/DXASECP1/HOST.rrd: expected 4 data source readings (got 2) from 1568123048
2019-09-10 08:44:20 [10386] [2] /usr/local/nagios/share/perfdata/DXACAPP1/_usr_Disk_Usage.rrd updated

If you want to keep the historical data you'll have to follow one of guides below to delete or add data source(s) from an existing RRD file. Otherwise, you can delete the RRD file and let it generate a new one.

Data Source Decrease
Nagios XI - How To Delete A Data Source From An RRD File

Data Source Increase
Nagios XI - ICMP and Ping Checks Stopped Graphing