Graphs not working

[SavaSC]

A while back we moved our Nagios instances to new servers. When we did that, we removed all of the history to make the transition easier. I'm not sure if we turned any history logging off.

Today, we had an issue and my manager asked what the history of the Host was. Unfortunately, there are no graphs anywhere in Nagios. All of them say "No Data to Display."

I can't find where any logging might be turned on/off. If I go to Home -> Graph Explorer, these work. However, Home -> Performance Graphs just give that no data error.

Please help me figure out where I need to look.

Thanks!

[npolovenko]

Hello, @SavaSC. I encountered an issue recently where Performance Graphs menu would not show anything but the individual graphs were still working ok.
On the Home page under service status detail please click on a graph icon next to the individual service. Is the graph saying No Data as well? Also, please click on the service and go to the "advanced tab" and tell us what output you see next to the Performance Data.
You may also delete entries from the following folder and that will force Nagios to create new RRD files:

Code: Select all

/usr/local/nagios/share/perfdata/

Also, here an article dedicated to troubleshooting graphs in Nagios XI: https://support.nagios.com/kb/article.php?id=9
Let us know if this helps.

[SavaSC]

Hello, @SavaSC. I encountered an issue recently where Performance Graphs menu would not show anything but the individual graphs were still working ok.
On the Home page under service status detail please click on a graph icon next to the individual service. Is the graph saying No Data as well? Also, please click on the service and go to the "advanced tab" and tell us what output you see next to the Performance Data.
You may also delete entries from the following folder and that will force Nagios to create new RRD files:

Code: Select all

/usr/local/nagios/share/perfdata/

Also, here an article dedicated to troubleshooting graphs in Nagios XI.
Let us know if this helps.

@npolovenko. thank you for your reply.

The individual services say No Data as well. Next to Performance Data is a red circle indicating it is disabled. There is nothing in the column for Action[/b/] to enable me to turn it on. I looked on Admin -> Monitoring Engine Status -> Monitoring Engine Process and it says that Performance Data is Enabled. So, I went to our standard Service Template that we use and found that the Process perf data setting had been turned off. I turned that back on, saved, and applied.

Now, on the services, next to Performance Data it shows green / Enabled. However, it still shows "No Data" on any of the Performance Graphs.

No link came through for the article you mentioned.

Thanks again!

[npolovenko]

@SavaSC, I apologize, I added the link to my previous post. I recommend to go through it because you might actually find some useful info in perfdata and npcd log files.
Also, It might take a while for performance graphs to start working after you turned them on. I'd give it 30 minutes.

The key is to look at the Performance data output in the advanced tab in service detail. It will allow us to see whether service checks not returning any perfdata or if the graphing itself is not working.

screenshot-192.168.4.172-2017-11-29-10-33-06-806.png

Did you delete all rrd files in this folder?

Code: Select all

/usr/local/nagios/share/perfdata/

When done please run:

Code: Select all

service npcd restart

And:

Code: Select all

service nagios restart

And give Nagios some time before checking the graphs.

If all above doesn't do it please send us your profile. To send us your system profile. Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and submit it with your next post. Or you can upload it to the cloud storage of your choice and share a link with me in PM.

[SavaSC]

OK, now most services are showing graph data. YEAAAAA!!!

There are some that aren't, however. All of my Windows services Service checks have empty Performance Data fields. The Service Attributes -> Performance Attributes State shows as enabled.

Capture.PNG

Any ideas on what could cause these only to not work?

Incidentally, I have not deleted any of the rrd files. Should I go ahead and do that? If so, should I delete just those rrd files or everything in the /usr/local/nagios/share/perfdata/ folder?

Thanks!

[SavaSC]

Well, looks like I spoke too soon. There are, in fact, multiple Service checks that show Performance Data but say there is no data when you go to graph it. Unless you think there's something else I should do first, I'll try your suggestion of clearing out the rrd files from /usr/local/nagios/share/perfdata/. Do I kill just those files or everything in there?

Thanks!

[lmiltchev]

Don't delete EVERYTHING under the "/usr/local/nagios/share/perfdata"! There are multiple sub-directories there, named after each of your hosts. In each one, there is a rrd and xml file for the host, and all of it's services, provided the check returns perfdata. I would recommend deleting ONLY the rrd and xml files for the "problem" hosts/services (with no graphs displayed).

Note: Keep in mind that even though the RRDs and XML files will get recreated after deletion, it will still take some time before graphs start showing up (approximately 15-20 min).
Keep an eye on the load of the server, and make sure that npcd is indeed running.

Code: Select all

service npcd status

[npolovenko]

@SavaSC, Sorry to hear that. Just the corresponding RRD and XML files need to be removed. (Previous performance history will be lost for these checks). If that is an issue you may try to rename a couple service checks in CCM instead. That should also force XI to create a new RRD files.
Can you upload the following log files:

Code: Select all

tail -25 /usr/local/nagios/var/perfdata.log
tail -25 /usr/local/nagios/var/npcd.log

As far as windows checks, please attempt to run the command locally. You'd go to usr/local/nagios/libexec folder and run ./check_nt with the appropriate arguments. We need to see if the check itself returns any perfdata.
Can you also restart crond just in case:

Code: Select all

service crond restart

Thanks

[SavaSC]

Working on deleting some of the rrd & xml files. Here are the logs

tail -25 /usr/local/nagios/var/perfdata.log
2017-11-28 10:34:09 [23862] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2017-11-28 10:34:09 [23862] [2] Template is pnp-runtime.php
2017-11-28 10:34:09 [23861] [2] RRDs::update /usr/local/nagios/share/perfdata/HOU-RTR-INT/_HOST_.rrd 1511886829:0.335:0:0.675:0.243
2017-11-28 10:34:09 [23862] [2] No Custom Template found for pnp-runtime (/usr/local/nagios/etc/pnp/check_commands/pnp-runtime.cfg)
2017-11-28 10:34:09 [23862] [2] Template is pnp-runtime.php
2017-11-28 10:34:09 [23861] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] data2rrd called
2017-11-28 10:34:09 [23861] [1] 1 lines processed
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_runtime.rrd 1511886834:0.006624
2017-11-28 10:34:09 [23861] [1] /usr/local/nagios/var/spool/perfdata//1511886840.perfdata.host-PID-23861 deleted
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23861] [1] PNP exiting (runtime 0.001461s) ...
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_rows.rrd 1511886834:4
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_errors.rrd 1511886834:3
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_invalid.rrd 1511886834:0
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_skipped.rrd 1511886834:1
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_update.rrd 1511886834:0
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [2] RRDs::update /usr/local/nagios/share/perfdata/.pnp-internal/runtime_create.rrd 1511886834:0
2017-11-28 10:34:09 [23862] [1] RRDs::update ERROR This RRD was created on another architecture
2017-11-28 10:34:09 [23862] [1] PNP exiting (runtime 0.000272s) ...

tail -25 /usr/local/nagios/var/npcd.log
[11-28-2017 10:39:22] NPCD: ThreadCounter 0/5 File is 1511887155.perfdata.host
[11-28-2017 10:39:22] NPCD: Regular File: 1511887155.perfdata.host
[11-28-2017 10:39:22] NPCD: A thread was started on thread_counter = 0
[11-28-2017 10:39:22] NPCD: Processing file 1511887155.perfdata.host with ID 140114702046976 - going to exec /usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1511887155.perfdata.host
[11-28-2017 10:39:22] NPCD: DEBUG: load 0.040000/170.000000
[11-28-2017 10:39:22] NPCD: Processing file '1511887155.perfdata.host'
[11-28-2017 10:39:22] NPCD: ThreadCounter 1/5 File is 1511887155.perfdata.service
[11-28-2017 10:39:22] NPCD: Regular File: 1511887155.perfdata.service
[11-28-2017 10:39:22] NPCD: A thread was started on thread_counter = 1
[11-28-2017 10:39:22] NPCD: Processing file 1511887155.perfdata.service with ID 140114693654272 - going to exec /usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1511887155.perfdata.service
[11-28-2017 10:39:22] NPCD: Have to wait: Filecounter = 3 - thread_counter = 2
[11-28-2017 10:39:22] NPCD: Processing file '1511887155.perfdata.service'
[11-28-2017 10:39:24] NPCD: No more files to process... waiting for 15 seconds
[11-28-2017 10:39:25] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-28-2017 10:39:25] NPCD: npcd Daemon (0.4.14) started with PID=28645
[11-28-2017 10:39:25] NPCD: Please have a look at 'npcd -V' to get license information
[11-28-2017 10:39:25] NPCD: HINT: load_threshold is enabled - ('170.000000')
[11-28-2017 10:39:25] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-28-2017 12:05:07] NPCD: npcd Daemon (0.4.14) started with PID=5296
[11-28-2017 12:05:07] NPCD: Please have a look at 'npcd -V' to get license information
[11-28-2017 12:05:07] NPCD: HINT: load_threshold is enabled - ('170.000000')
[11-28-2017 12:22:18] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-28-2017 12:22:54] NPCD: npcd Daemon (0.4.14) started with PID=1198
[11-28-2017 12:22:54] NPCD: Please have a look at 'npcd -V' to get license information
[11-28-2017 12:22:54] NPCD: HINT: load_threshold is enabled - ('170.000000')

[SavaSC]

While deleting the xml & rrd files I noticed that all of the fully-graphic-functioning Hosts had files with today's date. While the non-functional Hosts had files of varying dates. I deleted all of the xml & rrd files older than 30 days. Now, it appears everything except the Windows service Services are functional. I think I could have deleted everything before today, but I wanted to be safe.

Now I just have to figure out why the Windows service Services aren't collecting Perf Data even though they are set to do so.