Performance Graphs Not Graphing

[mcwhorts]

Hello,

We submitted a post last week where we had issues with the monitoring engine not working in our NagiosXI instance (https://support.nagios.com/forum/viewto ... 68#p233368) . We were able to get Core updated and then our checks and alerts started triggering as expected.

One lingering thing we’ve noticed is that Performance data is no longer showing up for graphs. We see performance data coming in for services when we click the advanced tab. We’ve done the usual bounce of Nagios services but it hasn’t resolved the issue. We also restarted npcd service specifically but still had the same results.

The System Status shows Performance Grapher as status green.

/usr/local/Nagios/var/npcd.log shows this:

[09-25-2017 15:46:48] NPCD: ERROR: Executed command exits with return code ‘20’
[09-25-2017 15:46:48] NPCD: ERROR: Command line was ‘/usr/local/nagios/libexec/process_perfdata.pl -n -b /var/nagiosramdisk/spool/perfdata//1506375994.perfdata.service’
[09-25-2017 15:47:03] NPCD: ERROR: Executed command exits with return code ‘20’
[09-25-2017 15:47:03] NPCD: ERROR: Command line was ‘/usr/local/nagios/libexec/process_perfdata.pl -n -b /var/nagiosramdisk/spool/perfdata//1506376010.perfdata.host’
[09-25-2017 15:47:03] NPCD: ERROR: Executed command exits with return code ‘20’
[09-25-2017 15:47:03] NPCD: ERROR: Command line was ‘/usr/local/nagios/libexec/process_perfdata.pl -n -b /var/nagiosramdisk/spool/perfdata//1506376010.perfdata.service’
[09-27-2017 16:15:37] NPCD: Caught Termination Signal - Hasta la vista... baby
[09-27-2017 16:15:38] NPCD: npcd Daemon (0.4.14) started with PID=45452
[09-27-2017 16:15:38] NPCD: Please have a look at ‘npcd -V’ to get license information
[09-27-2017 16:15:38] NPCD: HINT: load_threshold is enabled - (‘20.000000’)
[09-28-2017 12:07:27] NPCD: Caught Termination Signal - Hasta la vista... baby
[09-28-2017 12:07:27] NPCD: npcd Daemon (0.4.14) started with PID=29789
[09-28-2017 12:07:27] NPCD: Please have a look at ‘npcd -V’ to get license information
[09-28-2017 12:07:27] NPCD: HINT: load_threshold is enabled - (‘20.000000’)
[09-29-2017 14:16:32] NPCD: Caught Termination Signal - Hasta la vista... baby
[09-29-2017 14:16:32] NPCD: npcd Daemon (0.4.14) started with PID=43122
[09-29-2017 14:16:32] NPCD: Please have a look at ‘npcd -V’ to get license information
[09-29-2017 14:16:32] NPCD: HINT: load_threshold is enabled - (‘20.000000’)
[09-29-2017 14:29:38] NPCD: Caught Termination Signal - Hasta la vista... baby
[09-29-2017 14:29:44] NPCD: npcd Daemon (0.4.14) started with PID=1585
[09-29-2017 14:29:44] NPCD: Please have a look at ‘npcd -V’ to get license information
[09-29-2017 14:29:44] NPCD: HINT: load_threshold is enabled - (‘20.000000’)
[09-29-2017 14:36:16] NPCD: Caught Termination Signal - Hasta la vista... baby
[09-29-2017 14:39:21] NPCD: npcd Daemon (0.4.14) started with PID=17936
[09-29-2017 14:39:21] NPCD: Please have a look at ‘npcd -V’ to get license information
[09-29-2017 14:39:21] NPCD: HINT: load_threshold is enabled - (‘20.000000’)

And /usr/local/nagios/var/perfdata.log shows this (hasn’t been written to in a while):

2017-09-25 15:47:03 [47664] [1] Found Performance Data for EUFRK-FS12 / Uptime (uptime=8493)
2017-09-25 15:47:03 [47664] [1] 118 lines processed
2017-09-25 15:47:03 [47664] [1] /var/nagiosramdisk/spool/perfdata//1506376010.perfdata.service-PID-47664 deleted

Any ideas?

[dwhitfield]

In /usr/local/nagios/etc/pnp/process_perfdata.cfg you'll find the directive LOG_LEVEL. That's probably set to 0. Please set it to 2. Restart npcd. Let it run for a bit and and then set it back.

After that PM me your Profile. You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.

You can also generate a profile manually using the script at /usr/local/nagiosxi/html/includes/components/profile/getprofile.sh

That should generate a profile in /usr/local/nagiosxi/var/components/ which you can get off the server with an application such as FileZilla.

After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.


Also, what's the output of ll /var/nagiosramdisk/spool/. Please put the output in a code block. The "Code" button is the fifth from the left on the post input screen (between Quote and List).

UPDATE: Profile received and shared with techs

[mcwhorts]

So /var/nagiosramdisk/ was filling up so we did a reboot of the vm. Upon bringing things back up, we got some errors about /var/nagiosramdisk/tmp and /var/nagiosramdisk/spool/checkresults missing so nagios was unable to be started. We added those directories back and started nagios up. I sent you our system profile. Also here is what have have in /var/nagiosramdisk/spool/
/usr/local/nagios/var# ll /var/nagiosramdisk/spool/
total 0
drwxrwxr-x 5 nagios nagios 100 Oct 2 11:04 ./
drwxrwxrwt 4 nagios nagios 160 Oct 2 11:24 ../
drwxrwxr-x 2 nagios nagios 40 Oct 2 11:06 checkresults/
drwxrwxr-x 2 nagios nagios 40 Oct 2 10:57 perfdata/
drwxrwxr-x 2 nagios nagios 40 Oct 2 10:57 xidpe/

[dwhitfield]

It looks like page 8 of https://assets.nagios.com/downloads/nag ... giosXI.pdf got skipped:

Code: Select all

define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		/bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

Code: Select all

define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		/bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

Possibly, that was done correctly and then the CCM got reverted to an earlier state?

Please let us know if going back through the ramdisk document does not resolve the issue. If it doesn't, please send a new profile from after you are done going back through.

UPDATE: Profile received and shared with techs

[mcwhorts]

So after going through that document about setting up the ramdisk, we still aren't seeing updates to the graphs. I sent you the newest profile in a PM.

[tgriep]

One thing I found is that the commands that process the performance data have a space in them so go to the Core Config Manager > Commands menu

Edit this command process-host-perfdata-file-bulk and change the command from

Code: Select all

/bin/mv /var/nagiosramdisk/host-perfdata /var/nagiosramdisk/spool/xidpe/ $TIMET$.perfdata.host

to

Code: Select all

/bin/mv /var/nagiosramdisk/host-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.host

Save it and then edit this command process-service-perfdata-file-bulk and change the command from

Code: Select all

/bin/mv /var/nagiosramdisk/service-perfdata /var/nagiosramdisk/spool/xidpe/ $TIMET$.perfdata.service

to

Code: Select all

/bin/mv /var/nagiosramdisk/service-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.service

Save the command and Apply the Config.

Wait for 15 to 20 minutes and see if the graphs start to populate.

[mcwhorts]

It was that space that got us. Thanks for catching that. We can close the ticket

[kyang]

Sounds great! I'll be closing this topic as resolved!

If you have any more questions, feel free to create another thread!

Thank you for using the Nagios Support Forum!