Home » Categories » Products » Nagios XI » Troubleshooting » Common Problems

Nagios XI - Performance Graph Problems

Problem Description

Performance graphs not displaying data when their checks are returning true performance data.

The following troubleshooting steps may help you to find the root cause of the issue, and resolve it.

 

 

Editing Files

In many steps of this article you will be required to edit files. This documentation will use the vi text editor. When using the vi
editor:

  • To make changes press i on the keyboard first to enter insert mode
  • Press Esc to exit insert mode
  • When you have finished, save the changes in vi by typing :wq and press Enter

 

 

Count The Amount Of Spooled Files

Nagios spools performance data into small files which get moved around and processed. Sometimes a problem can arise which stops the files from being processed and they begin to spool up. The following commands will count the amount of files in these locations:

ls /usr/local/nagios/var/spool/perfdata/ | wc -l
ls /usr/local/nagios/var/spool/xidpe/ | wc -l

Note: The pipe | symbol is used before the wc -l command.

 

If the commands return a number greater than 20,000 then it is likely you will need to delete the files as the processes will get caught in a loop and not be able to process them. In the logging steps further on in this article, if there are too many files you'll find that nothing gets logged, hence checking the amount of files first can save a lot of time.

To delete a large amount of files in a directory, execute this command:

find /usr/local/nagios/var/spool/perfdata/ -type f -delete

 

Note: This command can take a long time to execute, simply due to the volume of files. You can open another SSH session and observe the count of files in the folder with this command:

watch 'ls /usr/local/nagios/var/spool/perfdata/ | wc -l'

 

When the files are finally deleted, wait approximately thirty minutes to see if performance graphs start to work. If they do not, proceed with the next troubleshooting steps in this guide.

 

 

Increase Performance Data Logging Verbosity

Edit the following file from an SSH session:

/usr/local/nagios/etc/pnp/process_perfdata.cfg

Change:

LOG_LEVEL = 0

To:

LOG_LEVEL = 2

The process_perfdata.pl script should now log all errors and debug information to the file /usr/local/nagios/var/perfdata.log which can be watched using this command:

tail -f /usr/local/nagios/var/perfdata.log

 

Look for any errors, incorrect exit codes, and/or timeouts.

Remember to return this value to it's default setting when troubleshooting is completed.

One of the top problem causes you can find in this log is a typical timeout error, to resolve that temporarily, you can increase the performance data processor's timeout range by editing:

/usr/local/nagios/etc/pnp/process_perfdata.cfg

Change:

TIMEOUT = 5

To:

TIMEOUT = 20

Or a greater number if this continues to be a issue.

 

 

Increase NPCD Logging Verbosity

NPCD is a bulk processing tool which reaps and processes your performance data once received. To increase it's logging verbosity edit the following file in an SSH session:

/usr/local/nagios/etc/pnp/npcd.cfg

Change:

log_level = 0

To:

log_level = -1

 

Yes, that -1 is not a typing mistake, it will increase the verbosity. Now, restart the NPCD service using one of the commands below:

RHEL 6 | CentOS 6 | Oracle Linux 6 | Ubuntu 14

service npcd restart

 

RHEL 7 | CentOS 7 | Oracle Linux 7 | Debian | Ubuntu 16/18

systemctl restart npcd.service

 

Remember to return this value to it's default setting when troubleshooting is completed.

NPCD should now log all errors and debug information to the file /usr/local/nagios/var/npcd.log which can be watched using this command:

tail -f /usr/local/nagios/var/npcd.log

 

One of the top problem causes to look for in the above log is lines indicating that you are hitting a load threshold, this is common if you are either receiving too much data for NPCD to keep up with the current system's load, or that it is trying to crunch through stacked up performance data.

You can increase this threshold by editing the following file:

/usr/local/nagios/etc/pnp/npcd.cfg

Change:

load_threshold = 10.0

To a value greater than your system's current load. Use this with caution however, as the NPCD process will eat as much load as you give it, so watch your resources!

 

 

Check Nagios User Account

In some situations the nagios user account can expire causing issues like this to occur. Run this command to see if the nagios user is expired:

chage -l nagios

 

If it is, run this command to enable the expired nagios user:

chage -I -1 -m 0 -M 99999 -E -1 nagios

 

 

Final Thoughts

For any support related questions please visit the Nagios Support Forums at:

http://support.nagios.com/forum/

1 (1)
Article Rating (1 Votes)
Rate this article
  • Icon PDFExport to PDF
  • Icon MS-WordExport to MS Word
Attachments Attachments
There are no attachments for this article.
Related Articles RSS Feed
Nagios XI - HTTP 500 Error / White Screen After Login
Viewed 742 times since Mon, Feb 29, 2016
Nagios XI - Empty Screen for Wizard, Component, Dashlet
Viewed 843 times since Wed, Jan 27, 2016
Nagios XI - Resolving "Cannot connect to database" Error - Core Config Manager
Viewed 970 times since Mon, Jan 25, 2016
Nagios XI - Hosts Pending/Gray
Viewed 2064 times since Tue, Jan 6, 2015
Nagios XI - Oracle Services Critical After Nagios XI Upgrade
Viewed 721 times since Wed, Jan 27, 2016
Nagios XI - Upgrade errors - root.crontab.orig: cannot overwrite existing file
Viewed 640 times since Tue, Jan 26, 2016
Nagios Core - Failed to register iobroker
Viewed 703 times since Wed, Sep 20, 2017
ERROR: Please add the ’Optional’ channel to your Red Hat systems subscriptions
Viewed 4197 times since Tue, Jan 26, 2016
Nagios XI - SNMPTT Service generates Cannot find module errors
Viewed 789 times since Sun, Apr 9, 2017
Nagios XI - MK Livestatus Problems With Mod-Gearman
Viewed 777 times since Thu, Feb 25, 2016