Home » Categories » Products » Nagios XI » Troubleshooting » Common Problems

Nagios XI - Performance Graph Problems

Problem Description

Performance graphs not displaying data when their checks are returning true performance data.

The following troubleshooting steps may help you to find the root cause of the issue, and resolve it.

 

 

Editing Files

In many steps of this article you will be required to edit files. This documentation will use the vi text editor. When using the vi
editor:

  • To make changes press i on the keyboard first to enter insert mode
  • Press Esc to exit insert mode
  • When you have finished, save the changes in vi by typing :wq and press Enter

 

 

Count The Amount Of Spooled Files

Nagios spools performance data into small files which get moved around and processed. Sometimes a problem can arise which stops the files from being processed and they begin to spool up. The following commands will count the amount of files in these locations:

ls /usr/local/nagios/var/spool/perfdata/ | wc -l
ls /usr/local/nagios/var/spool/xidpe/ | wc -l

Note: The pipe | symbol is used before the wc -l command.

 

If the commands return a number greater than 20,000 then it is likely you will need to delete the files as the processes will get caught in a loop and not be able to process them. In the logging steps further on in this article, if there are too many files you'll find that nothing gets logged, hence checking the amount of files first can save a lot of time.

To delete a large amount of files in a directory, execute this command:

find /usr/local/nagios/var/spool/perfdata/ -type f -delete

 

Note: This command can take a long time to execute, simply due to the volume of files. You can open another SSH session and observe the count of files in the folder with this command:

watch 'ls /usr/local/nagios/var/spool/perfdata/ | wc -l'

 

When the files are finally deleted, wait approximately thirty minutes to see if performance graphs start to work. If they do not, proceed with the next troubleshooting steps in this guide.

 

 

Increase Performance Data Logging Verbosity

Edit the following file from an SSH session:

/usr/local/nagios/etc/pnp/process_perfdata.cfg

Change:

LOG_LEVEL = 0

To:

LOG_LEVEL = 2

The process_perfdata.pl script should now log all errors and debug information to the file /usr/local/nagios/var/perfdata.log which can be watched using this command:

tail -f /usr/local/nagios/var/perfdata.log

 

Look for any errors, incorrect exit codes, and/or timeouts.

Remember to return this value to it's default setting when troubleshooting is completed.

One of the top problem causes you can find in this log is a typical timeout error, to resolve that temporarily, you can increase the performance data processor's timeout range by editing:

/usr/local/nagios/etc/pnp/process_perfdata.cfg

Change:

TIMEOUT = 5

To:

TIMEOUT = 20

Or a greater number if this continues to be a issue.

 

 

Increase NPCD Logging Verbosity

NPCD is a bulk processing tool which reaps and processes your performance data once received. To increase it's logging verbosity edit the following file in an SSH session:

/usr/local/nagios/etc/pnp/npcd.cfg

Change:

log_level = 0

To:

log_level = -1

 

Yes, that -1 is not a typing mistake, it will increase the verbosity. Now, restart the NPCD service using one of the commands below:

RHEL 6 | CentOS 6 | Oracle Linux 6 | Ubuntu 14

service npcd restart

 

RHEL 7 | CentOS 7 | Oracle Linux 7 | Debian | Ubuntu 16/18

systemctl restart npcd.service

 

Remember to return this value to it's default setting when troubleshooting is completed.

NPCD should now log all errors and debug information to the file /usr/local/nagios/var/npcd.log which can be watched using this command:

tail -f /usr/local/nagios/var/npcd.log

 

One of the top problem causes to look for in the above log is lines indicating that you are hitting a load threshold, this is common if you are either receiving too much data for NPCD to keep up with the current system's load, or that it is trying to crunch through stacked up performance data.

You can increase this threshold by editing the following file:

/usr/local/nagios/etc/pnp/npcd.cfg

Change:

load_threshold = 10.0

To a value greater than your system's current load. Use this with caution however, as the NPCD process will eat as much load as you give it, so watch your resources!

 

 

Check Nagios User Account

In some situations the nagios user account can expire causing issues like this to occur. Run this command to see if the nagios user is expired:

chage -l nagios

 

If it is, run this command to enable the expired nagios user:

chage -I -1 -m 0 -M 99999 -E -1 nagios

 

 

Final Thoughts

For any support related questions please visit the Nagios Support Forums at:

http://support.nagios.com/forum/

1 (1)
Article Rating (1 Votes)
Rate this article
  • Icon PDFExport to PDF
  • Icon MS-WordExport to MS Word
Attachments Attachments
There are no attachments for this article.
Related Articles RSS Feed
Nagios XI - Debugging Bandwidth Performance Graphs
Viewed 1458 times since Tue, Jan 27, 2015
Nagios XI - mysql_error out of range value for column
Viewed 563 times since Tue, Aug 14, 2018
Nagios Core - Nagios did not exit in a timely manner
Viewed 776 times since Wed, Jan 27, 2016
Nagios XI - Hosts Pending/Gray
Viewed 1767 times since Tue, Jan 6, 2015
Nagios XI - How To Delete A Data Source From An RRD File
Viewed 1179 times since Wed, Apr 27, 2016
Nagios XI - Defining Global Environment Variables
Viewed 1213 times since Thu, Mar 17, 2016
Nagios XI - Migrate Performance Data
Viewed 1234 times since Tue, Jan 26, 2016
Nagios XI - Crashed Database Tables
Viewed 3910 times since Tue, Jan 27, 2015
CCM says unapplied changes exist, but none listed
Viewed 493 times since Mon, Feb 27, 2017
Pages Not Displaying Correctly
Viewed 696 times since Mon, Jan 25, 2016