No Performance Data for this host

[afdevbank]

Hi,

My Nagiosxi setup is running fine and churning out perfdata but I don't see the graphs. When I trying checking the performance graphs I get a error saying there is no performance data for the host. All hosts have no graphs. All hosts have 'process performance data' activated.

I've seen similar issues on the forums but none of the suggested fixes worked.

Attached is a file with some more info.

[tmcdonald]

I've seen similar issues on the forums but none of the suggested fixes worked.

Attached is a file with some more info.

What fixes specifically have you tried? It'll be hard for us to help if we don't know what you have and have not done.

Also, no file was attached.

[afdevbank]

Hi tmcdonald,

Thanks for your response. I restarted npcd daemon. I increased the timeout in /usr/local/nagios/etc/pnp/process_perfdata.cfg from 5 to 20 and the threshold in 10.0 to 30.0 and restarted npcd daemon. Both did not resolve the issue. Since this is the 2014R1 release I did not want to tamper with the host creation wizard as suggested in one 2011 post (of course the bug would have been fixed). I'm now going to empty the /usr/local/nagios/var/spool/perdata directory as it's really big. I'm surprised at the absence of the attachment but it appear the reason is the extension (pdf) which isn't allowed. Trying again.

Kind regards,

Osay

config.png

contentsperfdata.png

empty graphs.png

[slansing]

It is likely that there are so many files backed up that it is maxing out your load as you redefined it, and continually trying to catch up instead of processing new data. Lets do some looking, please share the output of:

Code: Select all

ll /usr/local/nagios/var/spool/pefdata | wc -l
ll /usr/local/nagios/var/spool/checkresults | wc -l
ll /usr/local/nagios/var/spool/xidpe | wc -l

Code: Select all

tail -30 /usr/local/nagios/var/npcd.log

tail -30 /usr/local/nagios/var/perfdata.log

Code: Select all

TOP

[afdevbank]

Hi slansing,

Thanks for the response and sorry for coming back late with a reponse to your request (been away a bit). It appears the npcd daemon is dying on startup. See below.

Code: Select all

[07-12-2013 10:36:48] NPCD: npcd Daemon (0.4.14) started with PID=16723
[07-12-2013 10:36:48] NPCD: Please have a look at 'npcd -V' to get license information
[07-12-2013 10:36:48] NPCD: HINT: load_threshold is enabled - ('10.000000')
[07-26-2013 12:19:55] NPCD: Caught Termination Signal - Hasta la vista... baby
[07-26-2013 12:21:15] NPCD: npcd Daemon (0.4.14) started with PID=1258
[07-26-2013 12:21:15] NPCD: Please have a look at 'npcd -V' to get license information
[07-26-2013 12:21:15] NPCD: HINT: load_threshold is enabled - ('10.000000')
[08-01-2013 07:29:39] NPCD: ERROR: Executed command exits with return code '7'
[08-01-2013 07:29:39] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1375334954.perfdata.service'
[10-23-2013 10:20:58] NPCD: Caught Termination Signal - Hasta la vista... baby

Code: Select all

[root@nagios-sarc var]# tail perfdata.log
2013-08-01 07:29:38 [3844] [0] *** TIMEOUT: Timeout after 5 secs. ***
2013-08-01 07:29:39 [3844] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2013-08-01 07:29:39 [3844] [0] *** TIMEOUT: Please check your npcd.cfg
2013-08-01 07:29:39 [3844] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1375334954.perfdata.service-PID-3844 deleted
2013-08-01 07:29:39 [3844] [0] *** Timeout while processing Host: "afdbkms.afdb.local" Service: "Ping"
2013-08-01 07:29:39 [3844] [0] *** process_perfdata.pl terminated on signal ALRM

Code: Select all

[root@nagios-sarc spool]# ll perfdata/ | wc -l
1813

I emptied this directory and it quickly built up again.

Code: Select all

[root@nagios-sarc spool]# ll checkresults/ | wc -l
1

Code: Select all

[root@nagios-sarc spool]# ll xidpe/ | wc -l
1

Thanks once again for your help.

Osay

[scottwilkerson]

These logs from NPCD are from 7 months ago...

Lets restart NPCD

Code: Select all

service npcd restart

Also, what is the current load on the system (1, 5, 15 min)?

[afdevbank]

Hi,

I've restarted the npcd daemon but it does not produce a log file. I remove the log files listed above and restarted the daemon but no new log files are created.

The status shows it's running

Code: Select all

[root@nagios-sarc share]# service npcd status
npcd (pid 19931) is running...

The load is 0.02, 0.05, 0.06.

[afdevbank]

Hi,

Found something interesting.

I checked in /usr/local/nagiosxi/var and found perfdataproc.log, the contents of which are below.

Code: Select all

[root@nagios-sarc var]# tail perfdataproc.log
Outbound data DISABLED Wed, 28 May 2014 17:38:02 +0200
mv: cannot stat `/usr/local/nagios/var/spool/xidpe/*': No such file or directory
mv: cannot stat `/usr/local/nagios/var/spool/xidpe/*': No such file or directory

[lmiltchev]

Have you tried utilizing a RAM disk by following this document? If you did, are you sure you followed all of the steps from the doc?

What is the output of the following command?

Code: Select all

locate xidpe

Can you post the "nagios.cfg", "npcd.cfg", and "process_perfdata.cfg" files?

[afdevbank]

Hi Imiltchev,

Thanks for your response. Here's the location of xidpe

Code: Select all

/usr/local/nagios/var/spool/xidpe

Also here are the cfg files requested.

nagios.cfg

Code: Select all

# MODIFIED
admin_email=root@localhost
admin_pager=root@localhost
translate_passive_host_checks=1
log_event_handlers=0
use_large_installation_tweaks=1
enable_environment_macros=0


# NDOUtils module
broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg


# PNP settings - bulk mode with NCPD
process_performance_data=1
# service performance data
service_perfdata_file=/usr/local/nagios/var/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATE
TYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tSERVICEOUTPUT::$SERVICEOUTPUT$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=process-service-perfdata-file-bulk
# host performance data
host_perfdata_file=/usr/local/nagios/var/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tHOSTOUTPUT::$HOSTOUTPUT
$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=process-host-perfdata-file-bulk


# OBJECTS - UNMODIFIED
#cfg_file=/usr/local/nagios/etc/objects/commands.cfg
#cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
#cfg_file=/usr/local/nagios/etc/objects/localhost.cfg
#cfg_file=/usr/local/nagios/etc/objects/templates.cfg
#cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg


# STATIC OBJECT DEFINITIONS (THESE DON'T GET EXPORTED/IMPORTED BY NAGIOSQL)
cfg_dir=/usr/local/nagios/etc/static

# OBJECTS EXPORTED FROM NAGIOSQL
cfg_file=/usr/local/nagios/etc/contacttemplates.cfg
cfg_file=/usr/local/nagios/etc/contactgroups.cfg
cfg_file=/usr/local/nagios/etc/contacts.cfg
cfg_file=/usr/local/nagios/etc/timeperiods.cfg
cfg_file=/usr/local/nagios/etc/commands.cfg
cfg_file=/usr/local/nagios/etc/hostgroups.cfg
cfg_file=/usr/local/nagios/etc/servicegroups.cfg
cfg_file=/usr/local/nagios/etc/hosttemplates.cfg
cfg_file=/usr/local/nagios/etc/servicetemplates.cfg
cfg_file=/usr/local/nagios/etc/servicedependencies.cfg
cfg_file=/usr/local/nagios/etc/serviceescalations.cfg
cfg_file=/usr/local/nagios/etc/hostdependencies.cfg
cfg_file=/usr/local/nagios/etc/hostescalations.cfg
cfg_file=/usr/local/nagios/etc/hostextinfo.cfg
cfg_file=/usr/local/nagios/etc/serviceextinfo.cfg
cfg_dir=/usr/local/nagios/etc/hosts
cfg_dir=/usr/local/nagios/etc/services

# GLOBAL EVENT HANDLERS
global_host_event_handler=xi_host_event_handler
global_service_event_handler=xi_service_event_handler



# UNMODIFIED
accept_passive_host_checks=1
accept_passive_service_checks=1
additional_freshness_latency=15
auto_reschedule_checks=0
auto_rescheduling_interval=30
auto_rescheduling_window=180
bare_update_check=0
cached_host_check_horizon=15
cached_service_check_horizon=15
check_external_commands=1
check_for_orphaned_hosts=1
check_for_orphaned_services=1
check_for_updates=1
check_host_freshness=0
check_result_path=/usr/local/nagios/var/spool/checkresults
check_result_reaper_frequency=10
check_service_freshness=1
#command_check_interval=-1
command_file=/usr/local/nagios/var/rw/nagios.cmd
daemon_dumps_core=0
date_format=us
debug_file=/usr/local/nagios/var/nagios.debug
debug_level=0
debug_verbosity=1
#enable_embedded_perl=1
enable_event_handlers=1
enable_flap_detection=1
enable_notifications=1
enable_predictive_host_dependency_checks=1
enable_predictive_service_dependency_checks=1
event_broker_options=-1
event_handler_timeout=30
execute_host_checks=1
execute_service_checks=1
#external_command_buffer_slots=4096
high_host_flap_threshold=20.0
high_service_flap_threshold=20.0
host_check_timeout=30
host_freshness_check_interval=60
host_inter_check_delay_method=s
illegal_macro_output_chars=`~$&|'"<>
illegal_object_name_chars=`~!$%^&*|'"<>?,()=
interval_length=60
lock_file=/usr/local/nagios/var/nagios.lock
log_archive_path=/usr/local/nagios/var/archives
log_external_commands=0
log_file=/usr/local/nagios/var/nagios.log
log_host_retries=1
log_initial_states=0
log_notifications=1
log_passive_checks=0
log_rotation_method=d
log_service_retries=1
low_host_flap_threshold=5.0
low_service_flap_threshold=5.0
max_check_result_file_age=3600
max_check_result_reaper_time=30
max_concurrent_checks=0
max_debug_file_size=1000000
max_host_check_spread=30
max_service_check_spread=30
nagios_group=nagios
nagios_user=nagios
notification_timeout=30
object_cache_file=/usr/local/nagios/var/objects.cache
obsess_over_hosts=0
obsess_over_services=0
ocsp_timeout=5
#p1_file=/usr/local/nagios/bin/p1.pl
passive_host_checks_are_soft=0
perfdata_timeout=5
precached_object_file=/usr/local/nagios/var/objects.precache
resource_file=/usr/local/nagios/etc/resource.cfg
retained_contact_host_attribute_mask=0
retained_contact_service_attribute_mask=0
retained_host_attribute_mask=0
retained_process_host_attribute_mask=0
retained_process_service_attribute_mask=0
retained_service_attribute_mask=0
retain_state_information=1
retention_update_interval=60
service_check_timeout=60
service_freshness_check_interval=60
service_inter_check_delay_method=s
service_interleave_factor=s
#sleep_time=0.25
soft_state_dependencies=0
state_retention_file=/usr/local/nagios/var/retention.dat
status_file=/usr/local/nagios/var/status.dat
status_update_interval=10
temp_file=/usr/local/nagios/var/nagios.tmp
temp_path=/tmp
use_aggressive_host_checking=0
#use_embedded_perl_implicitly=1
use_regexp_matching=0
use_retained_program_state=1
use_retained_scheduling_info=1
use_syslog=1
use_true_regexp_matching=0

npcd.cfg from /usr/local/nagios/etc/pnp/

Code: Select all

# NPCD.cfg - sample configuration file for PNPs NPCD
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation;
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
#
# $Id: npcd.cfg-sample.in 440 2008-04-24 09:08:20Z hendrikb $

# Privilege Options

user = nagios
group = nagios


# Logging Options

# log_type - define if you want your logs to
# 'syslog' or to a 'file'
#
# log_type = <value>
#

log_type = file
#log_type = syslog


# log_file - define a path to your logfile
# needed if 'log_type'='file'
#
# log_file = </path/to/logpath/filename>
#

log_file = /usr/local/nagios/var/npcd.log


# max_logfile_size - defines the maximum filesize (bytes)
# before the logfile will rotated.
#
# max_logfile_size = <value> (default 10Mbyte)
#

max_logfile_size = 10485760


# log_level - how much should we log?
#
# log_level = <integer value>
#
#  0 = No logging - except errors
#  1 = Small logging - some few more output
#  2 = More Logging (actual ALL logs)
# -1 = DEBUG Mode - ALL Logging and slower processing
#

log_level = 0


# NEEDED OPTIONS
#
# perfdata_spool_dir - where we can find the
# performance data files
#
# perfdata_spool_dir = </path/to/directory/>
#

perfdata_spool_dir = /usr/local/nagios/var/spool/perfdata/


# Execute following command for each found file
# in 'perfdata_spool_dir'
#
# perfdata_file_run_cmd = </path/to/command>
#
# Must be executable by user/group from above
#
# perfdata_file_run_cmd = </path/to/filename>
#

perfdata_file_run_cmd = /usr/local/nagios/libexec/process_perfdata.pl


# perfdata_file_run_cmd_args (optional)
#
# If you wish, you can apply more arguments to the
# perfdata_file_run_cmd
#
# Hint:
# NPCD will create a command line like this:
# '<perfdata_file_runc_cmd> <perfdata_file_runc_cmd_args> <filename_from_spool_dir>'
#

perfdata_file_run_cmd_args = -b


# npcd_max_threads - define how many parallel threads we
# should start

npcd_max_threads = 5

# sleep_time - how many seconds should npcd wait between dirscans
#
# sleep_time = 15 (default)

sleep_time = 15


# EXPERIMENTAL
#
# use_load_threshold - enables/disables load watching
#
# use_load_threshold = <0 / 1> (default: 0)
#

#use_load_threshold = 0


# EXPERIMENTAL
#
# load_threshold - npcd won't start new threads
# if your system load is over this threshold
#
# load_threshold = <float value> (default: 10.0)
#
# Hint: Do not use "," as decimal delimeter
#

load_threshold = 30.0


# location of your pid file

#pid_file=/var/run/npcd.pid
pid_file=/usr/local/nagiosxi/var/subsys/npcd.pid


# We have to end with a newline

npcd.cfg from /usr/local/pnp4nagios/etc/

Code: Select all

# NPCD.cfg - sample configuration file for PNPs NPCD
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation;
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA

# Privilege Options

user = nagios
group = nagios

#########################
#                       #
# Logging Options       #
#                       #
#########################

# log_type - define if you want your logs to
# 'syslog' or to a 'file'
#
# log_type = <value>
#

#log_type = file
log_type = syslog


# log_file - define a path to your logfile
# needed if 'log_type'='file'
#
# log_file = </path/to/logpath/filename>
#

log_file = /usr/local/pnp4nagios/var/npcd.log


# max_logfile_size - defines the maximum filesize (bytes)
# before the logfile will rotated.
#
# max_logfile_size = <value> (default 10Mbyte)
#

max_logfile_size = 10485760


# log_level - how much should we log?
#
# log_level = <integer value>
#
#  0 = No logging - except errors
#  1 = Small logging - some few more output
#  2 = More Logging (actual ALL logs)
# -1 = DEBUG Mode - ALL Logging and slower processing
#

log_level = 0

#########################
#                       #
# NEEDED OPTIONS        #
#                       #
#########################

# perfdata_spool_dir - where we can find the
# performance data files
#
# perfdata_spool_dir = </path/to/directory/>
#

perfdata_spool_dir = /usr/local/pnp4nagios/var/spool


# Execute following command for each found file
# in 'perfdata_spool_dir'
#
# perfdata_file_run_cmd = </path/to/command>
#
# Must be executable by user/group from above
#
# perfdata_file_run_cmd = </path/to/filename>
#

perfdata_file_run_cmd = /usr/local/pnp4nagios/libexec/process_perfdata.pl


# perfdata_file_run_cmd_args (optional)
#
# If you wish, you can apply more arguments to the
# perfdata_file_run_cmd
#
# Hint:
# NPCD will create a command line like this:
# '<perfdata_file_run_cmd> <perfdata_file_run_cmd_args> <filename_from_spool_dir>'
#

perfdata_file_run_cmd_args = --bulk


# identify_npcd (optional)
#
# If set to one (by default) npcd will append
# '-n' to the perfdata_file_run_cmd
#
# identify_npcd = 0|1 (default: 1)

identify_npcd = 1


# npcd_max_threads - define how many parallel threads we
# should start

npcd_max_threads = 5

# sleep_time - how many seconds npcd should wait between dirscans
#
# sleep_time = 15 (default)

sleep_time = 15


# EXPERIMENTAL
#
# load_threshold - npcd won't start new threads
# if your system load is above this threshold
#
# load_threshold = <float value> (default: 0.0)
#
# Hint: Do not use "," as decimal delimiter
#
# 07/15/2008: Every value above 0.0 will
#             enable this feature

load_threshold = 0.0


# location of your pid file

pid_file=/var/run/npcd.pid


#########################
#                       #
# NPCDMOD OPTIONS       #
#                       #
#########################


# perfdata_file - where should the npcdmod.o
# write the performance data
#
# must not be within the same directory as
# perfdata_spool_dir
#
# perfdata_file = </path/to/file>
#

perfdata_file = /usr/local/pnp4nagios/var/perfdata.dump


# perfdata_spool_filename - declare the destination
# filename for the spooled files
#
# This option allows you a customized filename.
# Usefull if you own different nagios servers
# which write their data to a shared storage.
#
# perfdata_spool_filename = <filename>
#
# Hint:
# The final files will be moved to
#  'perfdata_spool_dir/perfdata_spool_filename-TIMESTAMP'
#
# Example:
#
# perfdata_spool_filename = perfdata-NY
# perfdata_spool_filename = perfdata-LA

perfdata_spool_filename = perfdata

#
# perfdata_file_processing_interval
#
perfdata_file_processing_interval = 15

# We have to end with a newline

process_perfdata.cfg from /usr/local/nagios/etc/pnp/

Code: Select all

#
# Config File for process_perfdata.pl
#
# $Id: process_perfdata.cfg-sample.in 520 2008-09-16 12:50:10Z pitchfork $
#
# process_perfdata.pl Timout
#
TIMEOUT = 20
#
# Use RRDs Perl Module
#
USE_RRDs = 1
#
#
#
RRDPATH = /usr/local/nagios/share/perfdata
#
#
#
RRDTOOL = /usr/bin/rrdtool
#
#
#
CFG_DIR = /usr/local/nagios/etc/pnp
#
#
#
RRD_HEARTBEAT = 8460
#
#
#
RRA_CFG = /usr/local/nagios/etc/pnp/rra.cfg
#
#
#
RRA_STEP = 60
#
#
#
LOG_FILE = /usr/local/nagios/var/perfdata.log
#
# Loglevel 0=silent 1=normal 2=debug
#
LOG_LEVEL = 0
#
# XML encoding
# The supported encodings are ISO-8859-1, UTF-8 and US-ASCII.
# http://www.php.net/xml-parser-create
XML_ENC = UTF-8
#
# EXPERIMENTAL rrdcached Support
# Use only with rrdtool svn revision 1511+
#
# RRD_DAEMON_OPTS = unix:/tmp/rrdcached.sock

process_perfdata.cfg from /usr/local/pnp4nagios/etc/

Code: Select all

# pnp4nagios–0.6.21
# Config File for process_perfdata.pl
#
# More info on RRDtool can be found at www.rrdtool.org

#
# process_perfdata.pl timeout in seconds
#
TIMEOUT = 15

#
# Use RRDs Perl module
#
USE_RRDs = 1

#
# Path to XML/RRD files
#
RRDPATH = /usr/local/pnp4nagios/var/perfdata

#
# Location of RRDtool binary
#
RRDTOOL = /usr/bin/rrdtool

#
# Location of PNP config files
#
CFG_DIR = /usr/local/pnp4nagios/etc

#
# Use a single RRD database per service
# one or more datasources per RRD database
# RRD_STORAGE_TYPE = SINGLE
#
# Use multiple RRD databases per service
# one RRD database per datasource.
# RRD_STORAGE_TYPE = MULTIPLE
#
RRD_STORAGE_TYPE = SINGLE

#
# Max. interval between samples/updates
#
RRD_HEARTBEAT = 8460

#
# File with RRA options used to create new RRD files
#
RRA_CFG = /usr/local/pnp4nagios/etc/rra.cfg

#
# Interval at which PDPs are generated
#
RRA_STEP = 60

#
# Name of the log file
#
LOG_FILE = /usr/local/pnp4nagios/var/perfdata.log

#
# Loglevel 0=silent 1=normal 2=debug
#
LOG_LEVEL = 0

#
# XML encoding
# The supported encodings are ISO-8859-1, UTF-8 and US-ASCII.
# http://www.php.net/xml-parser-create
#
XML_ENC = UTF-8

#
# XML update delay in seconds
# 0 = Update XML Files everytime new data arrives
#
# Use this option to reduce disk I/O
#
XML_UPDATE_DELAY = 0

#
# Use only with RRDtool svn revision 1511+
#
# RRD_DAEMON_OPTS = unix:/tmp/rrdcached.sock
RRD_DAEMON_OPTS =

#
# Spool directory used for internal statistics
#
STATS_DIR = /usr/local/pnp4nagios/var/stats


#########################################################
# Gearman Worker Config
# Only used when running as Gearman worker

#
# How many child processes
#
PREFORK = 1

#
# Gearman server to connect to
# Comma separated list of Gearman job servers
#
GEARMAN_HOST = localhost:4730

#
# Restart child process after a given count of requests
#
REQUESTS_PER_CHILD = 10000

#
# Enables or disables encryption.
# It is strongly advised to not disable encryption, or
# anybody will be able to inject packages to your worker.
# When using encryption, you will have to specify a shared
# secret eithr via the KEY or the KEY_FILE option.
# Default is 1.
#
ENCRYPTION = 1

#
# A shared password which will be used for
# encryption of data pakets. Should be at least 8
# bytes long. Maximum length is 32 characters.
#
KEY = should_be_changed

#
# The shared password will be read from this file.
# Only the first 32 characters will be used.
#
# KEY_FILE = /usr/local/pnp4nagios/etc/secret.key

I'm not sure if this is not related to my pnp installation. I've not made any changes to the cfg files though.

Osay