Performance data not processing

[dicenl]

Hi all,

I have a problem after migrating Nagios XI from a RedHat 32bit server to a CentOS 64bit (vmware image) server. I'm not able to get the performance data to show in Nagios XI. So when i go to a service and want to see the CPU load i only see "No data to display"

In /var/nagiosramdisk the are a host-perdata and service-perfdata with lots of data in it. But it just doesnt seem to get processed.

In the perfdata.log i see:

Code: Select all

2016-06-01 15:26:45 [26817] [2] Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg parameters
2016-06-01 15:26:45 [26817] [1] process_perfdata.pl-0.6.11 starting in SYNC Mode
2016-06-01 15:26:45 [26817] [1] Cant find Nagios Environment. Exiting ....

My nagios.cfg

Code: Select all

# MODIFIED
admin_email=root@localhost
admin_pager=root@localhost
translate_passive_host_checks=1
log_event_handlers=0
use_large_installation_tweaks=1
enable_environment_macros=1
use_syslog=0
date_format=euro


# NDOUtils module
broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg
#broker_module=/usr/lib/naemon/naemon-livestatus/livestatus.so /usr/local/nagios/var/rw/live
broker_module=/usr/local/lib/mk-livestatus/livestatus.o /usr/local/nagios/var/rw/live
event_broker_options=-1

# PNP settings - bulk mode with NCPD
process_performance_data=1
# service performance data
#service_perfdata_file=/usr/local/nagios/var/service-perfdata
service_perfdata_file=/var/nagiosramdisk/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tSERVICEOUTPUT::$SERVICEOUTPUT$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=process-service-perfdata-file-bulk
# host performance data
#host_perfdata_file=/usr/local/nagios/var/host-perfdata
host_perfdata_file=/var/nagiosramdisk/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tHOSTOUTPUT::$HOSTOUTPUT$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=process-host-perfdata-file-bulk


# OBJECTS - UNMODIFIED
#cfg_file=/usr/local/nagios/etc/objects/commands.cfg
#cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
#cfg_file=/usr/local/nagios/etc/objects/localhost.cfg
#cfg_file=/usr/local/nagios/etc/objects/templates.cfg
#cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg


# STATIC OBJECT DEFINITIONS (THESE DON'T GET EXPORTED/IMPORTED BY NAGIOSQL)
cfg_dir=/usr/local/nagios/etc/static

# OBJECTS EXPORTED FROM NAGIOSQL
cfg_file=/usr/local/nagios/etc/contacttemplates.cfg
cfg_file=/usr/local/nagios/etc/contactgroups.cfg
cfg_file=/usr/local/nagios/etc/contacts.cfg
cfg_file=/usr/local/nagios/etc/timeperiods.cfg
cfg_file=/usr/local/nagios/etc/commands.cfg
cfg_file=/usr/local/nagios/etc/hostgroups.cfg
cfg_file=/usr/local/nagios/etc/servicegroups.cfg
cfg_file=/usr/local/nagios/etc/hosttemplates.cfg
cfg_file=/usr/local/nagios/etc/servicetemplates.cfg
cfg_file=/usr/local/nagios/etc/servicedependencies.cfg
cfg_file=/usr/local/nagios/etc/serviceescalations.cfg
cfg_file=/usr/local/nagios/etc/hostdependencies.cfg
cfg_file=/usr/local/nagios/etc/hostescalations.cfg
cfg_file=/usr/local/nagios/etc/hostextinfo.cfg
cfg_file=/usr/local/nagios/etc/serviceextinfo.cfg
cfg_dir=/usr/local/nagios/etc/hosts
cfg_dir=/usr/local/nagios/etc/services

# GLOBAL EVENT HANDLERS
global_host_event_handler=xi_host_event_handler
global_service_event_handler=xi_service_event_handler



# UNMODIFIED
accept_passive_host_checks=1
accept_passive_service_checks=1
additional_freshness_latency=15
auto_reschedule_checks=1
auto_rescheduling_interval=30
auto_rescheduling_window=180
bare_update_check=0
cached_host_check_horizon=15
cached_service_check_horizon=15
check_external_commands=1
check_for_orphaned_hosts=1
check_for_orphaned_services=1
check_for_updates=1
check_host_freshness=0
#check_result_path=/usr/local/nagios/var/spool/checkresults
check_result_path=/var/nagiosramdisk/spool/checkresults
check_result_reaper_frequency=10
check_service_freshness=1
#command_check_interval=-1
command_file=/usr/local/nagios/var/rw/nagios.cmd
daemon_dumps_core=0
debug_file=/usr/local/nagios/var/nagios.debug
debug_level=0
debug_verbosity=1
#enable_embedded_perl=1
enable_event_handlers=1
enable_flap_detection=1
enable_notifications=1
enable_predictive_host_dependency_checks=1
enable_predictive_service_dependency_checks=1
event_broker_options=-1
event_handler_timeout=30
execute_host_checks=1
execute_service_checks=1
#external_command_buffer_slots=4096
high_host_flap_threshold=20.0
high_service_flap_threshold=20.0
host_check_timeout=30
host_freshness_check_interval=60
host_inter_check_delay_method=s
illegal_macro_output_chars=`~$&|'"<>
illegal_object_name_chars=`~!$%^&*|'"<>?,()=
interval_length=60
lock_file=/usr/local/nagios/var/nagios.lock
log_archive_path=/usr/local/nagios/var/archives
log_external_commands=0
log_file=/usr/local/nagios/var/nagios.log
log_host_retries=1
log_initial_states=0
log_notifications=1
log_passive_checks=0
log_rotation_method=d
log_service_retries=1
low_host_flap_threshold=5.0
low_service_flap_threshold=5.0
max_check_result_file_age=3600
max_check_result_reaper_time=30
max_concurrent_checks=0
max_debug_file_size=1000000
max_host_check_spread=30
max_service_check_spread=30
nagios_group=nagios
nagios_user=nagios
notification_timeout=30
object_cache_file=/usr/local/nagios/var/objects.cache
object_cache_file=/var/nagiosramdisk/objects.cache
obsess_over_hosts=0
obsess_over_services=0
ocsp_timeout=5
#p1_file=/usr/local/nagios/bin/p1.pl
passive_host_checks_are_soft=0
perfdata_timeout=5
precached_object_file=/usr/local/nagios/var/objects.precache
resource_file=/usr/local/nagios/etc/resource.cfg
retained_contact_host_attribute_mask=0
retained_contact_service_attribute_mask=0
retained_host_attribute_mask=0
retained_process_host_attribute_mask=0
retained_process_service_attribute_mask=0
retained_service_attribute_mask=0
retain_state_information=1
retention_update_interval=60
service_check_timeout=60
service_freshness_check_interval=60
service_inter_check_delay_method=s
service_interleave_factor=s
#sleep_time=0.25
soft_state_dependencies=0
state_retention_file=/usr/local/nagios/var/retention.dat
#status_file=/usr/local/nagios/var/status.dat
status_file=/var/nagiosramdisk/status.dat
status_update_interval=10
temp_file=/usr/local/nagios/var/nagios.tmp
#temp_path=/tmp
temp_path=/var/nagiosramdisk/tmp
use_aggressive_host_checking=0
#use_embedded_perl_implicitly=1
use_regexp_matching=0
use_retained_program_state=1
use_retained_scheduling_info=1
use_true_regexp_matching=0

Does anybody know what i'm doing wrong?

[rkennedy]

How did you do the migration from 32bit to the 64bit CentOS?

Additionally, can you PM over a profile for us to look at? You can download it by navigating to Admin -> System Profile -> Download Profile

[dicenl]

i just upgraded the 32bit nagios to the latest version so it was the same as it is on the 64bit. After that i dit an nagios_xi_restore.sh on the 64bit server. I forgot to migrate the rrd files to 64bit but i just finished the instruction on https://support.nagios.com/kb/article.php?id=166 and the historic data is now available. i'm not quite sure if new data will be added now. i gues not. i cant send a pm because i am a noob on this forum. can i post a link for you to download?

Many tnx in advance! it has been a bit of a stressfull day

[rkennedy]

New data should be added, nice to see you got it working!

I believe after 2 posts you can send PM's, so you should be able to now. Please post back once you've sent it over.

[dicenl]

Okay thx. I wont be able to check until tomorrow so, fingers crossed! I did send you the profile.zip file in pm. Maybe you could check for any human errors. The one thing i think might be a problem is that on the 32bit system nagios xi was installed by hand, so no vmware image. when starting nagios for the first time on the new system i got an error about a /var/nagiosramdisk folder not existing. i just created it and set the permissions. after that i did start. maybe there are some more directorys on 'non standard' locations, but i dont know.

[rkennedy]

There is a bit more then just creating the folder, take a look at this document for the ram disk and creating it -
https://assets.nagios.com/downloads/nag ... giosXI.pdf

Profile received - things look good at a quick glance, however it does look like your load is spiking which can affect performance data.

Code: Select all

[05-31-2016 15:38:42] NPCD: WARN: MAX load reached: load 11.930000/10.000000 at i=0
[05-31-2016 15:38:57] NPCD: WARN: MAX load reached: load 18.750000/10.000000 at i=1

Also seeing this - possibly also related to the disk IO? Did anything change in regards to the disk that hosts Nagios?

Code: Select all

[Wed Jun 01 17:27:23 2016] [error] [client 192.168.101.10] PHP Warning:  simplexml_load_file(): I/O warning : failed to load external entity "/usr/local/nagios/share/perfdata/bbinst-sql01/Memory_Usage.xml" in /usr/local/nagiosxi/html/includes/components/capacityplanning/capacityplanning.inc.php on line 227, referer: http://nagios/nagiosxi/includes/components/xicore/status.php?show=services&sortby=&sortorder=asc&host=&hostgroup=&servicegroup=&search=memory&searchButton=

[dicenl]

Okee, tnx. How could i change the configuration so i just uses the default file location? The ramdisk is for better performance i gues but this was only nessacery for the old server i gues. Now its on a 64bit server with 2 cores and 8GB memory. Is it easy to change to the default locaction and is there a link you can provide with the steps to follow. I'm more a Nagios admin and this is my first Nagios XI customer so im a little lost in the configs ..

[dicenl]

I think i found a solution. Because of the backup / restore to a different server the nagios.cfg on the new server has been altered by the old config. So the nagios.cfg on the new server had the complete old config. Now i just installed a new nagios server (default) and copied the nagios.cfg to my new restored server. I think this will set all settings back to de default locations and it seems to be working. Is there something anybody could think of that would go wrong is i do this of does it sound like a good solution?

[dicenl]

well that wasnt the solution eighter. i'm pretty lost right now. All is working fine but the performance graphs. Monitoring and notifying is al working. statistics are showing in the service detail section of nagios XI i also have the historical graphics data when i go to performance graph tab. but the new performance data is not added to the graphs. I dont know what is going wrong. Logging is not showing any errors (for what i can find)

The only thing i see is in /usr/local/nagios/var/perfdata.log

Code: Select all

Using Config File /usr/local/nagios/etc/pnp/process_perfdata.cfg
process_perfdata.pl-0.6.11 starting in SYNC Mode
Cant find Nagios Environment. Exiting ....

But this is al from yesterday before i converted the 32bit rrd data to the new server. Last update time is 2016-06-01 15:26:45 and no new enty's.

Any help would be much appriciated.

[dicenl]

i need to say sorry to rkennedy! You were right about the ramdisk configuration. i didnt read the whole document because im a technician (nerd) and think i know better

The script didnt work, but below the script was the manual configuration instruction. After checking the whole document and al the config files and folders it was obvious about 80% of the config was incorrect. i corrected it al and now the performance graphs are building again.

Tnx for the support!!