host & service perfdata increasing alarmingly

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
dwhitfield
Former Nagios Staff
Posts: 4583
Joined: Wed Sep 21, 2016 10:29 am
Location: NoLo, Minneapolis, MN
Contact:

Re: host & service perfdata increasing alarmingly

Post by dwhitfield »

Considering your pnp4nagios setup is non-standard, I think just reinstalling pnp4nagios is the way to go. It should take something like 20 minutes to set up.

Correcting the errors from your perfdata.log were a last-ditch effort to keep you from blowing it away, but clearly it didn't work.
pmithil7
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Post by pmithil7 »

Hello,

I finally tried re-installing pnp4nagios and am sad to say that it didn't help as well. I'm still facing the issue of data not going in the spool directory. Also facing the issue that the two files 'host-perfdata' and 'service-perfdata' keep on growing. Also, still facing the issue that inspite of having 'process_perfdata' = 0 in /usr/local/nagios/etc/nagios.cfg, the data keeps getting processing in those two files.

I don't even know how to approach with this now. :(

Thanks,
Mithil
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Post by tgriep »

We would have to see the following files from your Nagios server

Code: Select all

/usr/local/nagios/etc/nagios.cfg
/usr/local/nagios/etc/objects/commands.cfg
/usr/local/pnp4nagios/libexec/process_perfdata.pl
/usr/local/pnp4nagios/etc/npcd.cfg
/usr/local/pnp4nagios/etc/process_perfdata.cfg
And also, run the following as root on the server and post the output.

Code: Select all

ps -ef --cols=300
Just to make sure, what folder are the 'host-perfdata' and 'service-perfdata' that are growing located at?
After receiving the above, we can try and help out on the issue.
Be sure to check out our Knowledgebase for helpful articles and solutions!
pmithil7
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Post by pmithil7 »

Hi Guys,

I did a brainstorm yesterday and went through the entire configuration of the Nagios server. It took me 4 hours to find out few of the issues. The issue with the graphs seems to have been fixed and i see them being populated properly now and also the 2 files that were growing in an alarming manner have been fixed with data being generated in perfdata after being spooled. How this was solved was by making changes in the main 'nagios.cfg' file. As i re-installed pnp4nagios with Bulk+npcd mode, in the main configuration file: host_perfdata_command & service_perfdata_command don't have to be defined and i commented them out. Also, i found out that nothing in 'commands.cfg' was being executed by testing it and hence in the main 'nagios.cfg' i found out that the object configuration file path of 'commands.cfg' was commented out( i have no idea how this happened). I removed the comment and restarted the nagios service and i saw data being properly accumulated how it is supposed to. :D

However, there is one thing that is still odd and that is the 'process_performance_data=0' option is set in nagios.cfg but still performance data is being processed. This is the only odd thing happening right now.

Let me know if anyone can help or someone has faced such an issue.

Thanks,
Mithil
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Post by tgriep »

The only thing I can think of is that the global process_performance_data option is disabled in the nagios.cfg file but that setting can be enabled on each individual host or service check bypassing the global setting.
Be sure to check out our Knowledgebase for helpful articles and solutions!
pmithil7
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Post by pmithil7 »

How is that enabled on each host and service? I'm not aware of it.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Post by tgriep »

It could be enabled through a template and that template is assigned to a host or service, that is one way it could be set.
Be sure to check out our Knowledgebase for helpful articles and solutions!
pmithil7
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Post by pmithil7 »

As far as i know, there is no such template assigned. It used to previously not process data when 'process_performance_data=0' was set but now inspite of it being 0 it processes data. It isn;t much of an issue right now but it's just not right.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Post by tgriep »

Take a look in the status.dat file and see if it is shows up as being enabled. If it is, then somewhere it is being set.
Be sure to check out our Knowledgebase for helpful articles and solutions!
pmithil7
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Post by pmithil7 »

[root@nagios-eqx-01 var]# less status.dat
# THIS FILE IS AUTOMATICALLY GENERATED
# BY NAGIOS. DO NOT MODIFY THIS FILE!
########################################

info {
created=1490388675
version=4.0.8
last_update_check=1490354082
update_available=1
last_version=4.0.8
new_version=4.3.1
}

programstatus {
modified_host_attributes=65
modified_service_attributes=65
nagios_pid=20559
daemon_mode=1
program_start=1490387833
last_log_rotation=0
enable_notifications=1
active_service_checks_enabled=1
passive_service_checks_enabled=1
active_host_checks_enabled=1
passive_host_checks_enabled=1
enable_event_handlers=1
obsess_over_services=0
obsess_over_hosts=0
check_service_freshness=1
check_host_freshness=0
enable_flap_detection=1
process_performance_data=1

I see in the status.dat as posted above that process_performance_data=1. Is this the reason it is being processed and overwriting the option set in nagios.cfg?
Locked