host & service perfdata increasing alarmingly

An open discussion forum for obtaining help with Nagios Core. Nagios Core users of all experience levels are welcome here. Subforum have been created for the discussion of Nagios Core and Nagios Plugin development.

NOTE: The SourceForge.net mailing lists have been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

Re: host & service perfdata increasing alarmingly

Postby pmithil7 » Thu Feb 09, 2017 6:57 pm

I have this in my commands.cfg:
'''
# SAMPLE PERFORMANCE DATA COMMANDS
#
# These are sample performance data commands that can be used to send performance
# data output to two text files (one for hosts, another for services). If you
# plan on simply writing performance data out to a file, consider using the
# host_perfdata_file and service_perfdata_file options in the main config file.
#
################################################################################
# 'process-host-perfdata' command definition
define command{
command_name process-host-perfdata
command_line /usr/bin/printf "%b" "$LASTHOSTCHECK$\t$HOSTNAME$\t$HOSTSTATE$\t$HOSTATTEMPT$\t$HOSTSTATETYPE$\t$HOSTEXECUTIONTIME$\t$HOSTOUTPUT$\t$HOSTPERFDATA$\n" >> /usr/local/nagios/var/host-perfdata.out
}

# 'process-service-perfdata' command definition
define command{
command_name process-service-perfdata
command_line /usr/bin/printf "%b" "$LASTSERVICECHECK$\t$HOSTNAME$\t$SERVICEDESC$\t$SERVICESTATE$\t$SERVICEATTEMPT$\t$SERVICESTATETYPE$\t$SERVICEEXECUTIONTIME$\t$SERVICELATENCY$\t$SERVICEOUTPUT$\t$SERVICEPERFDATA$\n" >> /usr/local/nagios/var/service-perfdata.out
}

define command{
command_name process-service-perfdata-file
command_line /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
}

define command{
command_name process-host-perfdata-file
command_line /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
}
'''

Also, an interesting find that might help:
'''
[root@nagios-eqx-01 local]# grep -R -i 'process-host-perfdata-file' .
./nagios/var/nagios.log:[1486632671] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486632707] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486663091] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486663173] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486663610] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486664528] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486667224] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486668876] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486668923] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
./nagios/var/nagios.log:[1486676061] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
grep: ./nagios/var/rw/nagios.qh: No such device or address
^C
[root@nagios-eqx-01 local]# grep -R -i 'process-service-perfdata-file' .
./nagios/var/nagios.log:[1486632671] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486632707] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486663091] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486663173] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486663610] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486664528] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486667224] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486668876] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486668923] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
./nagios/var/nagios.log:[1486676061] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!
grep: ./nagios/var/rw/nagios.qh: No such device or address
'''
pmithil7
 
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Postby pmithil7 » Fri Feb 10, 2017 1:17 pm

Any updates on this?
pmithil7
 
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Postby rkennedy » Fri Feb 10, 2017 1:32 pm

pmithil7 wrote:Any updates on this?

Please do not bump your thread. We work through a queue, and you only push yourself to the bottom once again.

./nagios/var/nagios.log:[1486676061] Warning: Host performance file processing command 'process-host-perfdata-file' was not found - host performance data file will not be processed!
grep: ./nagios/var/rw/nagios.qh: No such device or address
^C
[root@nagios-eqx-01 local]# grep -R -i 'process-service-perfdata-file' .
./nagios/var/nagios.log:[1486632671] Warning: Service performance file processing command 'process-service-perfdata-file' was not found - service performance data file will not be processed!

Can you PM over myself and @dwhitfield an entire copy of your /usr/local/nagios/etc directory? The error mentions that something isn't processing properly with your performance data so it'll help to see everything at play.
rkennedy
 
Posts: 6562
Joined: Mon Oct 05, 2015 11:45 am

Re: host & service perfdata increasing alarmingly

Postby rkennedy » Fri Feb 10, 2017 2:57 pm

I was able to find this thread - viewtopic.php?f=6&t=41487 - it appears this issue may have been going on for a bit here. From what I can tell in your file -
Code: Select all
host_perfdata_file_processing_command=process-host-perfdata-file
service_perfdata_file_processing_command=process-service-perfdata-file


Is executing this -
Code: Select all
define command{
       command_name    process-service-perfdata-file
       command_line    /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
}

define command{
       command_name    process-host-perfdata-file
       command_line    /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
}


Mine is a bit different over here -
Code: Select all
host_perfdata_file_processing_command=process-host-perfdata-file-bulk

define command {
       command_name                             process-host-perfdata-file-bulk
       command_line                             /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}


From there, it would really depend on what's processing the perf data to pick it up. What other addons are you using in conjunction with Nagios? I saw in the previous post mentions of livestatus / pnp4nagios. What versions?
rkennedy
 
Posts: 6562
Joined: Mon Oct 05, 2015 11:45 am

Re: host & service perfdata increasing alarmingly

Postby pmithil7 » Fri Feb 10, 2017 2:58 pm

Sorry for being a little impatient. I have sent a PM message as requested with the directory contents.

From the logs i feel like changes need to be made in the commands.cfg for the definitions of 'process-service-perfdata-file' and 'process-host-perfdata-file'. Let me know your thoughts on it.

Thanks,
Mithil
pmithil7
 
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Postby rkennedy » Fri Feb 10, 2017 3:16 pm

Can you address the question above?
rkennedy wrote:I was able to find this thread - viewtopic.php?f=6&t=41487 - it appears this issue may have been going on for a bit here. From what I can tell in your file -
From there, it would really depend on what's processing the perf data to pick it up. What other addons are you using in conjunction with Nagios? I saw in the previous post mentions of livestatus / pnp4nagios. What versions?


It's hard to say what needs to be changed since we did not configure all of this. It might be revisiting what @ssax mentioned - viewtopic.php?f=6&t=41487#p205430
rkennedy
 
Posts: 6562
Joined: Mon Oct 05, 2015 11:45 am

Re: host & service perfdata increasing alarmingly

Postby tgriep » Fri Feb 10, 2017 3:32 pm

If has to be one of the pnp4nagios config files that are putting the performance data in the wrong folder.
Take a look at the files in the following folder and see if they are correct.
Code: Select all
/usr/local/pnp4nagios/etc

Or reinstall / remove pnp4nagios to fix the issue.
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 5307
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Postby pmithil7 » Fri Feb 10, 2017 5:24 pm

From the thread that you mentioned, my problem wasn't solved and hence i temporarily stopped processing performance data by putting process_performance_data = 0. This stopped data from accumulating and it after a month it again started accumulating data in those two files inspite of process_performance_data being 0. Hence, this grew up my concern about how data is still being gathered.

I have pnp4nagios of :

PKG_VERSION="0.6.25"
PKG_NAME="pnp4nagios"

Other then that i have nrpe, check_mk and wato as plugins on nagios.

I also checked /usr/local/pnp4nagios/etc and the configuration files seem to be ideal.

Any suggestion? Should i uninstall and re-install pnp4nagios again?
pmithil7
 
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

Re: host & service perfdata increasing alarmingly

Postby tgriep » Mon Feb 13, 2017 10:51 am

Yes, try uninstalling pnp4nagios and reinstalling to see if that fixes the issue.
If you have any other brokers enabled in the nagios.cfg file, try commenting them out as well to see if that stops the collecting of the performance data.
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 5307
Joined: Thu Oct 30, 2014 9:02 am

Re: host & service perfdata increasing alarmingly

Postby pmithil7 » Mon Feb 13, 2017 12:53 pm

Thank you for your help. Any document or instructions on how to safely uninstall pnp4nagios without affecting nagios and other plugins? Any specifics that i should look forward to that won't crash the server?

I'm just being cautious here. Thanks
pmithil7
 
Posts: 43
Joined: Wed Dec 07, 2016 7:06 pm

PreviousNext

Return to Nagios Core

Who is online

Users browsing this forum: Google [Bot] and 10 guests