host & service perfdata increasing alarmingly
host & service perfdata increasing alarmingly
Hi,
I'm facing a peculiar issue where my /var is increasing in an alrming manner. I found out that 'host-perfdata' & 'service-perdata' are increasing continuously in an alarming manner with 32G & 83G respectively and counting.
In the nagios.cfg file i even stopped processing performance data by putting the 'process_performance_data=0' and restarted nagios but still i see the two files growing. I'm not sure why is it still processing this data.
Can anyone help before it fills up my whole /var?
-Mithil
I'm facing a peculiar issue where my /var is increasing in an alrming manner. I found out that 'host-perfdata' & 'service-perdata' are increasing continuously in an alarming manner with 32G & 83G respectively and counting.
In the nagios.cfg file i even stopped processing performance data by putting the 'process_performance_data=0' and restarted nagios but still i see the two files growing. I'm not sure why is it still processing this data.
Can anyone help before it fills up my whole /var?
-Mithil
Re: host & service perfdata increasing alarmingly
That's huge - it shouldn't be getting that big. Can you PM over a profile to myself, and dwhitfield for us to review? (Admin -> System profile -> Download profile)
Can you also show us the exact path / file name that seems to be filling up? What version of XI are you running?
Can you also show us the exact path / file name that seems to be filling up? What version of XI are you running?
Former Nagios Employee
Re: host & service perfdata increasing alarmingly
Thank you for pitching in to help me. I have sent a PM to you as you requested. My files are in '/var/pnp4nagios/var/host-perfdata' & '/var/pnp4nagios/var/service=perfdata'.
[root@nagios-eqx-01 var]# du -sh *
3.5G host-perfdata
4.0K kohana
202M perfdata
3.7M perfdata.log
75G service-perfdata
4.0K spool
8.0K stats
[root@nagios-eqx-01 var]# du -sh *
3.5G host-perfdata
4.0K kohana
202M perfdata
3.7M perfdata.log
75G service-perfdata
4.0K spool
8.0K stats
-
- Former Nagios Staff
- Posts: 4583
- Joined: Wed Sep 21, 2016 10:29 am
- Location: NoLo, Minneapolis, MN
- Contact:
Re: host & service perfdata increasing alarmingly
You posted in the XI forums, but you PMed a Core version. Could you send us a screenshot to clarify what you are running?
Thanks!
Thanks!
Re: host & service perfdata increasing alarmingly
[root@nagios-eqx-01 bin]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL
Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...
Running pre-flight check on configuration data...
Checking objects...
Checked 10608 services.
Checked 635 hosts.
Checked 29 host groups.
Checked 1 service groups.
Checked 33 contacts.
Checked 52 contact groups.
Checked 77 commands.
Checked 1 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 635 hosts
Checked 374 service dependencies
Checked 0 host dependencies
Checked 1 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 0
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
Yes. It is a core 4.0.8. Can you help here or should i post it again over there?
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL
Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...
Running pre-flight check on configuration data...
Checking objects...
Checked 10608 services.
Checked 635 hosts.
Checked 29 host groups.
Checked 1 service groups.
Checked 33 contacts.
Checked 52 contact groups.
Checked 77 commands.
Checked 1 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 635 hosts
Checked 374 service dependencies
Checked 0 host dependencies
Checked 1 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 0
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
Yes. It is a core 4.0.8. Can you help here or should i post it again over there?
-
- Former Nagios Staff
- Posts: 4583
- Joined: Wed Sep 21, 2016 10:29 am
- Location: NoLo, Minneapolis, MN
- Contact:
Re: host & service perfdata increasing alarmingly
Please wrap the output of the following four commands in separate code blocks:
I see you set processing to zero, but I wonder if that last one might help us understand how you got in this jam in the first place.
Code: Select all
tail -50 /usr/local/nagios/var/perfdata.log
tail -50 /usr/local/nagios/var/npcd.log
cat /etc/*release
grep perfdata_file_processing_interval /usr/local/nagios/etc/nagios.cfg
Re: host & service perfdata increasing alarmingly
1) Attached is a file containing output of perfdata.log. I can explain that to you about why i have those errors in it. Some time back my /var got full due to a similar reason and my nagios server crashed and since then perfdata has stopped growing and only the 'host-perfdata' and 'service-perfdata' files were increasing. Also nothing would be going into the 'spool' directory. Since then i haven't been able to see data in the graphs. I haven't been able to figure out that yet and hence the errors are because of that.
2) I don't have a npcd.log file in my server.
3)[root@nagios-eqx-01 ~]# cat /etc/*release
CentOS release 6.6 (Final)
CentOS release 6.6 (Final)
CentOS release 6.6 (Final)
4) [root@nagios-eqx-01 ~]# grep perfdata_file_processing_interval /usr/local/nagios/etc/nagios.cfg
host_perfdata_file_processing_interval=15
service_perfdata_file_processing_interval=15
2) I don't have a npcd.log file in my server.
3)[root@nagios-eqx-01 ~]# cat /etc/*release
CentOS release 6.6 (Final)
CentOS release 6.6 (Final)
CentOS release 6.6 (Final)
4) [root@nagios-eqx-01 ~]# grep perfdata_file_processing_interval /usr/local/nagios/etc/nagios.cfg
host_perfdata_file_processing_interval=15
service_perfdata_file_processing_interval=15
- Attachments
-
- perfdata log.txt
- perfdata.log output
- (7.32 KiB) Downloaded 256 times
Re: host & service perfdata increasing alarmingly
Can you also show us all your performance data related commands definitions?
Former Nagios Employee
Re: host & service perfdata increasing alarmingly
The below are some of the definitions in the main configuration file for performance.
'''
# PROCESS PERFORMANCE DATA OPTION
# This determines whether or not Nagios will process performance
# data returned from service and host checks. If this option is
# enabled, host performance data will be processed using the
# host_perfdata_command (defined below) and service performance
# data will be processed using the service_perfdata_command (also
# defined below). Read the HTML docs for more information on
# performance data.
# Values: 1 = process performance data, 0 = do not process performance data
process_performance_data=0
# HOST AND SERVICE PERFORMANCE DATA PROCESSING COMMANDS
# These commands are run after every host and service check is
# performed. These commands are executed only if the
# enable_performance_data option (above) is set to 1. The command
# argument is the short name of a command definition that you
# define in your host configuration file. Read the HTML docs for
# more information on performance data.
#host_perfdata_command=process-host-perfdata
#service_perfdata_command=process-service-perfdata
# HOST AND SERVICE PERFORMANCE DATA FILES
# These files are used to store host and service performance data.
# Performance data is only written to these files if the
# enable_performance_data option (above) is set to 1.
host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
# HOST AND SERVICE PERFORMANCE DATA FILE TEMPLATES
# These options determine what data is written (and how) to the
# performance data files. The templates may contain macros, special
# characters (\t for tab, \r for carriage return, \n for newline)
# and plain text. A newline is automatically added after each write
# to the performance data file. Some examples of what you can do are
# shown below.
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
# HOST AND SERVICE PERFORMANCE DATA FILE MODES
# This option determines whether or not the host and service
# performance data files are opened in write ("w") or append ("a")
# mode. If you want to use named pipes, you should use the special
# pipe ("p") mode which avoid blocking at startup, otherwise you will
# likely want the defult append ("a") mode.
host_perfdata_file_mode=a
service_perfdata_file_mode=a
# HOST AND SERVICE PERFORMANCE DATA FILE PROCESSING INTERVAL
# These options determine how often (in seconds) the host and service
# performance data files are processed using the commands defined
# below. A value of 0 indicates the files should not be periodically
# processed.
host_perfdata_file_processing_interval=15
service_perfdata_file_processing_interval=15
# HOST AND SERVICE PERFORMANCE DATA FILE PROCESSING COMMANDS
# These commands are used to periodically process the host and
# service performance data files. The interval at which the
# processing occurs is determined by the options above.
host_perfdata_file_processing_command=process-host-perfdata-file
service_perfdata_file_processing_command=process-service-perfdata-file
'''
Thanks.
'''
# PROCESS PERFORMANCE DATA OPTION
# This determines whether or not Nagios will process performance
# data returned from service and host checks. If this option is
# enabled, host performance data will be processed using the
# host_perfdata_command (defined below) and service performance
# data will be processed using the service_perfdata_command (also
# defined below). Read the HTML docs for more information on
# performance data.
# Values: 1 = process performance data, 0 = do not process performance data
process_performance_data=0
# HOST AND SERVICE PERFORMANCE DATA PROCESSING COMMANDS
# These commands are run after every host and service check is
# performed. These commands are executed only if the
# enable_performance_data option (above) is set to 1. The command
# argument is the short name of a command definition that you
# define in your host configuration file. Read the HTML docs for
# more information on performance data.
#host_perfdata_command=process-host-perfdata
#service_perfdata_command=process-service-perfdata
# HOST AND SERVICE PERFORMANCE DATA FILES
# These files are used to store host and service performance data.
# Performance data is only written to these files if the
# enable_performance_data option (above) is set to 1.
host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
# HOST AND SERVICE PERFORMANCE DATA FILE TEMPLATES
# These options determine what data is written (and how) to the
# performance data files. The templates may contain macros, special
# characters (\t for tab, \r for carriage return, \n for newline)
# and plain text. A newline is automatically added after each write
# to the performance data file. Some examples of what you can do are
# shown below.
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
# HOST AND SERVICE PERFORMANCE DATA FILE MODES
# This option determines whether or not the host and service
# performance data files are opened in write ("w") or append ("a")
# mode. If you want to use named pipes, you should use the special
# pipe ("p") mode which avoid blocking at startup, otherwise you will
# likely want the defult append ("a") mode.
host_perfdata_file_mode=a
service_perfdata_file_mode=a
# HOST AND SERVICE PERFORMANCE DATA FILE PROCESSING INTERVAL
# These options determine how often (in seconds) the host and service
# performance data files are processed using the commands defined
# below. A value of 0 indicates the files should not be periodically
# processed.
host_perfdata_file_processing_interval=15
service_perfdata_file_processing_interval=15
# HOST AND SERVICE PERFORMANCE DATA FILE PROCESSING COMMANDS
# These commands are used to periodically process the host and
# service performance data files. The interval at which the
# processing occurs is determined by the options above.
host_perfdata_file_processing_command=process-host-perfdata-file
service_perfdata_file_processing_command=process-service-perfdata-file
'''
Thanks.
Re: host & service perfdata increasing alarmingly
Can you look up your command definitions for what's used to process the perf data? You may just be logging it to a file based on the name, which explains why it's filling up.
Code: Select all
process-host-perfdata-file
process-service-perfdata-file
Former Nagios Employee