Huge perfdata

[amprantino]

1st path:

nagios.cfg
# HOST AND SERVICE PERFORMANCE DATA PROCESSING COMMANDS
# These commands are run after every host and service check is
# performed. These commands are executed only if the
# enable_performance_data option (above) is set to 1. The command
# argument is the short name of a command definition that you
# define in your host configuration file. Read the HTML docs for
# more information on performance data.

host_perfdata_command=process-host-perfdata
service_perfdata_command=process-service-perfdata

commands.cfg:
# 'process-host-perfdata' command definition
define command{
command_name process-host-perfdata
command_line /usr/bin/printf "%b" "$LASTHOSTCHECK$\t$HOSTNAME$\t$HOSTSTATE$\t$HOSTATTEMPT$\t$HOSTSTATETYPE$\t$HOSTEXECUTIONTIME$\t$HOSTOUTPUT$\t$HOSTPERFDATA$\n" >> /var/log/nagios/host-perfdata.out
}

# 'process-service-perfdata' command definition
define command{
command_name process-service-perfdata
command_line /usr/bin/printf "%b" "$LASTSERVICECHECK$\t$HOSTNAME$\t$SERVICEDESC$\t$SERVICESTATE$\t$SERVICEATTEMPT$\t$SERVICESTATETYPE$\t$SERVICEEXECUTIONTIME$\t$SERVICELATENCY$\t$SERVICEOUTPUT$\t$SERVICEPERFDATA$\n" >> /var/log/nagios/service-perfdata.out
}

2nd path

nagios.cfg
# HOST AND SERVICE PERFORMANCE DATA FILES
# These files are used to store host and service performance data.
# Performance data is only written to these files if the
# enable_performance_data option (above) is set to 1.

host_perfdata_file=/opt/perfdata/host-perfdata
service_perfdata_file=/opt/perfdata/service-perfdata

The above two quotes lead to double-saving the service performance data; the data are saved twice at the following locations:
/opt/perfdata/service-perfdata &
/var/log/nagios/service-perfdata.out

Is it safe to disable "process-service-perfdata" command ?


Each file is 26GB in size!! Is any way to reduce their size by loosing historical data before a specific time (ex. 2012) ?


Thank you

[abrist]

Is there a reason why you are spooling all your perfdata into 1 file? Are you running pnp4nagios/npcd or any other reapers? What is the use case for this?

[amprantino]

At the moment I am not using pnp4nagios or npcd .

I don't know if nagios use these data for reporting.

Is any way to purge the info before a specific date? (for example before 2012)

[abrist]

I don't know if nagios use these data for reporting.

It is not. It only holds performance data and is usually only used by the graphing add ons (unless you rolled out something custom the relies on them).

Is any way to purge the info before a specific date? (for example before 2012)

Most likely, though it depends on their format. Could you post a few lines from each file?

[amprantino]

Data collected seems to be identical, so I kept only /opt/perfdata/service-perfdata file.

Code: Select all

# tail -f /opt/perfdata/service-perfdata 
[SERVICEPERFDATA]       1428498789      host9       Ambient-Temp    0.013   0.096   System Ambient Temperature OK - *24*    iso.3.6.1.4.1.534.1.3.4.1.2.1=24
[SERVICEPERFDATA]       1428498792      host11 CPU     0.127   0.110   4 CPU, average load 29.2%  50% : OK     cpu_prct_used=29.25%;50;85
[SERVICEPERFDATA]       1428498792      host7 FileServer      1.369   0.104   W:\ Label:PFileserver  Serial Number 3c61fa6f: 91%used(438626MB/482301MB) (89%) : WARNING   W:\_Label:PFileserver__Serial_Number_3c61fa6f=438626MB;429248;438894;0;482301
[SERVICEPERFDATA]       1428498792      host1  Disk-C  1.369   0.107   C:\ Label:  Serial Number 60ab5dc2: 49%used(29771MB/61337MB) (85%) : OK C:\_Label:__Serial_Number_60ab5dc2=29771MB;52136;59497;0;61337
[SERVICEPERFDATA]       1428498792      host1    REACHABLE       5.044   0.236   PING OK - Packet loss = 16%, RTA = 22.73 ms     rta=22.725000ms;900.000000;1500.000000;0.000000 pl=16%;60;79;0
[SERVICEPERFDATA]       1428498792      host3  REACHABLE       4.006   0.077   PING OK - Packet loss = 0%, RTA = 0.23 ms       rta=0.231000ms;900.000000;1500.000000;0.000000 pl=0%;60;79;0
[SERVICEPERFDATA]       1428498792      host2   REACHABLE       4.010   0.073   PING OK - Packet loss = 0%, RTA = 0.25 ms       rta=0.248000ms;900.000000;1500.000000;0.000000 pl=0%;60;79;0
[SERVICEPERFDATA]       1428498792      host7  REACHABLE       4.014   0.080   PING OK - Packet loss = 0%, RTA = 1.23 ms       rta=1.227000ms;900.000000;1500.000000;0.000000 pl=0%;60;79;0
[SERVICEPERFDATA]       1428498792      host4    REACHABLE       4.016   0.083   PING OK - Packet loss = 0%, RTA = 1.71 ms       rta=1.709000ms;900.000000;1500.000000;0.000000 pl=0%;60;79;0
[SERVICEPERFDATA]       1428498795      host6   Disk-C  0.127   0.122   C:\ Label:  Serial Number ec2fa45d: 18%used(25135MB/139290MB) (85%) : OK        C:\_Label:__Serial_Number_ec2fa45d=25135MB;118396;135111;0;139290

To drop all info collected before 1/1/2013 0:0:1 I have to drop all lines with Unix time less than 1356998401

So i think this will solve my problem :

awk -v m=1356998401 '{k=$1 OFS $2} $2<=m {next} {print}' inputfile

It seems to work on my test data

[abrist]

Yep, that does seem to do the trick. Though I still wonder what you are even using these files for . . . .

[kidzeivo]

Please use this configuration for NPCD and comment out the pnp4nagios command configuration

# pnp4nagios
#define command{
# command_name process-service-perfdata
# command_line /usr/bin/perl /usr/libexec/pnp4nagios/process_perfdata.pl
#}
#
#define command{
# command_name process-host-perfdata
# command_line /usr/bin/perl /usr/libexec/pnp4nagios/process_perfdata.pl -d HOSTPERFDATA
#}

# Bulk with NPCD mode
#

define command {
command_name process-service-perfdata-file
command_line /bin/mv /tmp/service-perfdata /var/spool/pnp4nagios/service-perfdata.$TIMET$
}

define command {
command_name process-host-perfdata-file
command_line /bin/mv /tmp/host-perfdata /var/spool/pnp4nagios/host-perfdata.$TIMET$
}

[dwhitfield]

@amprantino, it looks like your issue may be resolved. Is it ok if we lock this thread?

[amprantino]

One final question or feature request

Is it possible to add an auto-purge parameter: for example nagios to purge all historical data that is older that a specific time, for example 1 or 2 years?

[dwhitfield]

It looks like you already have an awk command for cleanup. You could set it to run with cron. Is that what you are doing now, or are you just running it manually?

If you'd like to see something more built-in, I'd suggest a feature request at https://github.com/NagiosEnterprises/nagioscore/issues . The devs will decide when/if the feature goes in. Of course, since it is open source, you could build it, and then the devs could potentially merge your code in.