checkresults issue

[kyang]

Sounds good, thanks!

If it persists with the error again, show us the logs again.

Code: Select all

tail -50 /usr/local/nagios/var/npcd.log

Code: Select all

tail -50 /usr/local/nagios/var/perfdataproc.log

Still a Ramdisk, may help with the load as @dwhitfield mentioned.

https://assets.nagios.com/downloads/nag ... giosXI.pdf

[sandeepatil]

Thanks for sharing details.

If again issue occurs will check with RAMDISK.

Questions for RAMDISK,

1) RAMDISK good for performance, so why not build with XI / Core install package.
2) RAMDISK will overcome the "inode" full issue.
3) Same configuration will work with Nagios Core.

[kyang]

No problem. Is the issue no longer occurring anymore?

1) RAMDISK good for performance, so why not build with XI / Core install package.

I think the reason why it is not shipped with a standard XI/Core install is that everyone has different requirements. Some could have less hardware on hand and fewer hosts to monitor for example.
It all comes down to the generic answer like "everyone's environment is different".

2) RAMDISK will overcome the "inode" full issue.

I think it's safe to say it helps performance, especially the files that are known to affect Disk\IO since they become rather large depending on your environment.

3) Same configuration will work with Nagios Core.

It should be the same configuration, as Nagios XI is basically Core under it.
In the documentation, it lists the example paths of regular file locations in an XI install.
It's safe to say that, if your Core matches up to those paths, then it definitely works.

I hope this helps! Are there any more questions or are we okay to lock this up?

[sandeepatil]

npcd.log

[02-18-2018 12:03:07] NPCD: ERROR: Executed command exits with return code '7'
[02-18-2018 12:03:07] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1518973363.perfdata.service'
[02-19-2018 12:03:17] NPCD: ERROR: Executed command exits with return code '7'
[02-19-2018 12:03:17] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1519059779.perfdata.service'
[02-19-2018 20:07:54] NPCD: WARN: MAX load reached: load 30.590000/10.000000 at i=0
[02-19-2018 20:08:09] NPCD: WARN: MAX load reached: load 23.810000/10.000000 at i=1
[02-19-2018 20:08:24] NPCD: WARN: MAX load reached: load 18.750000/10.000000 at i=1
[02-19-2018 20:08:39] NPCD: WARN: MAX load reached: load 14.750000/10.000000 at i=1
[02-19-2018 20:08:54] NPCD: WARN: MAX load reached: load 24.040000/10.000000 at i=1
[02-19-2018 20:09:17] NPCD: WARN: MAX load reached: load 33.400000/10.000000 at i=1
[02-19-2018 20:09:32] NPCD: WARN: MAX load reached: load 26.440000/10.000000 at i=1
[02-19-2018 20:09:47] NPCD: WARN: MAX load reached: load 21.230000/10.000000 at i=1
[02-19-2018 20:10:02] NPCD: WARN: MAX load reached: load 16.600000/10.000000 at i=1
[02-19-2018 20:10:17] NPCD: WARN: MAX load reached: load 13.740000/10.000000 at i=1
[02-19-2018 20:10:32] NPCD: WARN: MAX load reached: load 11.310000/10.000000 at i=1
[02-19-2018 20:12:57] NPCD: ERROR: Executed command exits with return code '7'
[02-19-2018 20:12:57] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1519089147.perfdata.service'
[02-19-2018 20:13:12] NPCD: WARN: MAX load reached: load 12.670000/10.000000 at i=0
[02-19-2018 20:13:27] NPCD: WARN: MAX load reached: load 46.810000/10.000000 at i=1
[02-19-2018 20:13:42] NPCD: WARN: MAX load reached: load 36.580000/10.000000 at i=1
[02-19-2018 20:13:57] NPCD: WARN: MAX load reached: load 28.830000/10.000000 at i=1
[02-19-2018 20:14:19] NPCD: WARN: MAX load reached: load 49.900000/10.000000 at i=1
[02-19-2018 20:14:34] NPCD: WARN: MAX load reached: load 57.720000/10.000000 at i=1
[02-19-2018 20:14:55] NPCD: WARN: MAX load reached: load 58.600000/10.000000 at i=1
[02-19-2018 20:15:10] NPCD: WARN: MAX load reached: load 46.340000/10.000000 at i=1
[02-19-2018 20:15:25] NPCD: WARN: MAX load reached: load 71.100000/10.000000 at i=1
[02-19-2018 20:15:40] NPCD: WARN: MAX load reached: load 55.510000/10.000000 at i=1
[02-19-2018 20:15:55] NPCD: WARN: MAX load reached: load 61.700000/10.000000 at i=1
[02-19-2018 20:16:17] NPCD: WARN: MAX load reached: load 61.140000/10.000000 at i=1
[02-19-2018 20:16:32] NPCD: WARN: MAX load reached: load 47.960000/10.000000 at i=1
[02-19-2018 20:16:47] NPCD: WARN: MAX load reached: load 37.420000/10.000000 at i=1
[02-19-2018 20:17:02] NPCD: WARN: MAX load reached: load 29.340000/10.000000 at i=1
[02-19-2018 20:17:17] NPCD: WARN: MAX load reached: load 22.840000/10.000000 at i=1
[02-19-2018 20:17:32] NPCD: WARN: MAX load reached: load 18.550000/10.000000 at i=1
[02-19-2018 20:17:47] NPCD: WARN: MAX load reached: load 14.790000/10.000000 at i=1
[02-19-2018 20:18:02] NPCD: WARN: MAX load reached: load 11.820000/10.000000 at i=1
[02-20-2018 04:42:46] NPCD: ERROR: Executed command exits with return code '7'
[02-20-2018 04:42:46] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1519119744.perfdata.service'

Please explain meaning :
1] NPCD: WARN: MAX load reached: load 14.790000/10.000000 at i=1

(i=1, i=12, etc)

2] ERROR: Executed command exits with return code '7'

Different code values.

[mcapra]

This KB article has relevant information:
https://support.nagios.com/kb/article.php?id=9

1] NPCD: WARN: MAX load reached: load 14.790000/10.000000 at i=1

NPCD has a configuration directive to produce a warning if the system is under particularly high load. It is mentioned in the KB article I have linked.

14.79 might not be an "excessive" load for this system though. In which case, you can tune that threshold.

2] ERROR: Executed command exits with return code '7'

This means the process_perfdata.pl script was terminated via unix signal. Generally speaking, it means the script was unable to properly exit in time.

Code: Select all

#
# Handle Signals
#
sub handle_signal {
    my ($signal) = (@_);
    if ( $signal eq "ALRM" ) {
        print_log( "*** TIMEOUT: Timeout after $opt_t secs. ***", 0 );
        if ( $opt_b && !$opt_n ) {
            print_log( "*** TIMEOUT: Deleting current file to avoid loops",   0 );
            print_log( "*** TIMEOUT: Please check your process_perfdata.cfg", 0 );
        }
        elsif ( $opt_b && $opt_n ) {
            print_log( "*** TIMEOUT: Deleting current file to avoid NPCD loops", 0 );
            print_log( "*** TIMEOUT: Please check your npcd.cfg",                0 );
        }
    }
    if ($opt_b) {
        my $pdfile = "$opt_b" . "-PID-" . $$;
        if ( unlink("$pdfile") == 1 ) {
            print_log( "*** TIMEOUT: $pdfile deleted", 0 );
        }
        else {
            print_log( "*** TIMEOUT: Could not delete $pdfile:$!", 0 );
        }
    }
    my $temp_file = "$conf{RRDPATH}/$NAGIOS{HOSTNAME}/$NAGIOS{SERVICEDESC}.xml.$$";
    if ( -f $temp_file ) {
        unlink($temp_file);
    }
    print_log( "*** Timeout while processing Host: \"$NAGIOS{HOSTNAME}\" Service: \"$NAGIOS{SERVICEDESC}\"", 0 );
    print_log( "*** process_perfdata.pl terminated on signal $signal", 0 );
    $t1 = [gettimeofday];
    $rt = tv_interval $t0, $t1;
    $stats{runtime} = $rt;
    exit 7;
}

It could also be indicative of a resource issue. The KB article has steps for adding richer debugging to the logs.

[kyang]

Thanks for the help @mcapra!

sandeepatil, let us know if you have any more questions!