Page 1 of 2
checkresults issue
Posted: Mon Nov 20, 2017 8:52 am
by sandeepatil
Hi,
We faced issue with checkresults directory keep max data and partition was 100% full. After removing all data from checkresults, nagios working fine.
During troubleshooting inode was shown 100% full.
Please explain the issue, Why large amount of files are created in checkresults directory ? and What is preventive steps for avoiding the same ?
Thanks,
Sandeep.
Re: checkresults issue
Posted: Mon Nov 20, 2017 3:59 pm
by dwasswa
Hi
@sandeepatil,
Issues with
checkresults can be caused by a couple of things:
1.
npcd daemon not running. To check that, please run the commands below:
service npcd restart then
service npcd status
Please attach
npcd.log file that contains your npcd logs. This file is located at
/usr/local/nagios/var
Please attach
perfdataproc.log file located at
/usr/local/nagiosxi/var
2.Have you installed ramdisk on your Nagios XI server recently?
3.Expired Nagios user account. Check this with this command...
chage -l nagios
4.Wrong group permissions or permission changes
5. enabled selinux; check this by running this command...
/usr/sbin/sestatus
Lastly, please PM your system profile by following the steps below:
Go to
Admin >>
System Profile >>
Download Profile
Download_system_profile.PNG
Re: checkresults issue
Posted: Mon Nov 20, 2017 4:01 pm
by lmiltchev
This could be caused by permissions on the checkresults directory. Please check out the KB article below:
https://support.nagios.com/kb/article/n ... d-416.html
Are you using a ramdisk with Nagios XI? Can you post the
nagios.cfg file, so that we could review it?
What is the output of the following command?
Code: Select all
ls /usr/local/nagios/var/spool/checkresults | wc -l
... or this one (if you are using a ramdisk):
Code: Select all
ls /var/nagiosramdisk/spool/checkresults/ | wc -l
Re: checkresults issue
Posted: Tue Nov 21, 2017 7:54 am
by sandeepatil
We are not using ramdisk with Nagios XI.
Please find status.
Code: Select all
[nagios@abcd123]$ ls /usr/local/nagios/var/spool/checkresults | wc -l
360
Nagios user account not expire.
selinux disabled.
Re: checkresults issue
Posted: Tue Nov 21, 2017 1:58 pm
by lmiltchev
1. Can you run the following commands and show the output in code wraps?
Code: Select all
service npcd restart
service npcd status
ps -ef | grep perf
ls -lad /usr/local/nagios/var/spool/checkresults
tail -100 /usr/local/nagios/var/npcd.log
tail -100 /usr/local/nagios/var/perfdataproc.log
Note: The commands above will show us if ncpd and perdataproc are running, what are the permissions of the checkresults directory, and if there are any clues in the log files.
2. PM me or any other member of the Nagios Support Team your profile (
profile.zip).
Note: We will review your configs and various logs for issues.
3. Also, run the following command:
Code: Select all
watch -n 1 'ls /usr/local/nagios/var/spool/checkresults | wc -l'
watch the output for a while (1-2 min), then exit by pressing Ctrl+c.
Let us know if the number of the files in the checkresults directory is increasing or decreasing over time.
Re: checkresults issue
Posted: Wed Nov 22, 2017 11:18 am
by sandeepatil
Please find requested details,
ps -ef | grep perf
Code: Select all
$ ps -ef | grep perf
root 177 2 0 Nov20 ? 00:00:00 [perf]
nagios 9955 9930 0 11:05 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios 9958 9955 0 11:05 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
sp470a 10227 27064 0 11:05 pts/9 00:00:00 grep --color=auto perf
ls -lad /usr/local/nagios/var/spool/checkresults
Code: Select all
$ tail -100 /usr/local/nagios/var/npcd.log
[11-21-2017 07:15:00] NPCD: WARN: MAX load reached: load 121.680000/10.000000 at i=1
[11-21-2017 07:15:15] NPCD: WARN: MAX load reached: load 122.130000/10.000000 at i=1
[11-21-2017 07:15:30] NPCD: WARN: MAX load reached: load 122.950000/10.000000 at i=1
[11-21-2017 07:15:45] NPCD: WARN: MAX load reached: load 123.340000/10.000000 at i=1
[11-21-2017 07:16:00] NPCD: WARN: MAX load reached: load 123.480000/10.000000 at i=1
[11-21-2017 07:16:15] NPCD: WARN: MAX load reached: load 124.060000/10.000000 at i=1
[11-21-2017 07:16:30] NPCD: WARN: MAX load reached: load 124.900000/10.000000 at i=1
[11-21-2017 07:16:45] NPCD: WARN: MAX load reached: load 125.650000/10.000000 at i=1
[11-21-2017 07:17:00] NPCD: WARN: MAX load reached: load 126.170000/10.000000 at i=1
[11-21-2017 07:17:15] NPCD: WARN: MAX load reached: load 128.730000/10.000000 at i=1
[11-21-2017 07:17:30] NPCD: WARN: MAX load reached: load 130.050000/10.000000 at i=1
[11-21-2017 07:17:45] NPCD: WARN: MAX load reached: load 129.680000/10.000000 at i=1
[11-21-2017 07:18:00] NPCD: WARN: MAX load reached: load 129.560000/10.000000 at i=1
[11-21-2017 07:18:15] NPCD: WARN: MAX load reached: load 130.090000/10.000000 at i=1
[11-21-2017 07:18:31] NPCD: WARN: MAX load reached: load 130.440000/10.000000 at i=1
[11-21-2017 07:18:46] NPCD: WARN: MAX load reached: load 130.320000/10.000000 at i=1
[11-21-2017 07:19:01] NPCD: WARN: MAX load reached: load 130.260000/10.000000 at i=1
[11-21-2017 07:19:16] NPCD: WARN: MAX load reached: load 130.440000/10.000000 at i=1
[11-21-2017 07:19:31] NPCD: WARN: MAX load reached: load 131.020000/10.000000 at i=1
[11-21-2017 07:19:46] NPCD: WARN: MAX load reached: load 131.800000/10.000000 at i=1
[11-21-2017 07:20:01] NPCD: WARN: MAX load reached: load 132.130000/10.000000 at i=1
[11-21-2017 07:20:16] NPCD: WARN: MAX load reached: load 133.390000/10.000000 at i=1
[11-21-2017 07:20:31] NPCD: WARN: MAX load reached: load 133.820000/10.000000 at i=1
[11-21-2017 07:20:46] NPCD: WARN: MAX load reached: load 133.850000/10.000000 at i=1
[11-21-2017 07:21:01] NPCD: WARN: MAX load reached: load 135.630000/10.000000 at i=1
[11-21-2017 07:21:16] NPCD: WARN: MAX load reached: load 136.280000/10.000000 at i=1
[11-21-2017 07:21:31] NPCD: WARN: MAX load reached: load 136.530000/10.000000 at i=1
[11-21-2017 07:21:46] NPCD: WARN: MAX load reached: load 136.730000/10.000000 at i=1
[11-21-2017 07:22:01] NPCD: WARN: MAX load reached: load 136.970000/10.000000 at i=1
[11-21-2017 07:22:16] NPCD: WARN: MAX load reached: load 137.540000/10.000000 at i=1
[11-21-2017 07:22:31] NPCD: WARN: MAX load reached: load 138.650000/10.000000 at i=1
[11-21-2017 07:22:46] NPCD: WARN: MAX load reached: load 138.440000/10.000000 at i=1
[11-21-2017 07:23:01] NPCD: WARN: MAX load reached: load 136.600000/10.000000 at i=1
[11-21-2017 07:23:16] NPCD: WARN: MAX load reached: load 132.470000/10.000000 at i=1
[11-21-2017 07:23:31] NPCD: WARN: MAX load reached: load 129.310000/10.000000 at i=1
[11-21-2017 07:23:47] NPCD: WARN: MAX load reached: load 127.090000/10.000000 at i=1
[11-21-2017 07:24:02] NPCD: WARN: MAX load reached: load 125.950000/10.000000 at i=1
[11-21-2017 07:24:17] NPCD: WARN: MAX load reached: load 124.460000/10.000000 at i=1
[11-21-2017 07:24:32] NPCD: WARN: MAX load reached: load 123.800000/10.000000 at i=1
[11-21-2017 07:24:47] NPCD: WARN: MAX load reached: load 122.510000/10.000000 at i=1
[11-21-2017 07:25:02] NPCD: WARN: MAX load reached: load 121.520000/10.000000 at i=1
[11-21-2017 07:25:17] NPCD: WARN: MAX load reached: load 123.800000/10.000000 at i=1
[11-21-2017 07:25:32] NPCD: WARN: MAX load reached: load 127.040000/10.000000 at i=1
[11-21-2017 07:25:47] NPCD: WARN: MAX load reached: load 124.150000/10.000000 at i=1
[11-21-2017 07:26:02] NPCD: WARN: MAX load reached: load 122.170000/10.000000 at i=1
[11-21-2017 07:26:17] NPCD: WARN: MAX load reached: load 120.430000/10.000000 at i=1
[11-21-2017 07:26:32] NPCD: WARN: MAX load reached: load 119.400000/10.000000 at i=1
[11-21-2017 07:26:47] NPCD: WARN: MAX load reached: load 119.550000/10.000000 at i=1
[11-21-2017 07:27:02] NPCD: WARN: MAX load reached: load 119.800000/10.000000 at i=1
[11-21-2017 07:27:17] NPCD: WARN: MAX load reached: load 119.270000/10.000000 at i=1
[11-21-2017 07:27:32] NPCD: WARN: MAX load reached: load 118.900000/10.000000 at i=1
[11-21-2017 07:27:47] NPCD: WARN: MAX load reached: load 118.400000/10.000000 at i=1
[11-21-2017 07:28:02] NPCD: WARN: MAX load reached: load 118.220000/10.000000 at i=1
[11-21-2017 07:28:03] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 08:19:09] NPCD: npcd Daemon (0.4.14) started with PID=78114
[11-21-2017 08:19:09] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 08:19:09] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 11:54:46] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 11:54:46] NPCD: npcd Daemon (0.4.14) started with PID=2531
[11-21-2017 11:54:46] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 11:54:46] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 12:39:31] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 12:39:31] NPCD: npcd Daemon (0.4.14) started with PID=34493
[11-21-2017 12:39:31] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 12:39:31] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 12:52:51] NPCD: WARN: MAX load reached: load 10.100000/10.000000 at i=0
[11-21-2017 12:53:06] NPCD: WARN: MAX load reached: load 10.120000/10.000000 at i=1
[11-21-2017 12:53:21] NPCD: WARN: MAX load reached: load 10.850000/10.000000 at i=1
[11-21-2017 12:53:36] NPCD: WARN: MAX load reached: load 10.220000/10.000000 at i=1
[11-21-2017 12:53:51] NPCD: WARN: MAX load reached: load 10.090000/10.000000 at i=1
[11-21-2017 12:54:06] NPCD: WARN: MAX load reached: load 10.070000/10.000000 at i=1
[11-21-2017 12:54:21] NPCD: WARN: MAX load reached: load 10.310000/10.000000 at i=1
[11-21-2017 13:46:40] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 13:46:40] NPCD: npcd Daemon (0.4.14) started with PID=69973
[11-21-2017 13:46:40] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 13:46:40] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 14:09:46] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 14:15:53] NPCD: npcd Daemon (0.4.14) started with PID=98403
[11-21-2017 14:15:53] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 14:15:53] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 14:41:13] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 14:41:13] NPCD: npcd Daemon (0.4.14) started with PID=113553
[11-21-2017 14:41:13] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 14:41:13] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 14:46:01] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 14:46:01] NPCD: npcd Daemon (0.4.14) started with PID=117277
[11-21-2017 14:46:01] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 14:46:01] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 14:47:52] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 14:48:06] NPCD: npcd Daemon (0.4.14) started with PID=119065
[11-21-2017 14:48:06] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 14:48:06] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-21-2017 14:58:57] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-21-2017 14:58:57] NPCD: npcd Daemon (0.4.14) started with PID=124355
[11-21-2017 14:58:57] NPCD: Please have a look at 'npcd -V' to get license information
[11-21-2017 14:58:57] NPCD: HINT: load_threshold is enabled - ('10.000000')
[11-22-2017 08:57:18] NPCD: Caught Termination Signal - Hasta la vista... baby
[11-22-2017 08:57:18] NPCD: npcd Daemon (0.4.14) started with PID=73070
[11-22-2017 08:57:18] NPCD: Please have a look at 'npcd -V' to get license information
[11-22-2017 08:57:18] NPCD: HINT: load_threshold is enabled - ('10.000000')
tail -100 /usr/local/nagios/var/perfdataproc.log
Code: Select all
$: tail -100 /usr/local/nagios/var/perfdata.log
2017-02-22 11:59:52 [55498] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-22 11:59:52 [55499] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-22 11:59:52 [55498] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-22 11:59:52 [55499] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-22 11:59:52 [55498] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-22 11:59:52 [55499] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-22 11:59:52 [55498] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487782769.perfdata.host-PID-55498 deleted
2017-02-22 11:59:52 [55499] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487782769.perfdata.service-PID-55499 deleted
2017-02-22 11:59:52 [55498] [0] *** Timeout while processing Host: "" Service: ""
2017-02-22 11:59:52 [55499] [0] *** Timeout while processing Host: "" Service: ""
2017-02-22 11:59:52 [55498] [0] *** process_perfdata.pl terminated on signal ALRM
2017-02-22 11:59:52 [55499] [0] *** process_perfdata.pl terminated on signal ALRM
2017-02-23 01:07:41 [122913] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-23 01:07:41 [122913] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-23 01:07:41 [122913] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-23 01:07:41 [122912] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-23 01:07:41 [122913] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487830032.perfdata.service-PID-122913 deleted
2017-02-23 01:07:41 [122912] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-23 01:07:41 [122912] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-23 01:07:41 [122915] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-23 01:07:41 [122913] [0] *** Timeout while processing Host: "" Service: ""
2017-02-23 01:07:41 [122914] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-02-23 01:07:41 [122913] [0] *** process_perfdata.pl terminated on signal ALRM
2017-02-23 01:07:41 [122915] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-23 01:07:41 [122914] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-02-23 01:07:41 [122912] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487830032.perfdata.host-PID-122912 deleted
2017-02-23 01:07:41 [122915] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-23 01:07:41 [122914] [0] *** TIMEOUT: Please check your npcd.cfg
2017-02-23 01:07:41 [122912] [0] *** Timeout while processing Host: "" Service: ""
2017-02-23 01:07:41 [122915] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487830047.perfdata.service-PID-122915 deleted
2017-02-23 01:07:41 [122914] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1487830047.perfdata.host-PID-122914 deleted
2017-02-23 01:07:41 [122912] [0] *** process_perfdata.pl terminated on signal ALRM
2017-02-23 01:07:41 [122915] [0] *** Timeout while processing Host: "" Service: ""
2017-02-23 01:07:41 [122914] [0] *** Timeout while processing Host: "" Service: ""
2017-02-23 01:07:41 [122915] [0] *** process_perfdata.pl terminated on signal ALRM
2017-02-23 01:07:41 [122914] [0] *** process_perfdata.pl terminated on signal ALRM
2017-03-10 15:47:29 [22708] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-03-10 15:47:29 [22709] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-03-10 15:47:29 [22708] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-03-10 15:47:29 [22708] [0] *** TIMEOUT: Please check your npcd.cfg
2017-03-10 15:47:29 [22709] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-03-10 15:47:29 [22709] [0] *** TIMEOUT: Please check your npcd.cfg
2017-03-10 15:47:29 [22708] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1489178832.perfdata.host-PID-22708 deleted
2017-03-10 15:47:29 [22709] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1489178832.perfdata.service-PID-22709 deleted
2017-03-10 15:47:29 [22708] [0] *** Timeout while processing Host: "" Service: ""
2017-03-10 15:47:29 [22708] [0] *** process_perfdata.pl terminated on signal ALRM
2017-03-10 15:47:29 [22709] [0] *** Timeout while processing Host: "" Service: ""
2017-03-10 15:47:29 [22709] [0] *** process_perfdata.pl terminated on signal ALRM
2017-03-12 23:37:48 [89092] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-03-12 23:37:48 [89092] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-03-12 23:37:48 [89092] [0] *** TIMEOUT: Please check your npcd.cfg
2017-03-12 23:37:48 [89092] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1489376247.perfdata.host-PID-89092 deleted
2017-03-12 23:37:48 [89093] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-03-12 23:37:48 [89093] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-03-12 23:37:48 [89092] [0] *** Timeout while processing Host: "" Service: ""
2017-03-12 23:37:48 [89092] [0] *** process_perfdata.pl terminated on signal ALRM
2017-03-12 23:37:48 [89093] [0] *** TIMEOUT: Please check your npcd.cfg
2017-03-12 23:37:48 [89093] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1489376247.perfdata.service-PID-89093 deleted
2017-03-12 23:37:48 [89093] [0] *** Timeout while processing Host: "" Service: ""
2017-03-12 23:37:48 [89093] [0] *** process_perfdata.pl terminated on signal ALRM
2017-06-08 12:31:54 [57173] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1496939510.perfdata.service-PID-57173 deleted
2017-06-08 12:31:54 [57173] [0] *** Timeout while processing Host: "csdev61.sbc.com" Service: "Filespace_STAR-extra"
2017-06-08 12:31:54 [57173] [0] *** process_perfdata.pl terminated on signal BUS
2017-11-17 11:03:41 [22173] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-17 11:03:41 [22173] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-17 11:03:41 [22173] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-17 11:03:41 [22173] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1510783605.perfdata.service-PID-22173 deleted
2017-11-17 11:03:41 [22173] [0] *** Timeout while processing Host: "zlp22389.vci.att.com" Service: "Filespace_STAR-root"
2017-11-17 11:03:41 [22173] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-18 11:28:06 [128831] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-18 11:28:06 [128831] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-18 11:28:06 [128831] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-18 11:28:06 [128831] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511021791.perfdata.service-PID-128831 deleted
2017-11-18 11:28:06 [128831] [0] *** Timeout while processing Host: "hixa804.vtil.att.com" Service: "Filespace_STAR-opt-openv"
2017-11-18 11:28:06 [128831] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-20 11:55:05 [8739] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-20 11:55:05 [8741] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-20 11:55:05 [8739] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-20 11:55:05 [8741] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-20 11:55:05 [8739] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-20 11:55:05 [8741] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-20 11:55:05 [8739] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511041590.perfdata.service-PID-8739 deleted
2017-11-20 11:55:05 [8741] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511041561.perfdata.service-PID-8741 deleted
2017-11-20 11:55:05 [8739] [0] *** Timeout while processing Host: "hidh206.hydc.sbc.com" Service: "Filespace_STAR-opt-openv"
2017-11-20 11:55:05 [8741] [0] *** Timeout while processing Host: "hidh206.hydc.sbc.com" Service: "Filespace_STAR-opt"
2017-11-20 11:55:05 [8739] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-20 11:55:05 [8741] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-20 11:55:19 [9368] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-20 11:55:19 [9368] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-20 11:55:19 [9368] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-20 11:55:19 [9368] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511074594.perfdata.service-PID-9368 deleted
2017-11-20 11:55:19 [9368] [0] *** Timeout while processing Host: "cspz0423.sldc.sbc.com" Service: "Filespace_STAR-usr-local"
2017-11-20 11:55:19 [9368] [0] *** process_perfdata.pl terminated on signal ALRM
2017-11-20 14:33:12 [62146] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-11-20 14:33:12 [62146] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-11-20 14:33:12 [62146] [0] *** TIMEOUT: Please check your npcd.cfg
2017-11-20 14:33:12 [62146] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1511196902.perfdata.service-PID-62146 deleted
2017-11-20 14:33:12 [62146] [0] *** Timeout while processing Host: "hixa105.vtil.att.com" Service: "Filespace_STAR-root"
2017-11-20 14:33:12 [62146] [0] *** process_perfdata.pl terminated on signal ALRM
watch -n 1 'ls /usr/local/nagios/var/spool/checkresults | wc -l'
Minimum count 0 and maximum count 230
For this issue need to monitor npcd service ? to avoid this issue.
Re: checkresults issue
Posted: Wed Nov 22, 2017 3:57 pm
by dwhitfield
sandeepatil wrote:We are not using ramdisk with Nagios XI.
You should.
You were having a massive amount of load, but that could have very well been because of the inode issue. The load will
We haven't gotten the npcd status either time we asked, but given what we know right now, I think making sure npcd stays on is a good idea.
If you post the entire /usr/local/nagios/var/perfdataproc.log and /usr/local/nagios/var/npcd.log that will give us a better idea of what was going on when the inodes were full.
Re: checkresults issue
Posted: Tue Dec 26, 2017 10:10 am
by sandeepatil
Please find attached profile.zip file.
This server having checkresult directory issue.
Other details,
# ps -ef | grep perf
root 177 2 0 Dec21 ? 00:00:00 [perf]
nagios 87552 87539 0 10:04 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios 87555 87552 0 10:04 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios 88051 62937 0 10:04 pts/1 00:00:00 grep --color=auto perf
tail -100 /usr/local/nagios/var/npcd.log
[12-24-2017 07:02:02] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514116900.perfdata.service'
[12-24-2017 12:01:49] NPCD: ERROR: Executed command exits with return code '7'
[12-24-2017 12:01:49] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514134886.perfdata.service'
[12-24-2017 15:00:32] NPCD: ERROR: Executed command exits with return code '7'
[12-24-2017 15:00:32] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514145611.perfdata.service'
[12-25-2017 12:01:37] NPCD: ERROR: Executed command exits with return code '7'
[12-25-2017 12:01:37] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514221282.perfdata.service'
[12-26-2017 05:15:06] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:15:06] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283283.perfdata.service'
[12-26-2017 05:15:41] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:15:41] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283314.perfdata.service'
[12-26-2017 05:16:31] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:16:31] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283379.perfdata.service'
[12-26-2017 05:17:06] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:17:06] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283410.perfdata.service'
[12-26-2017 05:17:41] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:17:41] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283443.perfdata.service'
[12-26-2017 05:18:32] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 05:18:32] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514283497.perfdata.service'
[12-26-2017 08:30:07] NPCD: Caught Termination Signal - Hasta la vista... baby
[12-26-2017 08:30:07] NPCD: npcd Daemon (0.4.14) started with PID=54721
[12-26-2017 08:30:07] NPCD: Please have a look at 'npcd -V' to get license information
[12-26-2017 08:30:07] NPCD: HINT: load_threshold is enabled - ('10.000000')
[12-26-2017 08:40:32] NPCD: npcd Daemon (0.4.14) started with PID=58026
[12-26-2017 08:40:32] NPCD: Please have a look at 'npcd -V' to get license information
[12-26-2017 08:40:32] NPCD: HINT: load_threshold is enabled - ('10.000000')
[12-26-2017 08:44:41] NPCD: Caught Termination Signal - Hasta la vista... baby
[12-26-2017 09:43:37] NPCD: npcd Daemon (0.4.14) started with PID=73392
[12-26-2017 09:43:37] NPCD: Please have a look at 'npcd -V' to get license information
[12-26-2017 09:43:37] NPCD: HINT: load_threshold is enabled - ('10.000000')
[12-26-2017 09:45:42] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:45:42] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299523.perfdata.service'
[12-26-2017 09:46:18] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:46:18] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299555.perfdata.service'
[12-26-2017 09:46:38] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:46:38] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299586.perfdata.service'
[12-26-2017 09:47:13] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:47:13] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299617.perfdata.service'
[12-26-2017 09:47:43] NPCD: WARN: MAX load reached: load 10.310000/10.000000 at i=0
[12-26-2017 09:47:58] NPCD: WARN: MAX load reached: load 10.190000/10.000000 at i=1
[12-26-2017 09:48:33] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:48:33] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299681.perfdata.service'
[12-26-2017 09:48:53] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:48:53] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299712.perfdata.service'
[12-26-2017 09:48:53] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:48:53] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299714.perfdata.service'
[12-26-2017 09:49:28] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:49:28] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299753.perfdata.service'
[12-26-2017 09:49:48] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:49:48] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299777.perfdata.service'
[12-26-2017 09:50:08] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:50:08] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299793.perfdata.service'
[12-26-2017 09:50:28] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:50:28] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299810.perfdata.service'
[12-26-2017 09:50:48] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:50:48] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299827.perfdata.service'
[12-26-2017 09:50:48] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:50:48] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299840.perfdata.service'
[12-26-2017 09:51:08] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:51:08] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299854.perfdata.service'
[12-26-2017 09:51:28] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:51:28] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299866.perfdata.service'
[12-26-2017 09:51:28] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:51:28] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299879.perfdata.service'
[12-26-2017 09:51:48] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:51:48] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299892.perfdata.service'
[12-26-2017 09:52:08] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:52:08] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299918.perfdata.service'
[12-26-2017 09:52:08] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:52:08] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299904.perfdata.service'
[12-26-2017 09:52:59] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:52:59] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514299952.perfdata.service'
[12-26-2017 09:54:34] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:54:34] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300053.perfdata.service'
[12-26-2017 09:54:54] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:54:54] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300079.perfdata.service'
[12-26-2017 09:54:54] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:54:54] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300067.perfdata.service'
[12-26-2017 09:55:14] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:55:14] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300099.perfdata.service'
[12-26-2017 09:55:14] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:55:14] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300091.perfdata.service'
[12-26-2017 09:55:34] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:55:34] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300112.perfdata.service'
[12-26-2017 09:56:14] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:56:14] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300142.perfdata.service'
[12-26-2017 09:56:14] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:56:14] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300157.perfdata.service'
[12-26-2017 09:56:34] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:56:34] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300173.perfdata.service'
[12-26-2017 09:56:54] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:56:54] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300187.perfdata.service'
[12-26-2017 09:57:34] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:57:34] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300232.perfdata.service'
[12-26-2017 09:58:10] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:58:10] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300262.perfdata.service'
[12-26-2017 09:58:10] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:58:10] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300277.perfdata.service'
[12-26-2017 09:58:30] NPCD: ERROR: Executed command exits with return code '7'
[12-26-2017 09:58:30] NPCD: ERROR: Command line was '/usr/local/nagios/libexec/process_perfdata.pl -n -b /usr/local/nagios/var/spool/perfdata//1514300293.perfdata.service'
tail -100 /usr/local/nagios/var/perfdataproc.log
2017-12-26 09:52:08 [77606] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:52:08 [77606] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514299918.perfdata.service-PID-77606 deleted
2017-12-26 09:52:08 [77606] [0] *** Timeout while processing Host: "blsd001.bhdc.att.com" Service: "Filespace_STAR-opt-tools"
2017-12-26 09:52:08 [77606] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:52:08 [77603] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:52:08 [77603] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:52:08 [77603] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:52:08 [77603] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514299904.perfdata.service-PID-77603 deleted
2017-12-26 09:52:08 [77603] [0] *** Timeout while processing Host: "ulpv0691.madc.att.com" Service: "CPU_STAR-Composite"
2017-12-26 09:52:08 [77603] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:52:58 [77907] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:52:58 [77907] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:52:58 [77907] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:52:58 [77907] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514299952.perfdata.service-PID-77907 deleted
2017-12-26 09:52:58 [77907] [0] *** Timeout while processing Host: "flth058.ffdc.sbc.com" Service: "Filespace_STAR-usr"
2017-12-26 09:52:58 [77907] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:54:34 [78655] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:54:34 [78655] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:54:34 [78655] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:54:34 [78655] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300053.perfdata.service-PID-78655 deleted
2017-12-26 09:54:34 [78655] [0] *** Timeout while processing Host: "zlt00268.vci.att.com" Service: "Filespace_STAR-boot"
2017-12-26 09:54:34 [78655] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:54:54 [78787] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:54:54 [78787] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:54:54 [78787] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:54:54 [78787] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300079.perfdata.service-PID-78787 deleted
2017-12-26 09:54:54 [78787] [0] *** Timeout while processing Host: "zldv7321.vci.att.com" Service: "Filespace_STAR-var"
2017-12-26 09:54:54 [78787] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:54:54 [78788] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:54:54 [78788] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:54:54 [78788] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:54:54 [78788] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300067.perfdata.service-PID-78788 deleted
2017-12-26 09:54:54 [78788] [0] *** Timeout while processing Host: "zldv4297.vci.att.com" Service: "Filespace_STAR-var"
2017-12-26 09:54:54 [78788] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:55:14 [79066] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:55:14 [79066] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:55:14 [79066] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:55:14 [79066] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300099.perfdata.service-PID-79066 deleted
2017-12-26 09:55:14 [79066] [0] *** Timeout while processing Host: "hltv0173.hydc.sbc.com" Service: "Filespace_STAR-usr"
2017-12-26 09:55:14 [79066] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:55:14 [79064] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:55:14 [79064] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:55:14 [79064] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:55:14 [79064] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300091.perfdata.service-PID-79064 deleted
2017-12-26 09:55:14 [79064] [0] *** Timeout while processing Host: "pldv0075.pedc.sbc.com" Service: "Filespace_STAR-boot"
2017-12-26 09:55:14 [79064] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:55:34 [79252] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:55:34 [79252] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:55:34 [79252] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:55:34 [79252] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300112.perfdata.service-PID-79252 deleted
2017-12-26 09:55:34 [79252] [0] *** Timeout while processing Host: "zlt10805.vci.att.com" Service: "Filespace_STAR-usr"
2017-12-26 09:55:34 [79252] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:56:14 [79576] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:56:14 [79576] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:56:14 [79576] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:56:14 [79576] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300142.perfdata.service-PID-79576 deleted
2017-12-26 09:56:14 [79576] [0] *** Timeout while processing Host: "zlt12893.vci.att.com" Service: "Filespace_STAR-home"
2017-12-26 09:56:14 [79576] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:56:14 [79577] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:56:14 [79577] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:56:14 [79577] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:56:14 [79577] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300157.perfdata.service-PID-79577 deleted
2017-12-26 09:56:14 [79577] [0] *** Timeout while processing Host: "zltv4437.vci.att.com" Service: "Filespace_STAR-home"
2017-12-26 09:56:14 [79577] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:56:34 [79892] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:56:34 [79892] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:56:34 [79892] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:56:34 [79892] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300173.perfdata.service-PID-79892 deleted
2017-12-26 09:56:34 [79892] [0] *** Timeout while processing Host: "cltv0309.sldc.sbc.com" Service: "Filespace_STAR-boot"
2017-12-26 09:56:34 [79892] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:56:54 [80035] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:56:54 [80035] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:56:54 [80035] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:56:54 [80035] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300187.perfdata.service-PID-80035 deleted
2017-12-26 09:56:54 [80035] [0] *** Timeout while processing Host: "flsv0074.ffdc.sbc.com" Service: "CPU_STAR-Composite"
2017-12-26 09:56:54 [80035] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:57:34 [80406] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:57:34 [80406] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:57:34 [80406] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:57:34 [80406] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300232.perfdata.service-PID-80406 deleted
2017-12-26 09:57:34 [80406] [0] *** Timeout while processing Host: "zlp17969.vci.att.com" Service: "Filespace_STAR-home"
2017-12-26 09:57:34 [80406] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:58:10 [80752] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:58:10 [80752] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:58:10 [80752] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:58:10 [80752] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300262.perfdata.service-PID-80752 deleted
2017-12-26 09:58:10 [80752] [0] *** Timeout while processing Host: "cltv0322.sldc.sbc.com" Service: "Filespace_STAR-usr"
2017-12-26 09:58:10 [80752] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:58:10 [80753] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:58:10 [80753] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:58:10 [80753] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:58:10 [80753] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300277.perfdata.service-PID-80753 deleted
2017-12-26 09:58:10 [80753] [0] *** Timeout while processing Host: "pldi064.pedc.sbc.com" Service: "Filespace_STAR-var-crash"
2017-12-26 09:58:10 [80753] [0] *** process_perfdata.pl terminated on signal ALRM
2017-12-26 09:58:30 [81604] [0] *** TIMEOUT: Timeout after 5 secs. ***
2017-12-26 09:58:30 [81604] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-12-26 09:58:30 [81604] [0] *** TIMEOUT: Please check your npcd.cfg
2017-12-26 09:58:30 [81604] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1514300293.perfdata.service-PID-81604 deleted
2017-12-26 09:58:30 [81604] [0] *** Timeout while processing Host: "ulpd475.madc.att.com" Service: "Filespace_STAR-root"
2017-12-26 09:58:30 [81604] [0] *** process_perfdata.pl terminated on signal ALRM
Re: checkresults issue
Posted: Wed Dec 27, 2017 11:41 am
by kyang
Thanks for sharing this information.
I shared your profile with the other techs!
Could you try increasing the
TIMEOUT value in the
process_perfdata.pl file?
Code: Select all
/usr/local/nagios/etc/pnp/process_perfdata.pl
Code: Select all
TIMEOUT = 5
Change it to
TIMEOUT = 50
Let us know how that goes.
Re: checkresults issue
Posted: Wed Dec 27, 2017 1:12 pm
by sandeepatil
Will configure as suggested and update you if facing any issue.