Checks perpetually pending

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
linuxnerd
Posts: 13
Joined: Mon Jul 02, 2018 12:57 am

Checks perpetually pending

Post by linuxnerd »

Some of my checks are in a perpetually pending state.
Even when I force a check, nothing happens.

when I force the check, i sometimes see in the log
[1531367787] EXTERNAL COMMAND: SCHEDULE_FORCED_HOST_SVC_CHECKS;xxxxxxx;1531367811
but nothing else pertaining to that host.

My config:
https://support.nagios.com/forum/viewto ... 14#p255589
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Checks perpetually pending

Post by scottwilkerson »

Make sure active checks are enabled for the failing services

Go to Service Information page for the service
Bottom center it should say
Active Checks: ENABLED
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
linuxnerd
Posts: 13
Joined: Mon Jul 02, 2018 12:57 am

Re: Checks perpetually pending

Post by linuxnerd »

they are active.
the checks did eventually run, but over 20 hours later.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Checks perpetually pending

Post by tgriep »

Can you run the following commands as root on your Nagios server and post the output?

Code: Select all

/usr/local/nagios/bin/nagiostats
ps -ef
I want to see the Nagios Stats and the processes that are running on the server.

Thanks
Be sure to check out our Knowledgebase for helpful articles and solutions!
linuxnerd
Posts: 13
Joined: Mon Jul 02, 2018 12:57 am

Re: Checks perpetually pending

Post by linuxnerd »

Code: Select all

Nagios Stats 3.5.1
Copyright (c) 2003-2008 Ethan Galstad (www.nagios.org)
Last Modified: 08-30-2013
License: GPL

CURRENT STATUS DATA
------------------------------------------------------
Status File:                            /var/cache/nagios3/status.dat
Status File Age:                        0d 0h 0m 8s
Status File Version:                    3.5.1

Program Running Time:                   1d 15h 28m 38s
Nagios PID:                             3064
Used/High/Total Command Buffers:        0 / 2 / 16384

Total Services:                         1613
Services Checked:                       1613
Services Scheduled:                     1613
Services Actively Checked:              1613
Services Passively Checked:             0
Total Service State Change:             0.000 / 12.370 / 0.125 %
Active Service Latency:                 0.002 / 5.224 / 0.140 sec
Active Service Execution Time:          0.009 / 23.779 / 1.213 sec
Active Service State Change:            0.000 / 12.370 / 0.125 %
Active Services Last 1/5/15/60 min:     55 / 549 / 1371 / 1390
Passive Service Latency:                0.000 / 0.000 / 0.000 sec
Passive Service State Change:           0.000 / 0.000 / 0.000 %
Passive Services Last 1/5/15/60 min:    0 / 0 / 0 / 0
Services Ok/Warn/Unk/Crit:              1611 / 0 / 0 / 2
Services Flapping:                      0
Services In Downtime:                   0

Total Hosts:                            237
Hosts Checked:                          237
Hosts Scheduled:                        237
Hosts Actively Checked:                 237
Host Passively Checked:                 0
Total Host State Change:                0.000 / 0.000 / 0.000 %
Active Host Latency:                    0.000 / 5.604 / 3.273 sec
Active Host Execution Time:             0.007 / 0.144 / 0.049 sec
Active Host State Change:               0.000 / 0.000 / 0.000 %
Active Hosts Last 1/5/15/60 min:        0 / 30 / 237 / 237
Passive Host Latency:                   0.000 / 0.000 / 0.000 sec
Passive Host State Change:              0.000 / 0.000 / 0.000 %
Passive Hosts Last 1/5/15/60 min:       0 / 0 / 0 / 0
Hosts Up/Down/Unreach:                  237 / 0 / 0
Hosts Flapping:                         0
Hosts In Downtime:                      0

Active Host Checks Last 1/5/15 min:     210 / 237 / 695
   Scheduled:                           210 / 237 / 694
   On-demand:                           0 / 0 / 1
   Parallel:                            210 / 237 / 696
   Serial:                              0 / 0 / 0
   Cached:                              0 / 0 / 0
Passive Host Checks Last 1/5/15 min:    0 / 0 / 0
Active Service Checks Last 1/5/15 min:  115 / 689 / 2125
   Scheduled:                           115 / 689 / 2125
   On-demand:                           0 / 0 / 0
   Cached:                              0 / 0 / 0
Passive Service Checks Last 1/5/15 min: 0 / 0 / 0

External Commands Last 1/5/15 min:      0 / 0 / 0

Code: Select all

root         1     0  0 Jul03 ?        00:02:38 /sbin/init text text
root         2     0  0 Jul03 ?        00:00:00 [kthreadd]
root         3     2  0 Jul03 ?        00:00:10 [ksoftirqd/0]
root         5     2  0 Jul03 ?        00:00:00 [kworker/0:0H]
root         7     2  0 Jul03 ?        00:07:17 [rcu_sched]
root         8     2  0 Jul03 ?        00:00:00 [rcu_bh]
root         9     2  0 Jul03 ?        00:00:33 [migration/0]
root        10     2  0 Jul03 ?        00:00:03 [watchdog/0]
root        11     2  0 Jul03 ?        00:00:03 [watchdog/1]
root        12     2  0 Jul03 ?        00:00:34 [migration/1]
root        13     2  0 Jul03 ?        00:00:12 [ksoftirqd/1]
root        15     2  0 Jul03 ?        00:00:00 [kworker/1:0H]
root        16     2  0 Jul03 ?        00:00:02 [watchdog/2]
root        17     2  0 Jul03 ?        00:00:31 [migration/2]
root        18     2  0 Jul03 ?        00:00:13 [ksoftirqd/2]
root        20     2  0 Jul03 ?        00:00:00 [kworker/2:0H]
root        21     2  0 Jul03 ?        00:00:03 [watchdog/3]
root        22     2  0 Jul03 ?        00:00:33 [migration/3]
root        23     2  0 Jul03 ?        00:00:18 [ksoftirqd/3]
root        25     2  0 Jul03 ?        00:00:00 [kworker/3:0H]
root        26     2  0 Jul03 ?        00:00:00 [kdevtmpfs]
root        27     2  0 Jul03 ?        00:00:00 [netns]
root        28     2  0 Jul03 ?        00:00:00 [perf]
root        29     2  0 Jul03 ?        00:00:00 [khungtaskd]
root        30     2  0 Jul03 ?        00:00:00 [writeback]
root        31     2  0 Jul03 ?        00:00:00 [ksmd]
root        32     2  0 Jul03 ?        00:00:13 [khugepaged]
root        33     2  0 Jul03 ?        00:00:00 [crypto]
root        34     2  0 Jul03 ?        00:00:00 [kintegrityd]
root        35     2  0 Jul03 ?        00:00:00 [bioset]
root        36     2  0 Jul03 ?        00:00:00 [kblockd]
root        37     2  0 Jul03 ?        00:00:00 [ata_sff]
root        38     2  0 Jul03 ?        00:00:00 [md]
root        39     2  0 Jul03 ?        00:00:00 [devfreq_wq]
root        45     2  0 Jul03 ?        00:00:10 [kswapd0]
root        46     2  0 Jul03 ?        00:00:00 [vmstat]
root        47     2  0 Jul03 ?        00:00:00 [fsnotify_mark]
root        48     2  0 Jul03 ?        00:00:00 [ecryptfs-kthrea]
root        64     2  0 Jul03 ?        00:00:00 [kthrotld]
root        65     2  0 Jul03 ?        00:00:00 [acpi_thermal_pm]
root        66     2  0 Jul03 ?        00:00:00 [bioset]
root        67     2  0 Jul03 ?        00:00:00 [bioset]
root        68     2  0 Jul03 ?        00:00:00 [bioset]
root        69     2  0 Jul03 ?        00:00:00 [bioset]
root        70     2  0 Jul03 ?        00:00:00 [bioset]
root        71     2  0 Jul03 ?        00:00:00 [bioset]
root        72     2  0 Jul03 ?        00:00:00 [bioset]
root        73     2  0 Jul03 ?        00:00:00 [bioset]
root        74     2  0 Jul03 ?        00:00:00 [scsi_eh_0]
root        75     2  0 Jul03 ?        00:00:00 [scsi_tmf_0]
root        76     2  0 Jul03 ?        00:00:00 [scsi_eh_1]
root        77     2  0 Jul03 ?        00:00:00 [scsi_tmf_1]
root        83     2  0 Jul03 ?        00:00:00 [ipv6_addrconf]
root        96     2  0 Jul03 ?        00:00:00 [deferwq]
root        97     2  0 Jul03 ?        00:00:00 [charger_manager]
root        98     2  0 Jul03 ?        00:00:00 [bioset]
root       143     2  0 Jul03 ?        00:00:00 [bioset]
root       144     2  0 Jul03 ?        00:00:00 [bioset]
root       145     2  0 Jul03 ?        00:00:00 [bioset]
root       146     2  0 Jul03 ?        00:00:00 [bioset]
root       147     2  0 Jul03 ?        00:00:00 [bioset]
root       148     2  0 Jul03 ?        00:00:00 [bioset]
root       149     2  0 Jul03 ?        00:00:00 [bioset]
root       150     2  0 Jul03 ?        00:00:00 [bioset]
root       151     2  0 Jul03 ?        00:00:00 [scsi_eh_2]
root       152     2  0 Jul03 ?        00:00:00 [scsi_tmf_2]
root       153     2  0 Jul03 ?        00:00:00 [vmw_pvscsi_wq_2]
root       154     2  0 Jul03 ?        00:00:00 [bioset]
root       155     2  0 Jul03 ?        00:00:00 [bioset]
root       172     2  0 Jul03 ?        00:00:00 [kpsmoused]
root       173     2  0 Jul03 ?        00:00:00 [kworker/2:1H]
root       174     2  0 Jul03 ?        00:00:00 [ttm_swap]
root       204     2  0 Jul03 ?        00:00:00 [raid5wq]
root       221     2  0 Jul03 ?        00:00:00 [kdmflush]
root       224     2  0 Jul03 ?        00:00:00 [bioset]
root       233     2  0 Jul03 ?        00:00:00 [kworker/0:1H]
root       250     2  0 Jul03 ?        00:00:00 [jbd2/dm-0-8]
root       251     2  0 Jul03 ?        00:00:00 [ext4-rsv-conver]
root       294     2  0 Jul03 ?        00:00:00 [kworker/1:1H]
root       295     1  0 Jul03 ?        00:01:01 /lib/systemd/systemd-journald
root       315     2  0 Jul03 ?        00:00:00 [kauditd]
root       329     2  0 Jul03 ?        00:00:00 [rpciod]
root       340     1  0 Jul03 ?        00:00:00 /sbin/lvmetad -f
root       347     1  0 Jul03 ?        00:00:01 /lib/systemd/systemd-udevd
root       440     2  0 Jul03 ?        00:00:00 [kdmflush]
root       441     2  0 Jul03 ?        00:00:00 [bioset]
root       445     2  0 Jul03 ?        00:00:00 [kdmflush]
root       446     2  0 Jul03 ?        00:00:00 [bioset]
root       545     2  0 Jul03 ?        00:00:00 [jbd2/sda1-8]
root       546     2  0 Jul03 ?        00:00:00 [ext4-rsv-conver]
root       551     2  0 Jul03 ?        00:00:36 [jbd2/dm-2-8]
root       552     2  0 Jul03 ?        00:00:00 [ext4-rsv-conver]
root       573     1  0 Jul03 ?        00:11:01 /usr/bin/vmtoolsd
root       752     1  0 Jul03 ?        00:00:17 /lib/systemd/systemd-logind
message+   754     1  0 Jul03 ?        00:00:35 /usr/bin/dbus-daemon --system --address=systemd: --nofork -
root       781     1  0 Jul03 ?        00:00:27 /usr/lib/accountsservice/accounts-daemon
root       795     1  0 Jul03 ?        00:00:05 /usr/sbin/cron -f
root       802     1  0 Jul03 ?        00:00:00 /usr/bin/VGAuthService
root       842     1  0 Jul03 ?        00:00:03 /usr/sbin/sshd -D
root       891     1  0 Jul03 ?        00:00:39 /usr/sbin/irqbalance --pid=/var/run/irqbalance.pid
root       941     1  0 Jul03 tty1     00:00:00 /sbin/agetty --noclear tty1 linux
ntp        962     1  0 Jul03 ?        00:00:45 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 109:116
root       963     1  0 Jul03 ?        00:00:08 /usr/lib/policykit-1/polkitd --no-debug
root       987     2  0 01:06 ?        00:00:00 [kworker/3:2]
snmp      1003     1  0 Jul03 ?        00:06:19 /usr/sbin/snmpd -Lsd -Lf /dev/null -u snmp -g snmp -I -smux
root      1085     2  0 Jul03 ?        00:00:01 [kworker/3:1H]
root      1091     1  0 Jul03 ?        00:00:07 /usr/lib/postfix/sbin/master
postfix   1093  1091  0 Jul03 ?        00:00:02 qmgr -l -t unix -u
root      1361     1 44 01:06 ?        00:14:00 /bin/bash /usr/local/bin/bulk_ocsp
root      1363     2  0 01:06 ?        00:00:00 [kworker/1:1]
nagios    3064     1  0 Jul11 ?        00:26:17 /usr/sbin/nagios3 -x -d /etc/nagios3/nagios.cfg
root      4191 17757  0 01:33 ?        00:00:00 sleep 764s
root      6503     2  0 Jul13 ?        00:00:00 [kworker/2:0]
root      8111   842  0 01:36 ?        00:00:00 sshd: XXX [priv]
XXX       8113     1  0 01:36 ?        00:00:00 /lib/systemd/systemd --user
root      8114     2  0 01:36 ?        00:00:00 [kworker/2:1]
root      8118     2  0 01:36 ?        00:00:00 [kworker/1:2]
XXX       8120  8113  0 01:36 ?        00:00:00 (sd-pam)
XXX       8301  8111  0 01:36 ?        00:00:00 sshd: XXX@pts/0
XXX       8320  8301  0 01:36 pts/0    00:00:00 -bash
root      8525     2  0 Jul13 ?        00:00:00 [kworker/1:0]
root     10062     2  0 01:38 ?        00:00:00 [kworker/u8:2]
XXX      10374  8320  0 01:38 pts/0    00:00:00 ps -ef
postfix  12994  1091  0 00:51 ?        00:00:00 pickup -l -t unix -u -c
root     16201     2  0 Jul13 ?        00:00:02 [kworker/3:1]
root     16203     2  0 Jul13 ?        00:00:00 [kworker/0:2]
syslog   16231     1  0 Jul12 ?        00:00:12 /usr/sbin/rsyslogd -n
root     20220     1  0 Jul08 ?        00:00:17 /usr/sbin/apache2 -k start
www-data 23553 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 23554 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 23555 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 23556 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
www-data 23557 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
root     28603     2  0 01:27 ?        00:00:00 [kworker/u8:0]
www-data 28655 20220  0 Jul13 ?        00:00:00 /usr/sbin/apache2 -k start
root     29174     2  0 Jul13 ?        00:00:00 [kworker/2:2]
root     29175     2  0 Jul13 ?        00:00:00 [kworker/0:1]
root     31196     2  0 01:05 ?        00:00:00 [kworker/u8:1]
nagios   32513     1  0 Jul09 ?        00:00:02 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d
linuxnerd
Posts: 13
Joined: Mon Jul 02, 2018 12:57 am

Re: Checks perpetually pending

Post by linuxnerd »

i think i forgot to reload the service.
the service check wasn't present into the objects.dat file.
after repushing my config / reloading, it seems to be running.
jomann
Development Lead
Posts: 611
Joined: Mon Apr 22, 2013 10:06 am
Location: Nagios Enterprises

Re: Checks perpetually pending

Post by jomann »

So are the checks happening now? Is this issue resolved?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
linuxnerd
Posts: 13
Joined: Mon Jul 02, 2018 12:57 am

Re: Checks perpetually pending

Post by linuxnerd »

i suppose so.
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Checks perpetually pending

Post by tmcdonald »

Did you have further (related) questions or are we good to lock this up?
Former Nagios employee
Locked