I stumbled on what I think is a bug while developing a NEB similar to mod_gearman but using RabbitMQ's AMQP broker instead of a Gearman broker: whenever I trigger a forced check — via a Thruk web interface, for instance — to be executed, Nagios executes it as intended but it also schedules it to be executed periodically along the originally scheduled check, instead of executing it just once.
Here is a log trace of the bug: I defined a host 'localhost' and a service 'Disk Space' scheduled to be ran every 10 seconds — which is the case —, however when I run a forced check at 13:49:59, Nagios seems to have it added to its scheduled checks because it is then ran every 10 seconds, 2 seconds after the "original" check):
(Note: this behaviour was observed with a stock Nagios 3.4.1 (on Linux Ubuntu), without my NEB used)
Code: Select all
# tail -f /var/log/nagios3/nagios.debug | grep -B1 'Service Check Event'
[1382363307.225777] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:48:27 2013
[1382363307.225793] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.225000 sec
--
[1382363317.030203] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:48:37 2013
[1382363317.030220] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.030000 sec
--
[1382363327.082986] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:48:47 2013
[1382363327.083001] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.083000 sec
--
[1382363337.134623] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:48:57 2013
[1382363337.134628] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.134000 sec
--
[1382363347.187396] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:07 2013
[1382363347.187412] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.187000 sec
--
[1382363357.241644] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:17 2013
[1382363357.241661] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.241000 sec
--
[1382363367.042537] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:27 2013
[1382363367.042551] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.042000 sec
--
[1382363377.091861] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:37 2013
[1382363377.091877] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.091000 sec
--
[1382363387.140221] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:47 2013
[1382363387.140235] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.140000 sec
--
[1382363397.184816] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:57 2013
[1382363397.184838] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.184000 sec
--
[1382363400.447903] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:49:59 2013
[1382363400.447910] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 1, Latency: 1.447000 sec
--
[1382363407.237351] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:07 2013
[1382363407.237367] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.237000 sec
--
[1382363410.249532] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:10 2013
[1382363410.249647] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.249000 sec
--
[1382363417.039990] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:17 2013
[1382363417.040006] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.040000 sec
--
[1382363420.059367] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:20 2013
[1382363420.059398] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.059000 sec
--
[1382363427.098155] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:27 2013
[1382363427.098170] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.098000 sec
--
[1382363430.112910] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:30 2013
[1382363430.112926] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.112000 sec
--
[1382363437.153421] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:37 2013
[1382363437.153437] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.153000 sec
--
[1382363440.168974] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:40 2013
[1382363440.169018] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.169000 sec
--
[1382363447.209418] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:47 2013
[1382363447.209432] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.209000 sec
--
[1382363450.225964] [008.0] [pid=5638] ** Timed Event ** Type: 0, Run Time: Mon Oct 21 13:50:50 2013
[1382363450.225981] [008.0] [pid=5638] ** Service Check Event ==> Host: 'localhost', Service: 'Disk Space', Options: 0, Latency: 0.225000 sec
Let me know if you need more information on my setup.
Regards,
m.