Custom Plugin fails with "Service Check Timed Out"
Posted: Fri Apr 05, 2019 10:23 am
A custom plugin I've written for nagios core is failing with status Critical - (Service Check Timed Out). The plugin loads a shiny app and reports any errors it finds on the page. It runs and returns the appropriate exit code ("1" in the example below) in ~23 seconds when run as the nagios user from the command line. service_check_timeout is set to 60. When I enable the plugin in the nagios config, it shows this failure on the nagios service dashboard. Why is it timing out on the dash and not the command line?
Nagios Core Version 3.5.1
Nagios Core Version 3.5.1
Shiny App Contents - Error Tracking CRITICAL 2019-04-05 11:09:03 3d 23h 17m 31s 4/4 (Service Check Timed Out)
Code: Select all
define command{
command_name check_shinycontents
command_line /usr/lib/nagios/plugins/check_appshot $ARGS1$
}
Code: Select all
# Define a service to check for errors within shiny apps
define service{
use long-interval-service
host_name localhost
service_description Shiny App Contents - Error Tracking
check_command check_shinycontents!error_tracking
}
Code: Select all
nagios@hostname:~$ time /usr/lib/nagios/plugins/check_appshot error_tracking
WARNING - "invalid first argument"
real 0m23.564s
user 0m2.751s
sys 0m0.611s
nagios@hostname:~$ /usr/lib/nagios/plugins/check_appshot error_tracking
WARNING - "invalid first argument"
nagios@hostname:~$ echo $?
1
Code: Select all
user@hostname:~$ sudo grep -r timeout /etc/nagios3/
[i]/etc/nagios3/nagios.cfg:service_check_timeout=60
[/i]/etc/nagios3/nagios.cfg:host_check_timeout=30
/etc/nagios3/nagios.cfg:event_handler_timeout=30
/etc/nagios3/nagios.cfg:notification_timeout=30
/etc/nagios3/nagios.cfg:ocsp_timeout=5
/etc/nagios3/nagios.cfg:perfdata_timeout=5
/etc/nagios3/nagios.cfg:service_check_timeout_state=c