Plugin timed out while executing system call error

[LittleNetworker]

I have a strange issue with check using check_snmp. The status information returned is "CRITICAL - Plugin timed out while executing system call" however when I use snmpget or run check_snmp on the Nagios server's CLI it returns the proper information. I have tried increasing the timeout for this specific check to 30 and 60 seconds but it only changes the content of the error returned. Any help is greatly appreciated!

Code: Select all

[root@jumbo objects]# snmpget -v2c -c XXX 172.16.252.17 1.3.6.1.4.1.3808.1.1.1.2.2.4.0
SNMPv2-SMI::enterprises.3808.1.1.1.2.2.4.0 = Timeticks: (5994000) 16:39:00.00

Code: Select all

[root@jumbo objects]# ../../libexec/check_snmp -C XXX -H 172.16.252.17 -o 1.3.6.1.4.1.3808.1.1.1.2.2.4.0
SNMP OK - 5994000 | iso.3.6.1.4.1.3808.1.1.1.2.2.4.0=5994000 

Code: Select all

check_command check_snmp! -C XXX -o .1.3.6.1.4.1.3808.1.1.1.2.2.4.0 -c 60: -m all

Current Status: CRITICAL (for 3d 22h 45m 18s) (Has been acknowledged)
Status Information: CRITICAL - Plugin timed out while executing system call

Thank you!!

[mcapra]

I have tried increasing the timeout for this specific check to 30 and 60 seconds but it only changes the content of the error returned. Any help is greatly appreciated!

If the plugin is taking longer than 60 seconds to complete it's run, you might need to tune your Nagios configurations to account for that. There are some directives in the main nagios.cfg file that determine what Nagios Core itself recognizes as a plugin that has run for too long:

Code: Select all

[root@nagios objects]# cat /usr/local/nagios/etc/nagios.cfg | grep check_timeout=
service_check_timeout=60
host_check_timeout=30

So with the above configurations, if a service check runs longer than 60 seconds, Nagios Core will attempt to terminate it regardless of what you're telling the plugin it's "timeout value" is.

[LittleNetworker]

Thanks for the info. I checked my timeouts and they match what you posted. I should note (I forgot to add this to the original message and have since edited it) that running the command via check_snmp on the Nagios' server CLI returns a proper result similar to running snmpget. I'm not sure if that makes a difference. Thanks!

[tgriep]

Edit the check_snmp command and add the following to the command.

Code: Select all

-t 60

That will increase the timeout of the command and hopefully that will fix the issue.
When you see the timeout error, how long did it run before the timeout?

[LittleNetworker]

Hi tgriep,
Thanks for the suggestion, here's what happened:
Updated the check command:

Code: Select all

check_command check_snmp! -C **** -o .1.3.6.1.4.1.3808.1.1.1.2.2.4.0 -c 60: -m all -t 60

Reply via Nagios: "Status Information: (Service check timed out after 60.02 seconds)"

Reply via Nagios' server CLI: SNMP OK - 5994000 | iso.3.6.1.4.1.3808.1.1.1.2.2.4.0=5994000

[tgriep]

The "-m all" option forces the check_snmp to go through all of the MIB files on the system and maybe that is still taking longer than 60 seconds,
Try removing it and see if that helps.
Can you post the check_snmp command_line that is setup in the commands.cfg file so I can check it to see if it it correct?

[LittleNetworker]

The "-m all" option forces the check_snmp to go through all of the MIB files on the system and maybe that is still taking longer that 60 seconds,
Try removing it and see if that helps.

Unfortunately I get the same results on GUI & CLI without the "-m all" option too

Can you post the check_snmp command_line that is setup in the commands.cfg file so I can check it to see if it it correct?

Here you go:

Code: Select all

define command{
        command_name    check_snmp
        command_line    $USER1$/check_snmp -H $HOSTADDRESS$ $ARG1$
        }

Thanks!

[tgriep]

What you could do is edit the nagios.cfg file, enable debugging and set the debug_level to 16 so it logs the host and service checks and see what the command nagios is executing.
Take a look at this link for more details on that option.
https://assets.nagios.com/downloads/nag ... gmain.html

The only thing left is that the check is running against the wrong host.

[wschwarz]

I have the same problem as you, however I have found that my system call works when ran as root but does not work (gives the plugin timed out message) when being ran as the nagios user.

Have you tried running this on the command line as nagios? I suspect that is your problem. I don't know how to fix that though, working through it myself currently.

[dwhitfield]

The output of ls -la /usr/local/nagios/libexec may give us a clue. Please put the output in a code block. The "Code" button is the fifth from the left on the post input screen (between Quote and List).