CHECK_BY_SSH intermittant

[redeming]

What exactly does "Return code of 255 for check of service" mean?

Here are some of the messages from the log. Please note that this is from a small time period and not representative of all systems and all services where we are seeing this issue.

Code: Select all

[1393574903] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG6015' was out of bounds.
[1393574903] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG3013' was out of bounds.
[1393574903] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG3013' was out of bounds.
[1393574953] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG6015' was out of bounds.
[1393574953] Warning: Return code of 255 for check of service 'Unix - Check Memory' on host 'ISCCAG3013' was out of bounds.
[1393575043] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG6015' was out of bounds.
[1393575123] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG3013' was out of bounds.
[1393575153] Warning: Return code of 255 for check of service 'Unix - Check Root File System' on host 'ISCCAG3013' was out of bounds.
[1393575163] Warning: Return code of 255 for check of service 'Unix - Check Processes' on host 'ISCCAG6015' was out of bounds.
[1393575203] Warning: Return code of 255 for check of service 'Unix - Check Processes' on host 'ISCCAG3013' was out of bounds.
[1393575253] Warning: Return code of 255 for check of service 'Unix - Check Var Directory' on host 'ISCCAG3013' was out of bounds.
[1393575333] Warning: Return code of 255 for check of service 'Unix - Check Var Directory' on host 'ISCCAG6015' was out of bounds.
[1393575333] Warning: Return code of 255 for check of service 'Unix - Check Memory' on host 'ISCCAG6015' was out of bounds.
[1393575453] Warning: Return code of 255 for check of service 'Unix - Check Root File System' on host 'ISCCAG6015' was out of bounds.
[1393575503] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG6015' was out of bounds.
[1393575503] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG3013' was out of bounds.
[1393575503] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG3013' was out of bounds.
[1393575553] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG6015' was out of bounds.
[1393575553] Warning: Return code of 255 for check of service 'Unix - Check Memory' on host 'ISCCAG3013' was out of bounds.
[1393575643] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG6015' was out of bounds.
[1393575723] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG3013' was out of bounds.
[1393575753] Warning: Return code of 255 for check of service 'Unix - Check Root File System' on host 'ISCCAG3013' was out of bounds.
[1393575763] Warning: Return code of 255 for check of service 'Unix - Check Processes' on host 'ISCCAG6015' was out of bounds.
[1393575803] Warning: Return code of 255 for check of service 'Unix - Check Processes' on host 'ISCCAG3013' was out of bounds.
[1393575853] Warning: Return code of 255 for check of service 'Unix - Check Var Directory' on host 'ISCCAG3013' was out of bounds.
[1393575933] Warning: Return code of 255 for check of service 'Unix - Check Var Directory' on host 'ISCCAG6015' was out of bounds.
[1393575933] Warning: Return code of 255 for check of service 'Unix - Check Memory' on host 'ISCCAG6015' was out of bounds.
[1393576053] Warning: Return code of 255 for check of service 'Unix - Check Root File System' on host 'ISCCAG6015' was out of bounds.
[1393576103] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG6015' was out of bounds.
[1393576103] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG3013' was out of bounds.
[1393576103] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG3013' was out of bounds.
[1393576153] Warning: Return code of 255 for check of service 'Unix - Check Load' on host 'ISCCAG6015' was out of bounds.
[1393576153] Warning: Return code of 255 for check of service 'Unix - Check Memory' on host 'ISCCAG3013' was out of bounds.
[1393576243] Warning: Return code of 255 for check of service 'Unix - Check Swap Usage' on host 'ISCCAG6015' was out of bounds.
[1393576323] Warning: Return code of 255 for check of service 'Unix - Check NTP Time' on host 'ISCCAG3013' was out of bounds.
[1393576353] Warning: Return code of 255 for check of service 'Unix - Check Root File System' on host 'ISCCAG3013' was out of bounds.

[lmiltchev]

Can you also post the "check_xi_by_ssh" command definition?

[redeming]

define command {
command_name check_xi_by_ssh
command_line $USER1$/check_by_ssh -H $HOSTADDRESS$ -l $ARG1$ -C $ARG2$ $ARGS3$
}

[lmiltchev]

I tested the same config and I didn't get any issues in the GUI... Are ALL of your plugins on a network drive? Is it possible that you lose connectivity once in a while, hence the "intermittent" issues. Have you tried to place the plugins on the local drive?

[redeming]

What EXACTLY does error code 255 mean?

It's possible but I should expect to see failures at the same time when I run it from the command line.

[slansing]

It's roughly an out of bounds error, remember, you can feel free to open the plugins yourself and look for the error codes that are echoed out. The fact that it happens intermittently, and is visible on the web interface points to a possible networking issue, or something on the remote host. Just because it works every time for you from the command line doesn't mean that it is working 100% of the time from there either, since you are actively checking this system from the XI UI it is likely that you are running into this problem and then being told about it there, since it is unlikely that you are manually running the commands 24/7 until the issue occurs.