I need the alert to be critical, not just a warning

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
Locked
aristosv
Posts: 12
Joined: Wed Apr 11, 2018 12:48 am

I need the alert to be critical, not just a warning

Post by aristosv »

Below is the basic configuration of my nagios core server. It creates an ssh connection to a remote computer and checks if the process "mpg123" is running. This is for a digital signage system.

/usr/local/nagios/etc/objects/commands.cfg

Code: Select all

define command{
command_name check_remote_music
command_line $USER1$/check_by_ssh -H $HOSTADDRESS$ -C 'pgrep mpg123'
}
/usr/local/nagios/etc/servers/client001.cfg

Code: Select all

define host{
        use linux-server
        host_name client001
        alias client001
        address <remote_ip_here>
        }

define service{
        use generic-service
        host_name client001
        service_description music
        check_command check_remote_music
        }
My problem is that when the remote computer is not accessible, I get a warning, with the message

Code: Select all

Remote command execution failed: ssh: connect to host <remote_ip_here> port 22: Connection refused 
I need this to be critical, and red. Not just a warning
How can I configure nagios to achieve this?

Thanks
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: I need the alert to be critical, not just a warning

Post by scottwilkerson »

Use the negate plugin

Here is an article explaining it
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

The article id for XI but you can use the negate plugin to change the status from one thing to another
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
aristosv
Posts: 12
Joined: Wed Apr 11, 2018 12:48 am

Re: I need the alert to be critical, not just a warning

Post by aristosv »

I'm sure I'm doing something wrong, I just need help finding out what it is.
Using negate doesn't change the command output, nor the exit code.

Code: Select all

$ /usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C 'pgrep mpg123'
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
$ echo $?
3
$

Code: Select all

$ /usr/local/nagios/libexec/negate /usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C 'pgrep mpg123'
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
$ echo $?
3
$

Code: Select all

$ /usr/local/nagios/libexec/negate -s /usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C 'pgrep mpg123'
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
$ echo $?
3
$
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: I need the alert to be critical, not just a warning

Post by scottwilkerson »

First you need to run these as the nagios user

Code: Select all

sudo su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C 'pgrep mpg123'"
However I am going to offer up a better solution to the problem
instead of running 'pgrep mpg123' I suggest running the following

Code: Select all

sudo su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C '(output=$(pgrep nagios);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
This will change your exit code...

The only change in your config should be to change

Code: Select all

pgrep mpg123
to

Code: Select all

(output=$(pgrep nagios);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
aristosv
Posts: 12
Joined: Wed Apr 11, 2018 12:48 am

Re: I need the alert to be critical, not just a warning

Post by aristosv »

I am assuming you meant output=$(pgrep mpg123), not output=$(pgrep nagios)

This is my result with a server that's supposed to be offline

Code: Select all

su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
This is my result with a server that's online

Code: Select all

su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: !=: unary operator expected
proof that 192.168.1.51 is up

Code: Select all

root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C 'pgrep mpg123'"
318
For some reason the same code doesn't work on live servers.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: I need the alert to be critical, not just a warning

Post by scottwilkerson »

aristosv wrote:This is my result with a server that's supposed to be offline

Code: Select all
su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
Expected, you cannot connect to an offline server
aristosv wrote:This is my result with a server that's online

Code: Select all
su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: !=: unary operator expected
You may need to tweak my examples per your shell, maybe change this

Code: Select all

if [ $return != 0 ]
to this

Code: Select all

if [ $return -gt 0 ]
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
aristosv
Posts: 12
Joined: Wed Apr 11, 2018 12:48 am

Re: I need the alert to be critical, not just a warning

Post by aristosv »

I tried multiple combinations to fix this, it's just not working. It's a debian machine, using bash.

Code: Select all

root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.60 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: ssh: connect to host 192.168.1.60 port 22: Connection refused
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return != 0 ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: !=: unary operator expected
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ $return -gt 0 ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: -gt: unary operator expected
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ "$return" -gt "0" ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: -gt: unary operator expected
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ "$return" != "0" ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: !=: unary operator expected
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if ("$return" -gt "0");then exit 2;fi)'"
Remote command execution failed: bash: -gt: command not found
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if ("$return" != "0");then exit 2;fi)'"
Remote command execution failed: bash: !=: command not found
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if ["$return" -gt "0"];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: missing `]'
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [ "$return" -gt "0" ];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: -gt: unary operator expected
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if ($return -gt 0);then exit 2;fi)'"
Remote command execution failed: bash: -gt: command not found
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [$return -gt 0];then exit 2;fi)'"
Remote command execution failed: bash: line 0: [: missing `]'
root@nagios:~#
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: I need the alert to be critical, not just a warning

Post by scottwilkerson »

You may need the extra braces [].

Try this, can you execute this directly on the debian machine? 192.168.1.51

Code: Select all

(output=$(pgrep mpg123);return=$?;echo $output;if [[ $return -gt 0 ]];then exit 2;fi)
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
aristosv
Posts: 12
Joined: Wed Apr 11, 2018 12:48 am

Re: I need the alert to be critical, not just a warning

Post by aristosv »

tried a bunch of things again, nothing worked. Maybe there's a better way?

Code: Select all

root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C 'pwd'"
/home/nagios
root@nagios:~# su nagios -c "/usr/local/nagios/libexec/check_by_ssh -H 192.168.1.51 -C '(output=$(pgrep mpg123);return=$?;echo $output;if [[ $return -gt 0 ]];then exit 2;fi)'"
Remote command execution failed: bash: -c: line 0: conditional binary operator expected
root@nagios:~#
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: I need the alert to be critical, not just a warning

Post by scottwilkerson »

Well the best way is to write a plugin and then just call the plugin from check_by ssh so it doesn't have to contend with being all on one line. Then your plugin can contain the login you want, and be writing in any language you understand.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked