Nagios Support Forum • Event handler not working

Page 1 of 4

Event handler not working

Posted: Wed Nov 13, 2013 2:08 pm

by BanditBBS

I created a command in XI CCM that does this:

Code: Select all

$USER1$/check_nrpe -H $HOSTADDRESS$ -t 30 -c clean_drive -a '$SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$'

if I run that command from the cli as nagios, it works great. But when I have that in the eventhandler it doesnt seem to be running. So my question is, are the single quotes OK or would that make XI actually pass $SERVICESTATE$ instead of OK or WARNING or.....etc

Re: Event handler not working

Posted: Wed Nov 13, 2013 6:46 pm

by abrist

Alright. There is a workaround I have discovered for this, but it is convoluted and complex.

NRPE will not allow you to pass an argument with nasty metachars ( ' " ` | etc). You can verify this by running a tail on the remote system's /var/log/messages:

Code: Select all

tail -f /var/log/messages

Run the event handler by hand, or from XI. You should see the following error on the remote host:

Code: Select all

Aug 23 18:14:25 gent nrpe[29338]: Host address is in allowed_hosts
Aug 23 18:14:25 gent nrpe[29338]: Handling the connection...
Aug 23 18:14:25 gent nrpe[29338]: Error: Request contained illegal metachars!
Aug 23 18:14:25 gent nrpe[29338]: Client request was invalid, bailing out...

This is a restriction of nrpe. I work around it by base encoding the parameters with a local script, then running nrpe from that script passing the base encoded string:

Code: Select all

$USER1$/encode_script.sh '$HOSTADDRESS'  '/path/to/clean_drive.sh "$SERVICESTATE$" "$SERVICESTATETYPE$" "$SERVICEATTEMPT$" '

encode_script.sh:

Code: Select all

#!/bin/bash 
HOST=$1
UNCODED=$2
ENCODED=$(echo '$UNCODED' | base64)
/usr/local/nagios/libexec/check_nrpe -h $HOST -c decode_script "$ENCODED"

One the remote system:
nrpe.cfg command:

Code: Select all

command[decode_script]=/usr/local/nagios/libexec/decode_script.sh $ARG1$

The remote system's script run by nrpe acts as a wrapper - decoding the arguments back into a string :
decode_script.sh:

Code: Select all

#!/bin/bash
ENCODED=$1
DECODED=`echo $ENCODED | base64 -d`
eval $DECODED

Does this make sense?

Re: Event handler not working

Posted: Wed Nov 13, 2013 7:05 pm

by BanditBBS

Everything you said made sense, except for one glaring issue....I said if I run it manually that it works great! LOL.

Code: Select all

./check_nrpe -H 192.168.1.50 -t 30 -c clean_drive -a 'WARNING PROBLEM 2'

If I would run the above from the libexec folder on my nagios box, it works great. Are you saying the $SERVICESTATE$ and other variables have the bad characters in them?

Re: Event handler not working

Posted: Wed Nov 13, 2013 7:15 pm

by BanditBBS

Do I even need the ' to pass the arguments?

......why was I thinking I needed to wrap the -a stuff in '

Re: Event handler not working

Posted: Wed Nov 13, 2013 7:36 pm

by abrist

The problem is the nested quotes that you are using to account for potential spaces in the macros. You cannot nest quotes inside the argument switch ( -a '<arguments>' ). At least this is what I have found. My suggestion would be to check the remote system's logs for 'illegal metachars'. That will shed some light on the potential causes of the problem.

Re: Event handler not working

Posted: Wed Nov 13, 2013 7:41 pm

by abrist

BanditBBS wrote:......why was I thinking I needed to wrap the -a stuff in '

Because they are needed so as to differentiate between the arguments passed to command and other arguments used for the check_nrpe plugin.
This:

Code: Select all

check_nrpe -H <host> -c <command> -a '-w 5 -c 10'

Is not the same as:

Code: Select all

check_nrpe -H <host> -c <command> -w 5 -c 10

Nor:

Code: Select all

check_nrpe -H <host> -c <command> -a -w 5 -c 10

EDIT: Looking over the macros you are using, you may be able to just eliminate the nested quotes as those macros do not, under normal circumstances, include any spaces . . .

SECOND EDIT: Well, I am a bit dense today. You don't have any nested quotes. Well, crap - that was a fun exercise. . . I will take a look at this again tomorrow when I am less of a dunce.

Re: Event handler not working

Posted: Wed Nov 13, 2013 7:59 pm

by abrist

For now, add a line to your event handler script to echo the arguments passed to it to a file in /tmp, run the handler, and then check the temp file.

Code: Select all

echo "$1 $2 $3" > /tmp/event_test

Re: Event handler not working

Posted: Wed Nov 13, 2013 8:09 pm

by BanditBBS

abrist wrote:For now, add a line to your event handler script to echo the arguments passed to it to a file in /tmp, run the handler, and then check the temp file.
Code: Select all
echo "$1 $2 $3" > /tmp/event_test

That's a good idea. I'll add that to my script. The only way to make the eventhandler run though, would be to submit a passive check that changes the state, right?

Also, can you explain the difference between the 3 examples you gave...for someone who is also being rather slow today as well.

Re: Event handler not working

Posted: Wed Nov 13, 2013 8:18 pm

by BanditBBS

Well, it is as I expected in the file:

Code: Select all

WARNING SOFT 2

Here is the code that ran:

Code: Select all

        echo "$1 $2 $3" > /tmp/event_test
        echo -n "Cleaning the drive...\n"
        /usr/local/nagios/libexec/clean_wlan_arc.sh

Contents of the clean_wlan_arc.sh:

Code: Select all

echo "Searching and removing older files"
/usr/bin/find /var/bu/wlanadmin/*.gpg -mtime +5 -exec sudo /bin/rm {} \;
/usr/bin/find /var/bu/wlanadmin/*.cfg -mtime +5 -exec sudo /bin/rm {} \;
exit

As stated, if I run the command from command line, it works fine...I'm lost

Re: Event handler not working

Posted: Wed Nov 13, 2013 8:34 pm

by BanditBBS

You know, maybe I'm being an idiot and the script isn't really working properly. When that eventhandler gets called and it runs. When NRPE runs the script on the host, it runs as nagios, correct? So if I "sudo su nagios" and run that script and it works as it should, shouldn't it work when running through nrpe? OMG, did I explain that well enough, or did that sound like a benhank post? (LOL)

From the problem server:

Code: Select all

[clarkj@svwdcnetmg02 log]$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                     272853152 243700152  15069316  95% /
/dev/sda1               101086     26965     68902  29% /boot
tmpfs                  4074116         0   4074116   0% /dev/shm
[clarkj@svwdcnetmg02 log]$ sudo su nagios
bash-3.2$ cd /usr/local/nagios/libexec
bash-3.2$ ./clean_drive WARNING SOFT 2
Cleaning the drive...\nSearching and removing older files
bash-3.2$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                     272853152 230906332  27863136  90% /
/dev/sda1               101086     26965     68902  29% /boot
tmpfs                  4074116         0   4074116   0% /dev/shm
bash-3.2$