nagios custom notifications do not work

[lukenio]

Hi,

I'm trying to make nagios send notifications via usb modem as text messages. This is not a part of the issue, because I have a working bash script that takes care of it and if I run it (via sudo as nagios user) it works flawlessy.
I have configured the user object (in contacts.cfg) like so:

Code: Select all

define contact{
.......
        service_notification_commands   notify-service-by-sms
        host_notification_commands      notify-host-by-sms
.......
}

These notification commands are configured in the main configuration file (/etc/nagios3/commands.cfg):

Code: Select all

# 'notify-host-by-sms' command definition
define command{
        command_name    notify-host-by-sms
        command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/lib/nagios/plugins/send_sms $CONTACTPAGER$
        }

Code: Select all

# 'notify-service-by-sms' command definition
define command{
        command_name    notify-service-by-sms
        command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /usr/lib/nagios/plugins/send_sms $CONTACTPAGER$
        }

and finally, the /usr/lib/nagios/plugins/send_sms is a bash script starting with:

Code: Select all

#!/bin/bash
date >/tmp/smssend.log

which never gets written unless I run the script manually.
It seems to me that nagios has some kind of a problem with executing the script, which is very very weird. I repeat, logging in as user 'nagios' i can exec the script just fine, and any contents passed via stdin is properly received on the mobile.
I am also monitoring the nagios log file (/var/lib/nagios3/nagios.log) and some events do appear frequently (I ocasionally just pull a plug here and there to generate some events if needed).
One more remark - before the transition to sms notifications I did have working email notifications.
It's driving me nuts already. So please could someone actually shed some light on what is really wrong here?

[abrist]

I repeat, logging in as user 'nagios' i can exec the script just fine,

I have a working bash script that takes care of it and if I run it (via sudo as nagios user) i

Do you have to use 'sudo' to run the script correctly, or can the nagios user just run the script without elevating permissions?

[lukenio]

it runs properly with nagios account. to be exact, i go from root with su to nagios account and run it from there.

to put it more concisely, it runs fine on the nagios account, without any permission elevation.
the only permission elevation is switching from root account to nagios account prior to the test to ensure that the environment is similar to the one of nagios daemon.


at one moment I thought maybe it's the PATH env variable missing from the nagios daemon environment, so I rewrote date to /bin/date, and so on with other binaries in that script. but nothing changed.

[sreinhardt]

So the date logging portion does not work, however the sms is correctly sent despite this, indicating that the rest of the script is executing. correct? Sorry to keep reitterating what is said, just want to be 100% sure. I get that you are only su-ing into the nagios user and not executing the script as root. The /bin/date is a good idea. Path should be correct for the nagios user, but it never hurts. What permissions are on that temp file? Any chance its not writeable or owned by someone other than nagios\nagcmd?

Code: Select all

ls -lva /tmp/smssend.log

Also, what about changing it to /bin/date >> /tmp/smssend.log to append instead of overwrite?

[lukenio]

Thanks for quick reply and advice. I did check the temp file and it is owned by nagios. I went and deleted it to allow nagios to recreate it and it did as expected. Anyway now I am also logging output from the id command as well. I went and modified a few things a bit so presently the situation is a bit different. Let me clarify.
Currently there are two cases. First case is running the send_sms script manually (as nagios). Then the date command works, the id command works, the temp file is written and all the loggin information appears and finally the sms gets out and is received on the phone.
Then is the nagios case when it should run the command. And most often it does not. At least nothing is ever logged or sent out via sms. I said most often because there was one or two occasions when I saw something in the log file but then nothing was received by the phone. I suspect it might have been right after I deleted the temp file.
This makes me think that maybe it is not the nagios account which the send command is executed with, but the NOBODY account. Then it could not write to the log file created with manual command nor write to the modem. Ofcourse I had added nagios to the group dialout. But if it's the NOBODY account then it's a dead end there.
I have to do more testing.

[lukenio]

I changed debug level for nagios logging. This helped to explain why no sms was ever sent:

[1378410241.024868] [032.1] [pid=15935] We shouldn't notify about this recovery.
[1378410241.024877] [032.0] [pid=15935] Notification viability test failed. No notification will be sent out.

Seems the script was allright anyway

[sreinhardt]

Seems the script was allright anyway

Awesome! I'll lock this up for now then. Feel free to open another if you find other problems.