External command for notification won't work

An open discussion forum for obtaining help with Nagios Core. Nagios Core users of all experience levels are welcome here. Subforum have been created for the discussion of Nagios Core and Nagios Plugin development.

NOTE: The SourceForge.net mailing lists have been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

External command for notification won't work

Postby pilotmc » Tue Jun 13, 2017 5:52 pm

Hello.

I have an external command script for Slack notification which had been working fine but stopped a couple months ago. I believe it stopped right after doing a Debian upgrade.
I'm also unable to execute commands from the GUI due to: "Error: Could not open command file '/usr/local/nagios/var/rw/nagios.cmd' for update!".

Steps:

1) Install Nagios Core 4.2.3 and plugins on Debian 8.8 following this guide: https://support.nagios.com/kb/article/n ... tml#Debian
2) Set up a host, a slack user, a notification command and configure them according to: http://matthewcmcmillan.blogspot.com/20 ... -with.html (except without the slack shell script naming error on that post).
3) Watched the logs and saw errors and got emails (email notification was enabled as well), but no slack messages.
4) Running the slack script manually DOES send a notification to Slack.

So, something is broken somewhere and I can't figure out what.

Let me know what info is needed, as I don't want to pollute the board with endless gobs of data from config files that may be inconsequential.
Thanks
pilotmc
 
Posts: 9
Joined: Tue May 23, 2017 3:33 pm

Re: External command for notification won't work

Postby pilotmc » Tue Jun 13, 2017 9:46 pm

From nagios.log:

[1497407589] Warning: Notifying contact 'nagiosadmin' of service 'Load' on host 'newlive4.playnet.com' by command '/usr/local/bin/slack_nagios.sh > /tmp/slack.log 2>&1' timed out after 0.00 seconds
pilotmc
 
Posts: 9
Joined: Tue May 23, 2017 3:33 pm

Re: External command for notification won't work

Postby pilotmc » Tue Jun 13, 2017 9:53 pm

contacts.cfg:

Code: Select all
define contact{
    contact_name                nagiosadmin
    use                         generic-contact
    alias                       Nagios Admin
    service_notification_commands   notify-service-by-slack
    host_notification_commands     notify-host-by-slack
    email                       nagios@localhost
        }


/usr/local/bin/slack_nagios.sh:

Code: Select all
#!/bin/bash

WEBHOST_NAGIOS="monitor.mydomain.com"
SLACK_CHANNEL="#alerts"
SLACK_BOTNAME="Nagios"
WEBHOOK_URL="https://hooks.slack.com/services/********/**********/********************"

#Set the message icon based on Nagios service state
if [ "$NAGIOS_SERVICESTATE" = "OK" ]
then
    ICON_EMOJI=":thumbsup:"
elif [ "$NAGIOS_SERVICESTATE" = "WARNING" ]
then
    ICON_EMOJI=":warning:"
elif [ "$NAGIOS_SERVICESTATE" = "CRITICAL" ]
then
    ICON_EMOJI=":error:"
elif [ "$NAGIOS_SERVICESTATE" = "UNKNOWN" ]
then
    ICON_EMOJI=":troll:"
else
    ICON_EMOJI=":octocat:"
fi


#request for posting to a channel
curl -X POST --data "payload={\"channel\": \"${SLACK_CHANNEL}\", \"username\": \"${SLACK_BOTNAME}\", \"icon_emoji\": \":vertical_traffic_light:\", \"text\": \"${ICON_EMOJI} HOST: ${NAGIOS_HOSTNAME}   SERVICE: ${NAGIOS_SERVICEDISPLAYNAME} STATE: ${NAGIOS_SERVICESTATE} MESSAGE: ${NAGIOS_SERVICEOUTPUT} <http://${WEBHOST_NAGIOS}/cgi-bin/nagios3/extinfo.cgi?host=${NAGIOS_HOSTNAME}|See Nagios>\"}" ${WEBHOOK_URL}
pilotmc
 
Posts: 9
Joined: Tue May 23, 2017 3:33 pm

Re: External command for notification won't work

Postby tgriep » Wed Jun 14, 2017 10:26 am

When a command is run by the Nagios daemon, it is run as the nagios user account and if the script's permissions are not set correctly, it may not run.
Login to the Nagios server as root, run the following commands to change the permissions of the script.
Code: Select all
chown nagios.nagios /usr/local/bin/slack_nagios.sh
chmod a+x /usr/local/bin/slack_nagios.sh


After that, test it out and see if the notification is sent.
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 5330
Joined: Thu Oct 30, 2014 9:02 am

Re: External command for notification won't work

Postby pilotmc » Wed Jun 14, 2017 2:12 pm

Thanks, tgriep. This was how the perms were. However, I re-issued those commands just to say I did.
Still not working:

Code: Select all
[1497420041] wproc:   host=host4.domain.com; service=Disk Space; contact=nagiosadmin
[1497420041] wproc:   early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1497420041] Warning: Notifying contact 'nagiosadmin' of service 'Disk Space' on host 'host4.domain.com' by command '/usr/local/bin/slack_nagios.sh > /tmp/slack.log 2>&1' timed out after 0.00 seconds


One weird thing is that Nagios says it times out immediately, yet the slack.log has 30 seconds:

Code: Select all
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:29 --:--:--     0


The script relies on nagios env variables being set. I wonder if they're maybe not set when the script is called.
pilotmc
 
Posts: 9
Joined: Tue May 23, 2017 3:33 pm

Re: External command for notification won't work

Postby tgriep » Thu Jun 15, 2017 11:20 am

If the Environment variables are not enabled, that is probably causing the issue.
Edit the nagios.cfg file and set the following option to 1 and restart the nagios daemon.
Code: Select all
enable_environment_macros=1


If that doesn't work, in the link you provided on the first post, has examples on how to setup the command in nagios to pass the macros directly without enabling the environment variables.
Be sure to check out our Knowledgebase for helpful articles and solutions!
User avatar
tgriep
Madmin
 
Posts: 5330
Joined: Thu Oct 30, 2014 9:02 am


Return to Nagios Core

Who is online

Users browsing this forum: Bing [Bot] and 6 guests