Nagios Support Forum

Posted: **Mon Mar 09, 2020 5:28 pm**

hello,

I am trying to associate the nagios alerts with Slack. I referred this doc:
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

but i got 404 respond. can you help take a look?

Code: Select all

-bash-4.1# ./slack_nagios.pl -field slack_channel=#dvmt30load -field HOSTALIAS="Test Host" -field HOSTSTATE=“Warning” -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE=“LOADISSUE”
Request:
POST https://xxxxxx.slack.com/services/hooks/nagios?token=xxxxxxxx
Content-Length: 165
Content-Type: application/x-www-form-urlencoded

HOSTOUTPUT=Host+is+UP&NOTIFICATIONTYPE=%E2%80%9CLOADISSUE%E2%80%9D&slack_version=1.1&slack_channel=dvmt30load&HOSTSTATE=%E2%80%9CWarning%E2%80%9D&HOSTALIAS=Test+Host

Response:
HTTP/1.1 404 Not Found
Connection: close
Date: Mon, 09 Mar 2020 22:22:31 GMT
Server: Cisco Umbrella
Content-Type: text/html
Client-Date: Mon, 09 Mar 2020 22:22:31 GMT
Client-Peer: 146.112.61.106:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /O=Cisco/CN=Cisco Umbrella Secondary SubCA pao-SG
Client-SSL-Cert-Subject: /C=US/ST=California/L=San Francisco/O=OpenDNS, Inc./CN=*.opendns.com
Client-SSL-Cipher: AES256-GCM-SHA384
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

Posted: **Tue Mar 10, 2020 9:53 am**

This may not be all of your problem, but I did notice that you have matching quotes rather than regular double quotes around at least one of your variables. That's going to confuse things.

I ran it through octal dump to make sure it was what I thought it was:

Code: Select all

$  head -1 1.txt | od -c
0000000 342 200 234   L   O   A   D   I   S   S   U   E 342 200 235  \n
0000020

 
$  tail -1 1.txt | od -c
0000000   "   L   O   A   D   I   S   S   U   E   "  \n
0000014

So, try using regular double quotes ("like this") and see if that helps. If not, let us know and we'll look for the next part of the issue.

Thanks!

--Jeffrey

Posted: **Tue Mar 10, 2020 7:22 pm**

thanks. somehow the quotes might not be the only reason. i had corrected the quotes but still got the 404. thank you

Code: Select all

./slack_nagios.pl -field slack_channel=#dvmt30load -field HOSTALIAS="Test Host" -field HOSTSTATE="Warning" -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE="LOADISSUE"

Request:
POST https://xxxxslack.com/services/hooks/nagios?token=xxxxxx
Content-Length: 132
Content-Type: application/x-www-form-urlencoded

HOSTOUTPUT=Host+is+UP&NOTIFICATIONTYPE=LOADISSUE&slack_version=1.1&slack_channel=%23dvmt30load&HOSTSTATE=Warning&HOSTALIAS=Test+Host

Response:
HTTP/1.1 404 Not Found
Connection: close
Date: Wed, 11 Mar 2020 00:21:21 GMT
Server: Cisco Umbrella
Content-Type: text/html
Client-Date: Wed, 11 Mar 2020 00:21:21 GMT
Client-Peer: 146.112.61.106:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /O=Cisco/CN=Cisco Umbrella Secondary SubCA pao-SG
Client-SSL-Cert-Subject: /C=US/ST=California/L=San Francisco/O=OpenDNS, Inc./CN=*.opendns.com
Client-SSL-Cipher: AES256-GCM-SHA384
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

Posted: **Wed Mar 11, 2020 2:10 pm**

I don't think that notification type is correct.

Can you try your command like so:

Code: Select all

./slack_nagios.pl -field slack_channel=#dvmt30load -field HOSTALIAS="Test Host" -field HOSTSTATE="UP" -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE="RECOVERY"

and see if that works?

Posted: **Thu Mar 12, 2020 5:33 pm**

thanks. I tried. but no luck, same error.

even the previous notificationtype was directly copied from the nagios reference link: https://assets.nagios.com/downloads/nag ... ios-XI.pdf

Posted: **Thu Mar 12, 2020 8:57 pm**

hello, eventually i decded to use webhook instead of the Nagios slack app.

i was able to install the commandline, service, but somehow the service status always in pending status.

i had referenced this tool: https://github.com/RunlevelConsulting/N ... ifications
and created service and command as below:

Code: Select all

define command {
    command_name    notfify-slack-techops-channel
    command_line    $USER1$/notification_slack.sh -a "$NOTIFICATIONTYPE$" -b $HOSTNAME$ -c "$HOSTSTATE$" -d "$HOSTOUTPUT$" -y "techops-alerts" -z "xxxxxx"
}

define service {
    host_name              xxxx.dvms.local
    service_description    notify slack techops channel
    display_name           notify slack techops channel
    check_command          notfify-slack-techops-channel!!!!!!!!
    max_check_attempts     5
    check_interval         2
    retry_interval         1
    check_period           workhours
    register               1
}

is there anything wrong what i had done? please check attached screenshot.

jdunitz wrote:I don't think that notification type is correct.

Can you try your command like so:
Code: Select all
./slack_nagios.pl -field slack_channel=#dvmt30load -field HOSTALIAS="Test Host" -field HOSTSTATE="UP" -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE="RECOVERY"
and see if that works?

Posted: **Fri Mar 13, 2020 3:33 pm**

It's hard to say why you're getting the result you're getting, but let's investigate some more.

In the notification_slack.sh script, there are a few configuration lines that you'll have to edit to match your local setup:

SLACK_HOSTNAME="slack_hostname.slack.com"
SLACK_BOTUSERNAME="nagios-alerts"
MONITORING_URL="https://nagios_server.domain/nagios"
SLACK_CHANNEL="#${CHANNEL}"
WEBHOOK_ADDRESS="https://hooks.slack.com/services/${WEBHOOK_PATH}"

You made sure to set these to the correct values, yes?

If so, can you test the ./notification_slack.sh script on the command line, and does it work?

Also--and this applies to getting both plugins to work--be sure the slack channel you're pointing to exists, and that you can reach the slack servers from your nagios machine.

Hope this helps!

Finally, I recommend that, at least until you get things working, you set your notification time to 24x7, rather than workhours. It shouldn't make a difference, but if there's something strange with how your times are set up, we want to make sure that's not getting in the way.

Posted: **Fri Mar 13, 2020 5:49 pm**

thank you for your help.
I was able to leverage the script for slack integration. but.... i have to define one webhook for each slack channel. which the IT limited. so i have to stick to the previous perl solution

here is the output after I updated the openssl and curl:

Code: Select all

-bash-4.1# ./slack_nagios.pl -field slack_channel=#adfadfasdfdsa -field HOSTALIAS="Test Host" -field HOSTSTATE="UP" -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE="RECOVERY"
Request:
POST https://isi-eng.slack.com/services/hooks/nagios?token=ssdfdsfdsxxxx
Content-Length: 126
Content-Type: application/x-www-form-urlencoded

HOSTOUTPUT=Host+is+UP&NOTIFICATIONTYPE=RECOVERY&slack_version=1.1&slack_channel=%23dvmt30load&HOSTSTATE=UP&HOSTALIAS=Test+Host

SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL3 alert read:fatal:handshake failure
SSL_connect:error in SSLv2/v3 read server hello A
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL3 alert read:fatal:handshake failure
SSL_connect:failed in SSLv3 read server hello A
SSL_connect:before/connect initialization
SSL_connect:SSLv2 write client hello A
SSL_connect:failed in SSLv2 read server hello A
Response:
500 SSL negotiation failed: 
Content-Type: text/plain
Client-Date: Fri, 13 Mar 2020 22:46:25 GMT
Client-Warning: Internal response

500 SSL negotiation failed: 

You have new mail in /var/spool/mail/root

I have tried disable to use SSLv2/v3 but TLS1.2, but no luck:

Code: Select all


use warnings;
use strict;

use Getopt::Long;
use HTTP::Request::Common qw(POST);
use HTTP::Status qw(is_client_error);
use LWP::UserAgent;

$ENV{HTTPS_DEBUG} = 1;

#
# Customizable vars. Set these to the information for your team
#

my $opt_domain = "isi-eng.slack.com"; # Your team's domain
my $opt_token = "gDzxcp10OfkH5AyX4U64ywxx"; # The token from your Nagios services page


#
# Get command-line opts
#

my %opt_fields;
GetOptions("field=s%" => \%opt_fields);


#
# DO THINGS
#

my %event;

# Get all Nagios variables
while ((my $k, my $v) = each %ENV) {
	next unless $k =~ /^(?:NAGIOS|ICINGA)_(.*)$/;
	$event{$1} = $v;
}

# Merge in passed-in variables
%event = (%event, %opt_fields);

$event{"slack_version"} = "1.1";


#
# Make the request
#

#my $ua = LWP::UserAgent->new;

my $ua = LWP::UserAgent->new(ssl_opts => {
    verify_hostname => 0,
    SSL_version => 'TLSv1_2',
});

$ua->timeout(15);

my $req = POST("https://${opt_domain}/services/hooks/nagios?token=${opt_token}", \%event);

my $s = $req->as_string;
print STDERR "Request:\n$s\n";

my $resp = $ua->request($req);
$s = $resp->as_string;
print STDERR "Response:\n$s\n";

I also tried:
SSL_version => 'TLSv1_2',
SSL_version => 'TLSv12:!SSLv2:!SSLv3:!TLSv1',

Posted: **Mon Mar 16, 2020 12:08 pm**

Are you using CentOS or RHEL version 6 or 7?

I ask, because Slack just started enforcing SNI, and that may be an issue for the older versions of perl that come with those.

Let us know what you've got.

Thanks!

Posted: **Tue Mar 17, 2020 2:21 pm**

you are right. it's CentOS 6. it's caused by the old version of perl.
after i switched to use python, i am able to have the slack integration. it tested successfully on nagios server side.

even though, i had defined the commands, the contacts configuration defined, it doesn't push notification to the channel though. what i missed here?

for some commands, i imported from CCM, but when creating a contact, the command cannot be found from the list, why?

one more question on the contact configuration:
we have services associated to different hosts, for example service S applied to host h1, h2... h3. the notification config on the service S end will apply all hosts? we need for S on h1 send to channel teamA, but S issue on h2 send to teamB send to channel B. how to implement it?

Code: Select all

define contact {
    contact_name slack_techops
    alias slack_techops
    service_notification_period 24x7
    host_notification_period 24x7
    service_notification_options w,u,c,r
    host_notification_options d,r
    host_notification_commands slack_python_host_techops_c
    service_notification_commands slack_python_service_techops_c
}


define command {
    command_name    slack_python_host_techops_c
    command_line     $USER1$/slack2.py  -c techops-alerts -field HOSTALIAS=$HOSTNAME$ -field HOSTSTATE="$HOSTSTATE$" -field HOSTOUTPUT="$HOSTOUTPUT$" -field NOTIFICATIONTYPE="$NOTIFICATIONTYPE$"
}


define command {
    command_name    slack_python_service_techops_c
    command_line    $USER1$/slack2.py  -c techops-alerts -field HOSTALIAS="$HOSTNAME$" -field SERVICEDESC="$SERVICEDESC$" -field SERVICESTATE="$SERVICESTATE$" -field SERVICEOUTPUT="$SERVICEOUTPUT$" -field NOTIFICATIONTYPE="$NOTIFICATIONTYPE$"
}

jdunitz wrote:Are you using CentOS or RHEL version 6 or 7?

I ask, because Slack just started enforcing SNI, and that may be an issue for the older versions of perl that come with those.

Let us know what you've got.

Thanks!

Nagios Support Forum

integration with Slack

integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack

Re: integration with Slack