Nagios failed to send notification

[sib]

Hi

Nagios has spotted an issue and created a notification. But it was not able to send the email.

System: RHEL 6.6 64bit
Nagios XI 2014R2.6

Code: Select all

[1428453452] Unable to send check for host 'collrepp' to worker (ret=-2)
[1428453591] Unable to run check for service 'collrepp - assure.log' on host 'collrepp'
[1428453883] Unable to run check for service 'collrepp - assure.log' on host 'collrepp'
[1428453917] Unable to send check for host 'collrepp' to worker (ret=-2)
[1428454000] SERVICE ALERT: collrepp;collrepp - assure.log;CRITICAL;HARD;1;CRITICAL: Found 1 lines (limit=1/1): 2015-04-08 02:46:39,021 INFO  [pool-5-thread-3] [] (ProcessNotifier.java:55) - ERR_NOTN: type=invalidSecuritiesExist:supplier=BISinternal:reportDate=07-04-2015:env=Prod:sysDate=08-04-2015 02:46:39
[1428454000] SERVICE NOTIFICATION: Collrepp Contacts;collrepp;collrepp - assure.log;CRITICAL;notify-collrep;CRITICAL: Found 1 lines (limit=1/1): 2015-04-08 02:46:39,021 INFO  [pool-5-thread-3] [] (ProcessNotifier.java:55) - ERR_NOTN: type=invalidSecuritiesExist:supplier=BISinternal:reportDate=07-04-2015:env=Prod:sysDate=08-04-2015 02:46:39
[1428454058] Unable to run check for service 'collrepp - assure.log' on host 'collrepp'
[1428454116] SERVICE ALERT: collrepp;collrepp - assure.log;OK;HARD;1;OK: Found 0 lines (limit=1/1): No matches found.
[1428454290] Unable to run check for service 'collrepp - assure.log' on host 'collrepp'
[1428454349] Unable to send check for host 'collrepp' to worker (ret=-2)
[1428454524] Unable to send check for host 'collrepp' to worker (ret=-2)

The notify-collrep command. It formats the output given by SERVICEOUTPUT and sends it to CONTACTEMAIL

Code: Select all

define command {
       command_name                         notify-collrep
       command_line                         $USER1$/notifyCollrep.sh "$SERVICEOUTPUT$" $CONTACTEMAIL$
}

The Contact definition

Code: Select all

define contact {
    contact_name                        Collrepp Contacts
    host_notifications_enabled          1
    service_notifications_enabled       1
    host_notification_period            24x7
    service_notification_period         24x7
    host_notification_options           d,u,
    service_notification_options        c,
    host_notification_commands          notify-host-by-email
    service_notification_commands       notify-collrep
    email                               [email protected] [email protected] 
    }

The notification script is quite easy. It just formats the output

Code: Select all

#!/bin/bash

# Notifies collrep people
# CRITICAL: Found 1 lines (limit=1/1): 2015-03-19 15:23:13,982 INFO [pool-5-thread-2] [] (ProcessNotifier.java:55) - ERR_NOTN: type=invalidHoldingsExist:supplier=JPMorgan:reportDate=18-02-2015:env=Dev:sysDate=19-03-2015 15:23:13

MESSAGE=`echo $1 | sed 's/^.*ERR/ERR/'`
shift

SUBJECT=`echo $MESSAGE | sed 's/.*type=\(.*\):supplier=\(.*\):report.*env=\(.*\):sysDate.*$/ASSURE-Error = \1 \/ \2 (\3)/'`

echo $MESSAGE | /bin/mail -s "$SUBJECT" $@

[fortisit]

"ERR_NOTN: type=invalidSecuritiesExist:supplier=BISinternal:reportDate=07-04-2015:env=Prod:sysDate=08-04-2015 02:46:39" did you try resolve this?

[sib]

Not sure what you mean by this. But this is the actual error message generated by the check. Nagios was able to get the critical message from the check but failed to notify the contacts.

[fortisit]

Try send notification manualy, from custom notification. When fail then try send sample message from bash. Another notification are sended successfuly?

[sib]

It works fine if I send it manually. I suspect that maybe the variables were empty?

[abrist]

Looking at the script, I would suggest wrapping a number of your variables in quotes, as a string will spaces will cause all sorts of craziness:

Code: Select all

#!/bin/bash

# Notifies collrep people
# CRITICAL: Found 1 lines (limit=1/1): 2015-03-19 15:23:13,982 INFO [pool-5-thread-2] [] (ProcessNotifier.java:55) - ERR_NOTN: type=invalidHoldingsExist:supplier=JPMorgan:reportDate=18-02-2015:env=Dev:sysDate=19-03-2015 15:23:13

MESSAGE=`echo "$1" | sed 's/^.*ERR/ERR/'`
shift

SUBJECT=`echo "$MESSAGE" | sed 's/.*type=\(.*\):supplier=\(.*\):report.*env=\(.*\):sysDate.*$/ASSURE-Error = \1 \/ \2 (\3)/'`

echo "$MESSAGE" | /bin/mail -s "$SUBJECT" $@

I don't know if this will solve the issue with sending it though.