Email Notifications not being received

[StuartHurst]

Hi all
Hoping that my Obi Wan is here as I'm pulling my hair out here.

I have Nagios 3.4.1 running Ubuntu Server 13.04. I have postfix service installed. (I have changed the email address below to a generic one but the real one works)

I originally set the commands.cfg email to be as below as I was testing everything, but now I need this to go to different people depending on the fault. I have also tried using contact_groups too.

command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/bin/mailx -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" notification@domain.com

this sent out email perfectly fine, but now we're nearly ready to go live, I've created the contacts.cfg with the appropriate information

define contact{
contact_name ita
alias analysts
host_notifications_enabled 1
service_notifications_enabled 1
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,r,u
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
email notification@domain.com
}

In the service check, I use

contacts ita

in the definition

In the nagios log, I only ever see

[1375353854] HOST ALERT: BLC_Switch;DOWN;HARD;1;Unreachable
[1375353854] HOST NOTIFICATION: root;BLC_Switch;DOWN;notify-host-by-email;Unreachable

I've been looking and checked that all the macro names have a $ at each end. I'm absolutely stuck as looking on the web and here I see that some remove the end $, some put it in " or '.

Please - help me Obi Wan, you're my only hope.

[abrist]

You should replace your email in the command with:

Code: Select all

$CONTACTEMAIL$

So that the command reads:

Code: Select all

command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/bin/mailx -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$

This way, each contact's email will replace the $CONTACTEMAIL$ macro.

[StuartHurst]

Yes - I should have posted up my current commands.cfg for the notification - it is as below..

# 'notify-host-by-email' command definition
define command{
command_name notify-host-by-email
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
}

# 'notify-service-by-email' command definition
define command{
command_name notify-service-by-email
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /usr/bin/mail -s "** $NOTIFICATION$ TYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$

}

I am not sure if the problem is actually here or somewhere else - looking at the nagios.log file, I see the following

[1375372857] HOST ALERT: Cr1Sw9;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
[1375372857] SERVICE ALERT: Cr1Sw9;PING;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1375372857] PASSIVE HOST CHECK: BLC_Switch;1;100
[1375372857] HOST ALERT: BLC_Switch;DOWN;HARD;1;100
[1375372857] HOST NOTIFICATION: technicians;BLC_Switch;DOWN;notify-host-by-email;100
[1375372857] SERVICE ALERT: Cr1Sw8;PING;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1375372867] HOST ALERT: Cr1Sw9;UP;SOFT;2;PING OK - Packet loss = 0%, RTA = 0.58 ms
[1375372867] HOST ALERT: Cr1Sw8;UP;SOFT;2;PING OK - Packet loss = 0%, RTA = 0.88 ms
[1375372867] SERVICE ALERT: Cr1Sw9;PING;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 0.78 ms

I believe that the bolded area is where the notification is setup and should be sent to Nagios to send out via postfix but we get nothing.
As the system would receive email from the hardcoded email address, there isn't an issue with exchange (and I've set it up with a separate IP and set that to open relay from Nagios IP only anyway.

[StuartHurst]

This is mad - I've just re-submitted a passive check and blow me - it's not only gone and emailed me... I wonder if the email queue is full????

[abrist]

You can check the queue with:

Code: Select all

mailq

To find out more information about the mailing of notifications:

Code: Select all

tail -25 /var/log/maillog

[StuartHurst]

cheers - I will check tomorrow

[lmiltchev]

Sure - let us know when you have the output.

[StuartHurst]

Interesting as I'm using Exchange 2013 on Server 2012 that isn't live yet.... It has 8Gb RAM and 4 CPU's. Just going to take a look at it now.

-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
303477C1 559 Fri Aug 2 08:42:33 "nagios@nagios"@hambleton.gov.uk
(host 10.1.2.239[10.1.2.239] said: 452 4.3.1 Insufficient system resources (in reply to end of DATA command))
user@emailaddress.com

238715BC 554 Fri Aug 2 08:42:33 "nagios@nagios"@hambleton.gov.uk
(host 10.1.2.239[10.1.2.239] said: 452 4.3.1 Insufficient system resources (in reply to end of DATA command))
user@emailaddress.com

2C1765DF 559 Fri Aug 2 08:42:33 "nagios@nagios"@hambleton.gov.uk
(host 10.1.2.239[10.1.2.239] said: 452 4.3.1 Insufficient system resources (in reply to end of DATA command))
user@emailaddress.com

26AE15C2 554 Fri Aug 2 08:42:33 "nagios@nagios"@hambleton.gov.uk
(host 10.1.2.239[10.1.2.239] said: 452 4.3.1 Insufficient system resources (in reply to end of DATA command))
user@emailaddress.com

[StuartHurst]

Restarted the server, allocated more memory, and all is well. how annoying that this didn't come up when the email address was hardcoded in.

[abrist]

Restarted the server, allocated more memory, and all is well. how annoying that this didn't come up when the email address was hardcoded in.

That is odd. At least the error message was verbose enough that once you got the error it was easily remedied. Enjoy the weekend, cheers!