Would greatly appreciate some help configuring notifications

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
logic_bomb421
Posts: 43
Joined: Tue Jul 15, 2014 6:58 pm

Would greatly appreciate some help configuring notifications

Post by logic_bomb421 »

Hi all, new here. This seems like a great community for support.

I'm stumped at this point. I've got Nagios configured the way I like it, but I cannot seem to get it to actually send any notifications. I'm 99% sure the problem lies in my sendmail configuration, since the Nagios notification config is so straightforward. I am trying to get this to work with sendmail on Ubuntu, so I installed sendmail, and answered yes to all the config questions. My problem at this point is nothing will actually send. If I attempt to use the sendmail command from the terminal, the log located at /var/log/mail.log has the entries for the send attempt, but it always ends in "connection refused by alt4.gmail-smtp-in.l.google.com", or whatever domain I'm trying to send to.

After hours of research, I came across pages saying that this could be caused by an incorrect FQDN. I looked up how to set my FQDN and followed the guides. What I'm left with is this:

in the file /etc/hosts, I have:

Code: Select all

127.0.0.1                     localhost localhost.localdomain localhost [server's name]
[server's IP address]         notify.nagios.com notify

# The following lines are desirable for IPv6 capable hosts
::1                               ip6-localhost ip6-loopback
fe00::0                           ip6-localnet
ff00::0                           ip6-mcastprefix
ff02::1                           ip6-allnodes
ff02::2                           ip6-allrouters
and in /etc/hostname I have:

Code: Select all

notify.nagios.com
I'm so new to Linux that all I can really do is follow internet tutorials and try to piece everything together in my head (which reeeeally hinders my ability to troubleshoot issues like this), so I appreciate any help. If I forgot to include something you need to help me with this, please say so and I'll add it ASAP.

Thanks!
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: Would greatly appreciate some help configuring notificat

Post by tmcdonald »

Does your company/organization have a SMTP server? The biggest problem with sendmail is that a lot of spammers like to use it to blast out emails without needing a domain name, so it gets rejected frequently. If you have a SMTP you can relay through that will help matters a bit.
Former Nagios employee
logic_bomb421
Posts: 43
Joined: Tue Jul 15, 2014 6:58 pm

Re: Would greatly appreciate some help configuring notificat

Post by logic_bomb421 »

We do have an SMTP server. I was discussing this elsewhere on the web and was told I should look into Postfix instead of sendmail for similar reasons.

EDIT: So I just installed Postfix. I used the Internat host option (default), and then edited the main.cf file, setting the "relaythrough" option to my smtp server FQDN. I still seem to be unable to get notifications for some reason though?
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Would greatly appreciate some help configuring notificat

Post by abrist »

Can you post a tail of your maillog?

Code: Select all

tail -50 /var/log/maillog
We are looking for bounces, etc.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
logic_bomb421
Posts: 43
Joined: Tue Jul 15, 2014 6:58 pm

Re: Would greatly appreciate some help configuring notificat

Post by logic_bomb421 »

Alright, this is the last 10 entries from /var/mail/mail.log:

Code: Select all

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9iPnP041266: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+09:49:11, xdelay=00:00:00, mailer=esmtp, 
pri=31260526, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9caes040781: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+09:55:00, xdelay=00:00:00, mailer=esmtp, 
pri=31350593, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9Rk0j039908: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:05:50, xdelay=00:00:00, mailer=esmtp, 
pri=31440543, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9SbV9039989: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:04:59, xdelay=00:00:00, mailer=esmtp, 
pri=31440593, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9Saie039982: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:05:00, xdelay=00:00:00, mailer=esmtp, 
pri=31440594, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G9QUs8039793: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:07:06, xdelay=00:00:00, mailer=esmtp, 
pri=31440597, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G8iPqM036427: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:49:11, xdelay=00:00:00, mailer=esmtp, 
pri=31800526, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G8ca2P035940: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+10:55:00, xdelay=00:00:00, mailer=esmtp, 
pri=31890593, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G8RkHa035069: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+11:05:50, xdelay=00:00:00, mailer=esmtp, 
pri=31980543, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.

Jul 18 12:33:36 nm02 sm-mta[28147]: s6G8SbmW035166: to=<smmmhill@gmail.com>, ctladdr=<nagios@notify.nagios.com> (1001/1001), delay=2+11:04:59, xdelay=00:00:00, mailer=esmtp, 
pri=31980593, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.0.0, stat=Deferred: Connection refused by alt4.gmail-smtp-in.l.google.com.
Fyi, regarding the nagios@notify.nagios.com ctladdr, the notify.nagios.com was what I entered when I configured Postfix several days ago and was unsure what to put there. That is not what I want to show up there anymore, but I have no idea how to change it.

Thanks for you help.
chris.fixter
Posts: 22
Joined: Wed Jun 18, 2014 4:15 am

Re: Would greatly appreciate some help configuring notificat

Post by chris.fixter »

Looks like a connection issue. Please go to your linux shell and run the follow lines(line by line, not all at once) to see which part gets stuck.

Code: Select all

telnet alt4.gmail-smtp-in.l.google.com 25
helo notify.nagios.com
mail from: <nagios@notify.nagios.com>
rcpt to: <smmmhill@gmail.com>
quit
logic_bomb421
Posts: 43
Joined: Tue Jul 15, 2014 6:58 pm

Re: Would greatly appreciate some help configuring notificat

Post by logic_bomb421 »

Alright, so it fails on the telnet. I get:

Code: Select all

Trying 74.125.131.27
Trying 2607:f8b0:400c:c03:1a...
telnet: Unable to connect to remote host: Network is unreachable
Edit: Alright so I was able to do a bit more troubleshooting. I was able to telnet to our company SMTP server from my Nagios machine, and I was able to send an email through that telnet to my personal company mailbox. This leads me to believe that the problem lies with how I've configured postfix. I'm just not sure of the right configuration at this point.

I've posted my postfix main.cf below (edited for privacy). Anywhere you see "MYDOMAIN", I edited the original out.

Code: Select all

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = nm02.MYDOMAIN.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = MYDOMAIN.com, nm02.MYDOMAIN.com, localhost.MYDOMAIN.com, localhost
relayhost = [smtp01.MYDOMAIN.com]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
chris.fixter
Posts: 22
Joined: Wed Jun 18, 2014 4:15 am

Re: Would greatly appreciate some help configuring notificat

Post by chris.fixter »

When you set up SMTP on nagios localhost with sendmail or postfix, you have two options. 1. the MTA (your smtp server) itself delivers email using DNS; 2. the MTA delivers emails using smarthost.

You weren't able to send email in the first place was due to that your MTA was setup using #1. And your further troubleshoot was toward to the #2 option. Each option has its requirement and topics but since you tested option #2 already it might make sense to go with #2.

To go with #2, right now what you are missing is the smarthost config in your postfix. I haven't worked with postfix much myself. Search for "postfix smarthost" on google should get plenty answers. After this is done you should be able to send notification to your company email. However, if you want to send to email addresses in other domains, due to MTA relay rules, you need to set up authentication to the smarthost(google again), or ask your company mail server admin to whitelist relay for your nagios server IP.
User avatar
millisa
Posts: 69
Joined: Thu Jan 16, 2014 11:13 pm
Location: Austin, TX
Contact:

Re: Would greatly appreciate some help configuring notificat

Post by millisa »

Based on your logs, you are definitely looking at a mail server config issue, not really a nagios issue.

For postfix, the way to relay all your local server's mail through another is with a transport entry.

In main.cf you would have a transport_maps line like this:

Code: Select all

transport_maps = hash:/etc/postfix/transport
(assuming your postfix config is in /etc/postfix - I believe Ubuntu uses /etc/postfix Quick reference for postfix on ubuntu: https://help.ubuntu.com/community/Postfix)
Location in the main.cf doesn't matter, throw it at the end - just make sure you don't have one already. If you already have the transport line, note the path if it's different from the above; don't add a second transport_maps, proceed to the next step.

That /etc/postfix/transport file may not exist; if it doesn't create it. If you see /etc/postfix/transport.db, do *not* edit that file (You will regenerate the transport.db later). If you find the file exists, it probably is filled with commented lines.
Just add this to the bottom of the transport file:

Code: Select all

* smtp:putyourIPforYourSMTPRelayHere
So if your internal mail server is at 192.168.100.213 you would add:

Code: Select all

* smtp:192.168.100.213
to the end of /etc/postfix/transport (again, *not* transport.db). Note - the only space is between the asterisk and the 'smtp', there is no space before or after the colon.

Now, use 'postmap' to regenerate the transport db:

Code: Select all

sudo postmap /etc/postfix/transport
(if you look in /etc/postfix you'll notice the 'transport.db' file now has a fresh new date)

Before you restart your postfix service you probably want to clean out your mailqueue. You can check what's in the queue using the 'mailq' command - your logs showed it deferring those mails to gmail and chances are good they are still sitting in your local queue . . . postfix defaults to 5 days for retrying deferred mail. If you want to just clear the mail queue in postfix you would use 'postsuper':

Code: Select all

sudo /usr/sbin/postsuper -d ALL
(I don't have an Ubuntu box handy to verify the path to postsuper, you may not need the /usr/sbin)

Restart your postfix service and your shiny new transport should be active. Check your /var/mail/mail.log and you should see it delivering new mail over to that IP you specified in the transport entry.

Any mail your Ubuntu server tries to send is going to go over to that smtp server if it can't deliver it locally now.
logic_bomb421
Posts: 43
Joined: Tue Jul 15, 2014 6:58 pm

Re: Would greatly appreciate some help configuring notificat

Post by logic_bomb421 »

So I followed your advice in adding a transport file with my smtp server's info and I'm still unable to get notifications. One interesting thing is that when I bring up the mail queue, at the top it says:

Code: Select all

postqueue: warning: Mail system is down -- accessing queue directly
Not sure if this is normal, but it seems to indicate that the mail system still isn't running.
Locked