Page 1 of 1

Nagios E-mail Notifications

Posted: Wed Feb 29, 2012 10:55 am
by DLAS
Hi all,
I'm new to Nagios so let me tell you what I've done so far -

Configured Nagios on Ubuntu as per this guide - http://nagios.sourceforge.net/docs/3_0/ ... buntu.html
I've installed NSclient++ on a server in my local network
I've defined the services to monitor on this server

All of the above works correctly.

Now I've attempted to setup notifications in Nagios. I've followed the steps listed here http://www.thegeekstuff.com/2009/06/4-s ... ification/ and defined my contacts and who to notify when the services go down.

I created a G-mail account specifically to send notifcations. I installed Postfix and set it up as a satellite system (Although I wasn't sure what to pick for some settings in the main.cf) to use the G-mail SMTP server. I followed this guide roughly for that http://ubuntu-tutorials.com/2008/11/11/ ... pgmailcom/

Now, if I access Nagios through the web interface and select notifcations from the navigation bar on the left, I see a long log of when notifications were supposed to have been sent but nothing ever makes it through to the defined contacts. There are no errors in /var/log/mail.log either, the only thing listed in that log is when the Postfix services were started.

I've banged my head against this for a week now and can't find a solution, can anyone point me in the right troubleshooting direction? I'm probably least confident about the Postfix config in the main.cf file.

Big thanks for reading,
DLAS

Re: Nagios E-mail Notifications

Posted: Wed Feb 29, 2012 7:52 pm
by jsmurphy
Well take Nagios out of the equation for the moment... can you use the system mail utility to send mail to the address via postfix? If that's the case we can safely assume that your problem is not Nagios.

If you check mail.log it should have bounced mail messages or something that will tell you why that the mail is not getting through. You can also add -v to the process start in the init.d script to turn on verbose logging for postfix which may also provide some greater insight.

Re: Nagios E-mail Notifications

Posted: Fri Mar 02, 2012 10:24 am
by DLAS
I think I've finally had a breakthrough with this, it was a problem with Postfix and not with Nagios... Thanks for the help! Something to do with G-mail SMTP and the certificate I was using.

Now I've finally got a working system (albeit very basic, I'm only monitoring 1 local windows server) I do have a few more questions:

Is there any way to get around having to install Nsclient++ on to the servers I want to to monitor? I'm not keen on having to install this on everything.

and

How would I go about setting up the monitoring on a server outside of my local network? Is is the same procedure as inside the local network? The servers in question all have internet facing IP addresses that I can use to access through rdp etc...

and

What's the quickest and easiest way to remove E-mail alerts from something not using the web GUI?

Thanks

Re: Nagios E-mail Notifications

Posted: Sun Mar 04, 2012 6:33 pm
by jsmurphy
1. Well that really depends what level of monitoring you want on windows... The only interface available without some kind of agent realistically will be WMI... so as long as you can get everything you want via WMI then, no you don't need an agent. The agent will however make your life a better place.

2. Yep, same deal as with an internally hosted box... just make sure that if you do put a client on there that it is properly secured and you are judicious about what holes you are putting in the firewall, don't punch anything inbound.

3. Take a look at this: http://www.techopsguys.com/2010/01/05/a ... l-replies/ which has a tutorial on acknowledging alerts via email.

Re: Nagios E-mail Notifications

Posted: Thu Mar 08, 2012 1:06 pm
by DLAS
Thanks for the advice, after looking into your suggestions I decided to go with just installing Nsclient on each server. I've also implemented the acknowledgement you suggested which is a really nice feature!

So I'm now monitoring a few of my remote machines based in a data centre. So far I've just monitored the services that are recommended in the Monitoring Windows quickstart guide. Off the top of my head (explorer, memory, HDD, CPU explorer etc...)

Are there a list of service definitions somewhere that I can use to add to my windows.cfg?

I'd really like to monitor the SQL services on an SQL box. Does this have to be achieved through addons?

Can I also make my own definitions? For example, we have an application that relies on the running of 4-5 services, can I define these myself? If so, are there any guides or tutorials for this?

Re: Nagios E-mail Notifications

Posted: Thu Mar 08, 2012 6:08 pm
by jsmurphy
To give the cop out answer, every ones environment is unique so you will need to sort of discover what you think is important to monitor on your windows machines so there's no real definitive list. That said we monitor the following basics on every server: Hard disk usage, CPU usage, RAM usage, Any automatic service failure and event log messages for unexpected reboots.

SQL monitoring is a tricky beast... ask your friendly neighbourhood DBA's what they expect to know about... some of it you may need to script yourself, some of it you might use addons, some DBA's have their own thing going and don't expect much from monitoring at all. I've found DBA and Middleware to usually be the most painful monitoring tasks I undertake due to complex dependency chains and interfacing with some of their stuff is an uphill battle carrying a sack full of bricks. If you monitor all automatic services on a windows box this will at least catch when the SQL service stops all together.

For multiple services we are getting into the territory of service dependencies and "business process", there are a couple of business process addons you can download from Nagios exchange to assist with the grouping of services together and creating business logic... but if you just want something native and less involved:
http://nagios.sourceforge.net/docs/3_0/ ... dependency
http://nagios.sourceforge.net/docs/3_0/ ... ncies.html

You can customize very large portions of Nagios with the config files... I would bookmark the contents page of the above document because it really is the best resource for learning how to make Nagios do what you want.

Re: Nagios E-mail Notifications

Posted: Fri Mar 09, 2012 4:04 am
by DLAS
Thanks, good links, just working my way through them now.

After adding a few more Windows hosts i can no longer restart the Nagios daemon on my monitoring box, it says I have errors in my config. However, these errors aren't picked up by the pre-flight check, it tells me everything is fine. If I restart the whole machine then Nagios starts up and works fine, so whatever the error is it's not serious enough to stop it from actually starting, it just prevents the command line restart.

Is there anywhere I can look that will let me narrow down what this could be? I've always found the config test has worked in the past but whatever this error is doesn't register with that.

Cheers

Re: Nagios E-mail Notifications

Posted: Mon Mar 12, 2012 5:39 pm
by jsmurphy
I can't say I've ever seen that happen before where the daemon fails but the pre-flight succeeds (at least not for many, many years and definitely not with version 3.x)...

You might have to do some process of elimination work on this one I'm afraid... remove checks until you find the one causing the problem and post it up and we'll see if we can work out what's causing it.

Re: Nagios E-mail Notifications

Posted: Thu Mar 15, 2012 4:44 am
by DLAS
Thanks jsmurphy, I eventually narrowed it down to some brackets that were present in windows.cfg where they shouldn't have been. Which leads me on to my next question -

How can I escape the brackets in a service I define? So for example:

define service{
use generic-service
host_name SQL
service_description SQL Server --CFP--
check_command check_nt!SERVICESTATE!-d SHOWALL -l "SQL Server
(CFP)"
}


Nagios has a problem with "(CFP)" in the above service definition but that's how the service is listed in Windows, can I escape these characters in some way?

Thanks again

Re: Nagios E-mail Notifications

Posted: Thu Mar 15, 2012 5:17 pm
by jsmurphy
I think you can either use backslash \ so...
check_command check_nt!SERVICESTATE!-d SHOWALL -l "SQL Server \(CFP\)"

or you can encase everything in single quotes which I believe is the preferred solution...
check_command check_nt!SERVICESTATE!-d SHOWALL -l 'SQL Server (CFP)'

Is that new line in the config file? The one between SQL Server and (CFP)? You may want to remove that, I think that might be your issue if it is. Brackets normally don't stop it from starting (might cause the check to fail though) but the new line in the middle of a string might.