redundant sendmail

[nagiosnext]

Hello Supporter,

i'm not an expert, so maybe someone of you can help me. I have actually installed nagios 3.3.1 with the official v1.4.15-plugins. Furthermore, I have configured nagios to use my internal Exchange which works pretty fine. But, in case my 1st Exchange Node goes down, I have a 2nd. How do I configure nagios to use the 2nd if the 1st one fails for sending mails? Anybody does that before? It's a SPOF to use only one of my nodes, but as stated before i'm not such a deep expert, so I found no idea in my head till yet how to configure sendmail. I would prefer not to use both at the same time.

Thanks in advance

[LHammonds]

If your primary server goes offline, how does your 2nd server take over? Does it assume the IP address or does DNS route it to the 2nd server?

If it takes over the IP, then simply use the IP in your sendmail configuration.

If consistency is maintained at the DNS level, use the DNS name in your sendmail configuration.

If neither of those scenarios happen and you can only reference the server by specifying one IP and then a different IP, you might want to create a simple in-between script for your sendmail. You would basically configure the notifications to use your sendemail script and pass it all the same parameters you would for your sendmail program. The script would then verify if your primary server is online...if so, direct the sendmail program to that server and pass along all the arguments. If the primary server is not online, then test to see if the secondary server is up...if so, direct the sendmail program to the secondary server with all the arguments. If neither server is online, you might want to save the notifications to a flat file so you don't loose them...which would mean you'd need to manually check for the file or write another script that looks for the file and tries to re-send the message(s) until the servers come back online.

LHammonds

[nagiosnext]

If neither of those scenarios happen and you can only reference the server by specifying one IP and then a different IP, you might want to create a simple in-between script for your sendmail. You would basically configure the notifications to use your sendemail script and pass it all the same parameters you would for your sendmail program. The script would then verify if your primary server is online...if so, direct the sendmail program to that server and pass along all the arguments. If the primary server is not online, then test to see if the secondary server is up...if so, direct the sendmail program to the secondary server with all the arguments. If neither server is online, you might want to save the notifications to a flat file so you don't loose them...which would mean you'd need to manually check for the file or write another script that looks for the file and tries to re-send the message(s) until the servers come back online.LHammonds

In this case I would prefer to have the 2nd SMTP as a backup, in case the 1st goes down. Due to the fact that I'm not a scripting guy, may you be able to give me some sort of idea how to handle/script it, or any suitable documentation which I can study?

[LHammonds]

I created my 1st Nagios function script and it seems like a fairly easy process.

As I mentioned earlier, you would make your own "sendmail" script that all of your notifications would call, instead of directly referencing the actual sendmail binary program. This configuration should be in this file --> /usr/local/nagios/etc/objects/contacts.cfg

Type the following commands to get a script file prep'd and ready for use:

Code: Select all

touch /usr/local/nagios/libexec/sendmail
chmod 0755 /usr/local/nagios/libexec/sendmail

Since you need a way to see if a server is "alive", you could ping the IP address to see if you get a response. Of course, this does not validate the mail server is running but it does let you know network connectivity to the server is good and your server is powered on which is typically 99% of what you need to know.

Edit that script and add a "ping" check to see if your primary mail server is alive, if so, call the real sendmail binary program, direct it to your 1st mail server and pass it all the parameters your script received.

If the ping fails, then ping your secondary mail server, if it responds, call the real sendmail binary program, direct it to your 2nd mail server and pass sit all the parameters your script received.

If both pings fail, you will need to figure out what you are going to do in that situation...such as creating a text file and sending all the parameters to it.

When done making changes to the script, make sure the ownership is set correctly:

Code: Select all

chown nagios:nagios  /usr/local/nagios/libexec/sendmail

As far as learning to script, it is not too terribly difficult. You mainly just have to write down in English what you are wanting to accomplish. Then look for examples that perform similar functions. In this case, if you are making a bash script, you would use google to search for things like "how to ping server in bash"

To test out a ping function, create a temporary file (and make sure to set it executable with chmod) and add the following code:

Code: Select all

#!/bin/bash
ping -c 1 -w 5 "$1" &>/dev/null
if [ $? -ne 0 ]; then
  ## Ping failed to get a response from the IP.
  echo "$1 is down"
else
  ## Ping returned a positive confirmation.
  echo "$1 is up"
fi

Now try it out by running the script and pass it an IP address of a server that is up and then an IP that is not in use. If all goes as expected, the script should spit out the correct response that the IP is up or down based on the ping reply (or lack thereof).

Now that you have a way to ping a server and a basic structure to control what action is taken based on the ping result, you can then add in some more conditions and tests.

Example:

Code: Select all

#!/bin/bash
SERVER1=192.168.107.14
SERVER2=192.168.107.15
ping -c 1 -w 5 ${SERVER1} &>/dev/null
if [ $? -ne 0 ]; then
  ## Server #1 is down
  ping -c 1 -w 5 ${SERVER2} &>/dev/null
  if [ $? -ne 0 ]; then
    ## Server #2 is down too!
    echo "Both servers are down.  We're screwed."
  else
    ## Server #2 is up
    echo "Sending mail to ${SERVER2}"
  fi
else
  ## Server #1 is up
  echo "Sending mail to ${SERVER1}"
fi

The server IP addresses are hard-coded into the script so no parameters are necessary when running this one.

If the primary server is up, it will say it is sending mail to that server.
If the primary server is down, it will check the secondary server. If #2 is up, it will say it is sending mail to #2.
If #2 is down, then you are faced with a dilemma and will need to figure out what to do in that scenario.

Now that you have ping figured out and the basic control structure, the next step is to replace the echo commands with the actual sendmail commands and pass the parameters that was sent to the script on to the sendmail binary.

Example:

/usr/bin/mail $1 $2 $3 $4 $5 $6 $7 $8

It also depends on "what" you are sending to the script. You may need to feed the programs option codes as well.

Example:

/usr/bin/mail -from "[email protected]" -to $1 -subject $2 -body $3