sendmail and Nagios Core Alerting

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: sendmail and Nagios Core Alerting

Post by tgriep »

I missed the G Suite account in the link so good catch on that.

The "illegal for recipient addresses" error looks like one or more of the contacts has either an invalid email address assigned to it or that the email address field is empty.
Check your contacts for that.

The /var/spool/clientmqueue is what sendmail uses to queue the emails from what I found on the internet. I have not found a good solution for that permission error so you may have to go to Sendmail's forums for help. Also, disable selinux if it is enabled.

You said that you can send emails using telnet or sendmail from the command line, did you receive them?

The authfile looks like it should work but you may have to fill in the full name "smtp.gmail.com" for the relay server.

Code: Select all

AuthInfo:smtp.gmail.com "U:myemail@gmail.com" "P:mypassword" "M:PLAIN"
Be sure to check out our Knowledgebase for helpful articles and solutions!
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

Ok I will check on the permissions in sendmail forums. Yes I can send emails using telnet and sendmail commands and receive them. If Nagios tries to send the email I get the following below: It thinks the Recipient is nagios.apifw.com ??

Action: failed
Status: 5.1.1
Remote-MTA: DNS; [127.0.0.1]
Diagnostic-Code: SMTP; 550 5.1.1 <is@nagios.apifw.com>... User unknown
Last-Attempt-Date: Thu, 19 Dec 2019 10:35:17 -0600

Final-Recipient: RFC822; nagios.apifw.com$@nagios.apifw.com
Action: failed
Status: 5.1.1
Remote-MTA: DNS; [127.0.0.1]
Diagnostic-Code: SMTP; 550 5.1.1 <nagios.apifw.com$@nagios.apifw.com>... User unknown
Last-Attempt-Date: Thu, 19 Dec 2019 10:35:17 -0600

Final-Recipient: RFC822; Host@nagios.apifw.com
Action: failed
Status: 5.1.1
Remote-MTA: DNS; [127.0.0.1]
Diagnostic-Code: SMTP; 550 5.1.1 <Host@nagios.apifw.com>... User unknown
Last-Attempt-Date: Thu, 19 Dec 2019 10:35:17 -0600

--xBJGZH0Y008601.1576773317/nagios.apifw.com
Content-Type: message/rfc822

Return-Path: <root>
Received: (from root@localhost)
by nagios.apifw.com (8.14.7/8.14.7/Submit) id xBJGZH0X008601;
Thu, 19 Dec 2019 10:35:17 -0600
Date: Thu, 19 Dec 2019 10:35:17 -0600
From: root <root>
Message-Id: <201912191635.xBJGZH0X008601@nagios.apifw.com>

***** Nagios *****

Notification Type: $
Host: nagios.apifw.com$
State: $
Address: $
Info: $

Date/Time: $

--xBJGZH0Y008601.1576773317/nagios.apifw.com--
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

Here is my contacts.cfg



define contact {

contact_name nagiosadmin ; Short name of user
use generic-contact ; inherit from above
alias Nagios Admin ; Full name of user
email jason.payne@payneintheit.com ; your email

}


Here is my nagios.cfg




# OBJECT CONFIGURATION FILE(S)
# These are the object configuration files in which you define hosts,
# host groups, contacts, contact groups, services, etc.
# You can split your object definitions across several config files
# if you wish (as shown below), or keep them all in a single config file.

# You can specify individual object config files as shown below:
cfg_file=/usr/local/nagios/etc/objects/commands.cfg
cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg
cfg_file=/usr/local/nagios/etc/objects/templates.cfg

# Definitions for monitoring the local (Linux) host
cfg_file=/usr/local/nagios/etc/objects/localhost.cfg

# Definitions for monitoring a Windows machine
cfg_file=/usr/local/nagios/etc/objects/windows.cfg
cfg_file=/usr/local/nagios/etc/objects/windows2.cfg


###############################################################################
#
# CONTACT GROUPS
#
###############################################################################

# We only have one contact in this simple configuration file, so there is
# no need to create more than one contact group.

define contactgroup {

contactgroup_name admins
alias Nagios Administrators
members nagiosadmin
}


Here is my windows.cfg file that points to one of my servers being monitored:



Define a host for the Windows machine we'll be monitoring
# Change the host_name, alias, and address to fit your situation

define host {

use windows-server ; Inherit default values from a template
host_name ALPSBS2K11 ; The name we're giving to this host
alias ALPSBS2K11 ; A longer name associated with the host
address 192.168.1.8 ; IP address of the host
contact_groups admins ; contact person
}



###############################################################################
#
# HOST GROUP DEFINITIONS
#
###############################################################################

# Define a hostgroup for Windows machines
# All hosts that use the windows-server template will automatically be a member of this group

define hostgroup {

hostgroup_name windows-servers ; The name of the hostgroup
alias Windows Servers ; Long name of the group
}


define service {

use generic-service
host_name ALPSBS2K11
service_description Uptime
check_command check_nt!UPTIME
}




Here is my windows2.cfg file that points to my 2nd server being monitored:

define host {

use windows-server ; Inherit default values from a template
host_name ALPBACKSRV2008 ; The name we're giving to this host
alias ALPBACKSRV2008 ; A longer name associated with the host
address 192.168.1.10 ; IP address of the host
contact_groups admins ; contact person
}



###############################################################################
#
# HOST GROUP DEFINITIONS
#
###############################################################################

# Define a hostgroup for Windows machines
# All hosts that use the windows-server template will automatically be a member of this group


###############################################################################
#
# SERVICE DEFINITIONS
#
###############################################################################

# Create a service for monitoring the version of NSCLient++ that is installed
# Change the host_name to match the name of the host you defined above

define service {

use generic-service
host_name ALPBACKSRV2008
service_description NSClient++ Version
check_command check_nt!CLIENTVERSION
}



I need contact group for every service dont I? Maybe that is why it is not alerting me for services.
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

I placed the 'contact_groups admins'

inside my service funtion and not just my host function.. and I am still not getting the alert. Somewhere my Nagios configuration is broken.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: sendmail and Nagios Core Alerting

Post by tgriep »

OK, you cannot type in the email command in a shell as the macros are not defined so it will fail.

The nagios process has to run the commands which will fill in the macros with valid data.

When nagios generates a notification, it will replace all of these macros in the command with valid data.

Code: Select all

$NOTIFICATIONTYPE$
$HOSTNAME$
$HOSTSTATE$
$HOSTADDRESS$
$HOSTOUTPUT$
$LONGDATETIME$
$NOTIFICATIONTYPE$
$HOSTNAME$
$HOSTSTATE$
$CONTACTEMAIL$
When you run it from a shell, the macros do not have valid data so it fails.


To test it from Nagios, open that host in the Web interface and submit a "Send custom host notification" by clicking in the link in the Host Commands menu.
That will simulate a notification and send an email from nagios. See if you get that email.

One more thing, you should set a sender address in the email command as the server is sending from root@localhost and most email servers will block that.
Add the following to the email command right after the sendmail command and that sill set the from address.

Code: Select all

-s  jason.payne@payneintheit.com
Be sure to check out our Knowledgebase for helpful articles and solutions!
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

If I use the command:

[root@nagios objects]# /usr/sbin/sendmail -s jason.payne@payneintheit.com
test
.

then i get an email sent to jason.payne@payneintheit.com from root<spatchephunk@gmail.com>

because I am using smtp.gmail.com and authenticating as spatchephunk@gmail.com


HOWEVER, Nagios sends the command:

define command {

command_name notify-host-by-email
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/sbin/sendmail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
}

and it doesn't do ANYTHING. I get nothing, even if I send a custom host notification, and the only thing I can see in the logs now is:

NOQUEUE: SYSERR(nagios): can not chdir(/var/spool/clientmqueue/): Permission denied
Dec 20 16:49:40 nagios sendmail[2011]: NOQUEUE: SYSERR(nagios): can not write to queue directory /var/spool/clientmqueue/ (RunAsGid=1000, required=51): Permission denied
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

in my nagios.log I am getting:

[1576881875] EXTERNAL COMMAND: SEND_CUSTOM_HOST_NOTIFICATION;ALPBACKSRV2008;3;Nagios Admin;test
[1576881875] HOST NOTIFICATION: nagiosadmin;ALPBACKSRV2008;CUSTOM (UP);notify-host-by-email;PING OK - Packet loss = 0%, RTA = 0.69 ms;Nagios Admin;test
[1576881875] wproc: NOTIFY job 2 from worker Core Worker 1238 is a non-check helper but exited with return code 71
[1576881875] wproc: host=ALPBACKSRV2008; service=(none); contact=nagiosadmin
[1576881875] wproc: early_timeout=0; exited_ok=1; wait_status=18176; error_code=0;
[1576881875] wproc: stderr line 01: /usr/bin/printf: write error: Broken pipe
[1576881875] wproc: stdout line 01: WARNING: RunAsUser for MSP ignored, check group ids (egid=1000, want=51)
[1576881875] wproc: stdout line 02: can not write to queue directory /var/spool/clientmqueue/ (RunAsGid=1000, required=51): Permission denied
[1576882014] SERVICE NOTIFICATION: nagiosadmin;ALPSBS2K11;W3SVC;CRITICAL;notify-service-by-email;W3SVC: Not found
[1576882014] wproc: NOTIFY job 3 from worker Core Worker 1237 is a non-check helper but exited with return code 71
[1576882014] wproc: host=ALPSBS2K11; service=W3SVC; contact=nagiosadmin
[1576882014] wproc: early_timeout=0; exited_ok=1; wait_status=18176; error_code=0;
[1576882014] wproc: stdout line 01: WARNING: RunAsUser for MSP ignored, check group ids (egid=1000, want=51)
[1576882014] wproc: stdout line 02: can not write to queue directory /var/spool/clientmqueue/ (RunAsGid=1000, required=51): Permission denied
[1576882180] SERVICE NOTIFICATION: nagiosadmin;ALPBACKSRV2008;C:\ Drive Space;WARNING;notify-service-by-email;c:\ - total: 232.85 Gb - used: 196.01 Gb (84%) - free 36.83 Gb (16%)
[1576882180] wproc: NOTIFY job 5 from worker Core Worker 1238 is a non-check helper but exited with return code 71
[1576882180] wproc: host=ALPBACKSRV2008; service=C:\ Drive Space; contact=nagiosadmin
[1576882180] wproc: early_timeout=0; exited_ok=1; wait_status=18176; error_code=0;
[1576882180] wproc: stdout line 01: WARNING: RunAsUser for MSP ignored, check group ids (egid=1000, want=51)
[1576882180] wproc: stdout line 02: can not write to queue directory /var/spool/clientmqueue/ (RunAsGid=1000, required=51): Permission denied
[1576882292] SERVICE NOTIFICATION: nagiosadmin;ALPSBS2K11;C:\ Drive Space;WARNING;notify-service-by-email;c:\ - total: 149.90 Gb - used: 124.66 Gb (83%) - free 25.24 Gb (17%)
[1576882292] wproc: NOTIFY job 5 from worker Core Worker 1241 is a non-check helper but exited with return code 71
[1576882292] wproc: host=ALPSBS2K11; service=C:\ Drive Space; contact=nagiosadmin
[1576882292] wproc: early_timeout=0; exited_ok=1; wait_status=18176; error_code=0;
[1576882292] wproc: stdout line 01: WARNING: RunAsUser for MSP ignored, check group ids (egid=1000, want=51)
[1576882292] wproc: stdout line 02: can not write to queue directory /var/spool/clientmqueue/ (RunAsGid=1000, required=51): Permission denied
spatchE
Posts: 24
Joined: Tue Oct 30, 2018 8:05 pm

Re: sendmail and Nagios Core Alerting

Post by spatchE »

I ran chmod 777 -R on the /var/spool/clientmqueue directory and now custom host notifications are working.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: sendmail and Nagios Core Alerting

Post by scottwilkerson »

spatchE wrote:I ran chmod 777 -R on the /var/spool/clientmqueue directory and now custom host notifications are working.
Great!

Locking thread
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked