Notification (by email, or command) failed

[nabucho]

Dear all,

We just deploy our new monitor system and we have a problem with the notification process.

We configured the contact to be notified by the default "notify-service-by-mail" which execute /usr/bin/printf "%b" "***** Nagios Monitor XI Alert *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$

If we execute this command by the comamnd line (logged as nagios) no problem we received the mail.
If we test the mail from the webinterface (https://<server>/nagiosxi/admin/testemail.php), the mail is receuived too..

But once we have an event on the host... we do not receive the notification. (we tried with another command to log the arguments but no way to execute it too !)
In the nagios.log it seems that we have the notification... :
[1579774963] HOST ALERT: <fqdn>;DOWN;SOFT;4;CRITICAL - <fqdn>: Host unreachable @ <ip>. rta nan, lost 100%
[1579775033] HOST NOTIFICATION: xxx;<fqdn>;DOWN;notify-host-by-email;CRITICAL - <fqdn>: Host unreachable @ <ip>. rta nan, lost 100%

Any idea why the "command" is not executed ?

System profile :
Nagios XI version: 5.6.9
XI installed from: source
XI UUID: feae2ff8-06ed-4d25-a958-0efaa00ac9e1
Release info: <server> 4.1.12-124.34.1.el7uek.x86_64 x86_64
Red Hat Enterprise Linux Server release 7.7 (Maipo)
Gnome is not installed
Apache Information
PHP Version: 5.4.16
Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Server Name: <server>
Server Address: <ipserv>
Server Port: 443
Date/Time
PHP Timezone: Europe/Paris
PHP Time: Mon, 27 Jan 2020 13:23:46 +0100
System Time: Mon, 27 Jan 2020 13:23:46 +0100

Part of objects.cache :
objects.cache-define command {
objects.cache: command_name notify-host-by-email
objects.cache- command_line /usr/bin/printf "%b" "***** Nagios Monitor XI Alert *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
objects.cache- }

[...]

objects.cache-define command {
objects.cache: command_name notify-service-by-email
objects.cache- command_line /usr/bin/printf "%b" "***** Nagios Monitor XI Alert *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
objects.cache- }

[...]

objects.cache-define contact {
objects.cache- contact_name <user>
objects.cache- alias <user>
objects.cache- service_notification_period xi_timeperiod_24x7
objects.cache- host_notification_period xi_timeperiod_24x7
objects.cache- service_notification_options r,w,u,c,f,s
objects.cache- host_notification_options r,d,u,f,s
objects.cache: service_notification_commands notify-service-by-email
objects.cache: host_notification_commands notify-host-by-email
objects.cache- email <email>
objects.cache- minimum_importance 0
objects.cache- host_notifications_enabled 1
objects.cache- service_notifications_enabled 1
objects.cache- can_submit_commands 1
objects.cache- retain_status_information 1
objects.cache- retain_nonstatus_information 1
objects.cache- }

[...]

objects.cache-define host {
objects.cache: host_name <fqdn>
objects.cache: alias <hostname>
objects.cache: address <fqdn>
objects.cache- check_period xi_timeperiod_24x7
objects.cache- check_command check_xi_host_ping!3000.0!80%!5000.0!100%
objects.cache- contacts xxx
objects.cache- contact_groups yyyy
objects.cache- notification_period xi_timeperiod_24x7
objects.cache- initial_state o
objects.cache- importance 0
objects.cache- check_interval 5.000000
objects.cache- retry_interval 1.000000
objects.cache- max_check_attempts 5
objects.cache- active_checks_enabled 1
objects.cache- passive_checks_enabled 1
objects.cache- obsess 1
objects.cache- event_handler_enabled 0
objects.cache- low_flap_threshold 0.000000
objects.cache- high_flap_threshold 0.000000
objects.cache- flap_detection_enabled 1
objects.cache- flap_detection_options a
objects.cache- freshness_threshold 0
objects.cache- check_freshness 0
objects.cache- notification_options r,d,u,f
objects.cache- notifications_enabled 1
objects.cache- notification_interval 60.000000
objects.cache- first_notification_delay 0.000000
objects.cache- stalking_options n
objects.cache- process_perf_data 1
objects.cache- icon_image win_server.png
objects.cache- statusmap_image win_server.png
objects.cache- retain_status_information 1
objects.cache- retain_nonstatus_information 1
objects.cache- _XIWIZARD windowsserver
objects.cache- }

[tgriep]

Take a look at the /var/log/maillog file on the server at the times the notifications were generated in Nagios XI.
Do you see the entries for when the notifications were generated?
DO you see and errors on why the emails are not getting delivered?

[nabucho]

Hi,

Sorry I completelyforgot to post this part during my fist message ...

The test is shown into the /var/log/mail as delivred.. but the automatic notification does not reccording something... It seems that the notification process stop somewhere before executing the action

We use a gearman module... and the host/service is checked by another worker... do you think that the problem come from that ?
How to debug the notification process ?

[nabucho]

Problem fixed...

In fact we are using the mode gearman and the notifications were blocked on the gearmand process.

Thanks a lot this post could be closed

[mbellerue]

Awesome! Glad you were able to get it working! I will close the thread.