SNMP Traps sending two emails to same person

[jenstar13]

Hi
I can not figure where it's set up to send two emails, but i think i see it in the nagios.log. doing the 2 emails, then i checked the service and i do not see that user listed twice
can you help? Thank you in advance


(I had to disguise hostname and email recipient)
[1533569440] SERVICE NOTIFICATION: ussername;hostname.xxx.net;SNMP Traps;CRITICAL;xi_service_notification_handler;Accelerator Board Status Change (3038): Cache module board status is now 3 for board located in Slot 0. / sysName (OCTETSTR):hostname.xxx.net enterprises.232.11.2.11.1 ():8 enterprises.232.3.2.2.1.1.20.0 ():Slot 0 enterprises.232.3.2.2.1.1.2.0 ():39 enterprises.232.3.2.2.2.1.1.0 ():0 enterprises.232.3.2.2.2.1.11.0 ():PAAVP9SXZCZK enterprises.232.3.2.2.2.1.12.0 ():524288 enterprises.232.3.2.2.2.1.2.0 ():3 enterprises.232.3.2.2.2.1.5.0 ():2

[1533569440] SERVICE NOTIFICATION: username;hostname.xxx.net;SNMP Traps;CRITICAL;notify-service-by-email;Accelerator Board Status Change (3038): Cache module board status is now 3 for board located in Slot 0. / sysName (OCTETSTR):hostname.xxx.net enterprises.232.11.2.11.1 ():8 enterprises.232.3.2.2.1.1.20.0 ():Slot 0 enterprises.232.3.2.2.1.1.2.0 ():39 enterprises.232.3.2.2.2.1.1.0 ():0 enterprises.232.3.2.2.2.1.11.0 ():PAAVP9SXZCZK enterprises.232.3.2.2.2.1.12.0 ():524288 enterprises.232.3.2.2.2.1.2.0 ():3 enterprises.232.3.2.2.2.1.5.0 ():2

I even looked in the file on the server for it


define service {
host_name hostname.xxx.net
service_description SNMP Traps
use xiwizard_passive_service
max_check_attempts 1
check_interval 1
retry_interval 1
check_period xi_timeperiod_24x7
notification_interval 60
notification_period xi_timeperiod_24x7
contacts username
stalking_options n
_xiwizard passiveobject
register 1
}

[lmiltchev]

[1533569440] SERVICE NOTIFICATION: ussername;hostname.xxx.net;SNMP Traps;CRITICAL;xi_service_notification_handler;Accelerator Board Status Change (3038): Cache module board status is now 3 for board located in Slot 0. / sysName (OCTETSTR):hostname.xxx.net enterprises.232.11.2.11.1 ():8 enterprises.232.3.2.2.1.1.20.0 ():Slot 0 enterprises.232.3.2.2.1.1.2.0 ():39 enterprises.232.3.2.2.2.1.1.0 ():0 enterprises.232.3.2.2.2.1.11.0 ():PAAVP9SXZCZK enterprises.232.3.2.2.2.1.12.0 ():524288 enterprises.232.3.2.2.2.1.2.0 ():3 enterprises.232.3.2.2.2.1.5.0 ():2

[1533569440] SERVICE NOTIFICATION: username;hostname.xxx.net;SNMP Traps;CRITICAL;notify-service-by-email;Accelerator Board Status Change (3038): Cache module board status is now 3 for board located in Slot 0. / sysName (OCTETSTR):hostname.xxx.net enterprises.232.11.2.11.1 ():8 enterprises.232.3.2.2.1.1.20.0 ():Slot 0 enterprises.232.3.2.2.1.1.2.0 ():39 enterprises.232.3.2.2.2.1.1.0 ():0 enterprises.232.3.2.2.2.1.11.0 ():PAAVP9SXZCZK enterprises.232.3.2.2.2.1.12.0 ():524288 enterprises.232.3.2.2.2.1.2.0 ():3 enterprises.232.3.2.2.2.1.5.0 ():2

It is possible that this xi user/contact is defined with two notification handlers - xi_service_notification_handler and notify-service-by-email. Can you show us the contact's config, along with any relevant templates that are used by the contact?

Also, check if you have any multiple nagios processes running on this system.

Code: Select all

ps -ef | grep nagios.cfg | grep -v grep

[jenstar13]

Sorry, i never got notified that you had replied
for the nagios processes, i see the master and the child

ps -ef | grep nagios.cfg | grep -v grep
nagios 2127 1 0 Aug06 ? 00:01:15 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 2145 2127 0 Aug06 ? 00:00:20 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg


for the contact it looks like only one group, I anon'd his user info
define contact {
contact_name username
alias User Name
host_notifications_enabled 1
service_notifications_enabled 1
host_notification_period 24x7
service_notification_period 24x7
host_notification_options d,u,r,f,s,
service_notification_options w,u,c,r,f,s,
host_notification_commands notify-host-by-email
service_notification_commands notify-service-by-email
can_submit_commands 1
retain_status_information 1
retain_nonstatus_information 1
email [email protected]
use xi_contact_generic
}

and the trap definition

define service {
host_name prdnet01.xxx.net
service_description SNMP Traps
use xiwizard_passive_service
max_check_attempts 1
check_interval 1
retry_interval 1
check_period xi_timeperiod_24x7
notification_interval 60
notification_period xi_timeperiod_24x7
contacts Usename from above
stalking_options n
_xiwizard passiveobject
register 1
}

[lmiltchev]

Your contact config looks normal. The "xi_contact_generic" template uses the "xi_host_notification_handler" and the "xi_service_notification_handler" commands, but they *should* be overwritten by the "notify-host-by-email" and the "notify-service-by-email" commands, that are defined locally (on a contact level). I am not sure why both commands were used - you don't use additive inheritance, so unless you have some kind of global event handler, I don't see how these notifications are sent to both places.

Open up the objects.cache file in a text editor and look at your contact definition. What are the commands that you see on the two lines, listed below?

Code: Select all

service_notification_commands
host_notification_commands

Let try the following - you have all of the alert settings defined on a contact level, so you don't need to use the "xi_contact_generic" template. Remove it from the config, and apply configuration. Test your notifications again. How many notifications were sent?

[jenstar13]

The user said that as of 10:41 today he received two notification for the same trap at the same time

Here is the information from the object cache file with his info anon'd but every username below is the person getting two emails

define host {
host_name prdnet01.xxxx.net
alias prdnet01.xxxx.net
address prdnet01.xxxx.net
check_period xi_timeperiod_24x7
check_command check_dummy!0!"No data received yet."
contacts (problem username)
notification_period xi_timeperiod_24x7
initial_state o
importance 0
check_interval 5.000000
retry_interval 1.000000
max_check_attempts 5
active_checks_enabled 0
passive_checks_enabled 1
obsess 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 1
flap_detection_options a
freshness_threshold 0
check_freshness 0
notification_options a
notifications_enabled 1
notification_interval 60.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
icon_image passiveobject.png
statusmap_image passiveobject.png
retain_status_information 1
retain_nonstatus_information 1
_XIWIZARD passiveobject
}

define service {
host_name prdnet01.xxxx.net
service_description SNMP Traps
check_period xi_timeperiod_24x7
check_command check_dummy!0!"No data received yet."
contacts (problem username)
notification_period xi_timeperiod_24x7
initial_state o
importance 0
check_interval 1.000000
retry_interval 1.000000
max_check_attempts 1
is_volatile 0
parallelize_check 1
active_checks_enabled 0
passive_checks_enabled 1
obsess 1
event_handler_enabled 1
low_flap_threshold 0.000000
high_flap_threshold 0.000000
flap_detection_enabled 0
flap_detection_options a
freshness_threshold 0
check_freshness 0
notification_options a
notifications_enabled 1
notification_interval 60.000000
first_notification_delay 0.000000
stalking_options n
process_perf_data 1
retain_status_information 1
retain_nonstatus_information 1
_XIWIZARD passiveobject
}


define contact {
contact_name username
alias username
service_notification_period 24x7
host_notification_period 24x7
service_notification_options r,w,u,c,f,s
host_notification_options r,d,u,f,s
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
email [email protected]
minimum_importance 0
host_notifications_enabled 1
service_notifications_enabled 1
can_submit_commands 1
retain_status_information 1
retain_nonstatus_information 1
}

[lmiltchev]

Can you PM me your profile (profile.zip)?

Admin > System Profile > Download Profile

Also, in the PM, type the the name of the contact, service, and the host this service is attached to. Thank you!

[lmiltchev]

I sent a reply via PM.

[lmiltchev]

I am locking this topic as the issue has been resolved.

Note: For users who may have the same/similar issue - the problem was caused by the fact that there were 2 different host/service notification handlers defined in the contact template, used by the contact. So, 2 emails were sent for each problem.