SRMHOST08A notification for C drive full
Code: Select all
[1411110576] SERVICE NOTIFICATION: itnetworksupport;SRMSHOST08A;C Drive;CRITICAL;notify-service-by-email;SNMP CRITICAL - C:\ Label: Serial Number 30ae4373 at 82% with 5,220 of 30,482 MB free
[1411110576] SERVICE NOTIFICATION: itcim;SRMSHOST08A;C Drive;CRITICAL;notify-service-by-email;SNMP CRITICAL - C:\ Label: Serial Number 30ae4373 at 82% with 5,220 of 30,482 MB free
1411110576 = Fri, 19 Sep 2014 07:09:36 GMT, 15:09 localtime. Corresponding maillogs:
Code: Select all
Sep 19 15:09:36 SIT-NAGIOS postfix/pickup[10150]: 5609F60062: uid=106 from=<nagios>
Sep 19 15:09:36 SIT-NAGIOS postfix/cleanup[29168]: 5609F60062: message-id=<20140919070936.5609F60062@SIT-NAGIOS>
Sep 19 15:09:36 SIT-NAGIOS postfix/qmgr[976]: 5609F60062: from=<nagios@SIT-NAGIOS>, size=619, nrcpt=1 (queue active)
Sep 19 15:09:36 SIT-NAGIOS postfix/smtp[29172]: 5609F60062: to=<[email protected]>, relay=sit-smtp01.xxx.com[10.16.0.142]:25, delay=0.42, delays=0.06/0.03/0.01/0.32, dsn=2.6.0, status=sent (250 2.6.0 <20140919070936.5609F60062@SIT-NAGIOS> Queued mail for delivery)
Sep 19 15:09:36 SIT-NAGIOS postfix/qmgr[976]: 5609F60062: removed
------
Sep 19 15:09:36 SIT-NAGIOS postfix/pickup[10150]: 6CE616008B: uid=106 from=<nagios>
Sep 19 15:09:36 SIT-NAGIOS postfix/cleanup[29168]: 6CE616008B: message-id=<20140919070936.6CE616008B@SIT-NAGIOS>
Sep 19 15:09:36 SIT-NAGIOS postfix/qmgr[976]: 6CE616008B: from=<nagios@SIT-NAGIOS>, size=619, nrcpt=1 (queue active)
Sep 19 15:09:36 SIT-NAGIOS postfix/smtp[29177]: 6CE616008B: to=<[email protected]>, relay=sit-smtp01.xxx.com[10.16.0.142]:25, delay=0.35, delays=0.01/0.03/0.02/0.29, dsn=2.6.0, status=sent (250 2.6.0 <20140919070936.6CE616008B@SIT-NAGIOS> Queued mail for delivery)
Sep 19 15:09:36 SIT-NAGIOS postfix/qmgr[976]: 6CE616008B: removed
It sends both mails to itnetworksupport@xxx, (contact itcim has the just
[email protected] for it email., contact itnetworksupport also has just
[email protected]).
Starting with the service definition:
Code: Select all
define service{
use generic-service-cim
host_name SRMSHOST08A
service_description C Drive
check_command check_snmp_disk!public!C!70%!80%
notifications_enabled 1
}
No specific contacts defined here, just the template generic-service-cim
Template generic-service-cim:
Code: Select all
define service{
name generic-service-cim
use generic-service-template
contacts itcim
#contact_groups cim-admins
}
Has contacts itcim. This accounts for the one of the two notifications. It is missing the register 0 line. It includes the template generic-service-template:
Code: Select all
define service{
name generic-service-template ; The 'name' of this service template
active_checks_enabled 1 ; Active service checks are enabled
passive_checks_enabled 1 ; Passive service checks are enabled/accepted
parallelize_check 1 ; Active service checks should be parallelized (disabling this can lead to major performance problems)
obsess_over_service 1 ; We should obsess over this service (if necessary)
check_freshness 0 ; Default is to NOT check service 'freshness'
notifications_enabled 0 ; Service notifications are enabled
event_handler_enabled 1 ; Service event handler is enabled
flap_detection_enabled 1 ; Flap detection is enabled
failure_prediction_enabled 1 ; Failure prediction is enabled
process_perf_data 1 ; Process performance data
retain_status_information 1 ; Retain status information across program restarts
retain_nonstatus_information 1 ; Retain non-status information across program restarts
notification_interval 10 ; Only send notifications on status change by default.
is_volatile 0
check_period 24x7
normal_check_interval 5
retry_check_interval 1
max_check_attempts 6
notification_period 24x7
notification_options w,u,c,r
register 0 ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL SERVICE, JUST A TEMPLATE!
}
This is where the chain ends and does not account for a second mail for the C drive service alert. If it was set to use 'generic-service' instead of 'generic-service-template' that would explain things. But if this config hasn't been modified what is causing the notification to itnetworksupport on the current D drive check isn't in this config.
Do you have a serviceescalation defined somewhere in your config? If you do, post it (or them if there are more than one). It would be something that would cause an additional notification and would be outside the inheritance issue. That config above does not account for a second sending to itnetwork support.
As for SIT-NAS2 - all the service checks are empty of contact and contact group settings and all include the generic-service template. The generic service template has a contacts of null and a contact_groups of admins. Only admins are going to receive these alerts. The admins group consists of itnetworksupport. So, for each alert of a SIT-NAS2 service, you will receive 1 email to itnetworksupport due to its members in the admins group. This matches with the logs. Everything matches up with what I'd expect here.
For 6800SRETDB, you define neither contact or contact group in the end service. All services include the template generic-service-dba. generic-service-dba containts a contact_groups null line, and a contacts of itdba. This means regardless of whats set further up the chain, the 6800SRETDB services will only notify the itdba contact (which currently has its email set to itnetworksupport@XXX). This matches with the single notification showing in the logs. Everything matches up with what I'd expect here.