Service Check Notifications w/ Multiple Teams

[pcappell]

Hello,

We are looking at a re-architecture of our NagiosXI hosts and services to handle notifications for multiple teams.

I'm curious to see how other NagiosXI users are handling instances where the same service check is applied to multiple hosts, but the hosts belong to different teams, so notifications should be routed to the appropriate team?

What we're currently looking at doing it just duplicating the service check X number of times for each team and modifying the alert contacts, but there has to be a more refined way of handling these notifications.

Thanks!
Paul

[cdepaulis]

Hello,

I would like to better understand the problem you are facing with this. Is the primary issue that you can't get checks for services to send notifications to specific teams because the host is owned by a different team? If you could just clarify a little further then I can help make a possible path to the solution.

- Caleb

[lgute]

Hi @pcappell,

Thanks for reaching out. I think what you want is to setup your Hosts & Services with multiple Contact Groups. That way any number of contact groups will receive notifications, without having to run more checks.

This config example is for an individual server and you can configure something similar for hosts.

Code: Select all

define service {
    host_name               web-server
    service_description     CPU Load
    contact_groups          linux-admins,windows-admins,dba-team
}   

You can accomplish the same thing in the XI GUI.

1. Go to Configure > Core Config Manager > Hosts (or Services).
2. Select the host or service you want to edit.
3. Click on the Notification tab.
4. In the Contact Groups field, select multiple groups using Ctrl+click (or Cmd+click on Mac).
5. Click Update and then Apply Configuration.

This ensures all selected contact groups receive notifications for that host or service.

If you have an Enterprise License, you can use the Bulk Modification Tools to add Contact Groups to many Hosts/Hosts in Host Groups or Services/Services in Service Groups quickly.

[kg2857]

The OP means they have the same services running on multiple hosts and the hosts are owned by different groups. Group 1 shouldn't get notifications for group 2 service alerts.

[pcappell]

Hello,

kg2857 was spot on.

We have, for example, a Windows Disk Space check for the D:\ drive hitting multiple hosts. We tend to isolate applications to the D:\ drive for easier permission management.

With multiple groups managing multiple hosts with their own applications, to send out notifications to the correct group, the only solution I can think of at this time is having duplicate copies of the same service check, with the only difference between the checks being the Contacts. This would allow the appropriate groups to receive a notification if their D:\ drive is filling up.

I was wondering if anyone had a more scalable and/or creative way of managing this type of conundrum.

Thank you!

[-SN]

Not to clear on your end goal here.

1. Nagios Contact = a single contact
2. Contact Group = a collection of multiple contacts

Inside the ask:
Your "Group" = Group of human resources responsible for a particular service offering/application.
Your "Multiple Groups" = Multiple groups of human resources responsible for a particular service offering/application.
You're looking for a method that allows for select notifications to be sent to the appropriate location for actionable alerts.

I think what you're looking for is a notification strategy.
You'll need to science this a bit more to find a solution that fits for you and you'll have to be able to define a few variables.

1. What's the method of notification for your teams? Is that email, messenger ping, etc.?
2. Is that a single point of contact? A mail alias, a channel, or a list of items?
3. How is that contact defined in Nagios?
4. How are you assigning the contact in Nagios? Individual or Group Membership
5. Do you have the ability to create a contact notification command?
6. Do you have a tagging strategy for Host and Service Objects?

Now to the objects themselves.
If you have host1 which runs multiple applications and you have multiple teams hosting their application on drive D of host1 and you need to deploy monitoring service to meet this requirement.

Here you have multiple stakeholders that are all invested in the capacity of the drive on host1. Wouldn't they all receive the notification of the service crossing the tipping point threshold? Does that require a separate check for each stakeholder or are there other factors at play?

1. Does this need bifurcation with a primary and secondary contact?
2. Do you go through an upstream application like PagerDuty or ServiceNow?
3. Do you have a RACI flow that you have to follow?

There's not much that you can't accomplish with the feature set in XI when it comes to waking people up at 2:15AM because there is a problem. You just have to know how to get ahold of them and when to do it.

--SN