Page 1 of 2
Injecting alerts/events from Splunk to Nagios XI
Posted: Mon Dec 02, 2019 5:23 am
by solarmon
Hi,
We currently have a Splunk and Zenoss setup where we are able to perform a query on Splunk and the results would be injected as events in to Zenoss, causing it to alert for it.
Can this same functionality be done for/with Nagios XI? Please could you provide some pointers on how to achieve this.
Thank you in advance for any help you can provide.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Mon Dec 02, 2019 2:20 pm
by mbellerue
Are you looking to integrate Splunk with Nagios XI, or are you looking for a full Nagios environment to replace Splunk and Zenoss?
Just on a cursory search, it looks like a Splunk + Zenoss setup is equivalent to a Nagios Log Server + Nagios XI setup. There may be options for having Nagios XI get data from Splunk, and alert on it. But I'm not sure what the options are for Splunk as far as sending data, so I can't give any specific examples.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Tue Dec 03, 2019 4:59 am
by solarmon
@mbellerue
Thank you for your response.
We are looking to integrate our Splunk server with Nagios XI.
We use Splunk for our log file analysis and we currently send 'events' to Zenoss when a scheduled Splunk query matches a certain criteria. We are looking to replace our Zenoss setup with Nagios XI so require this integration.
I'm now aware that NRDP can be used to change the status of a service, but I do not think this is exactly what we are really looking for - as we would have to create a specific service for such Splunk based alerts.
In Zenoss we are able to generate events without having any specific 'service' created for a specific host, and Zenoss will alert on that event. We would like the equivalent on Nagios XI without having to create specific services, but to be able to alert for a host when Splunk triggers an 'event' for it.
I hope that answers your query. If not, please let me know what other information you require to give further advice.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Tue Dec 03, 2019 3:35 pm
by cdienger
In order to trigger an alert on Nagios, a service or host check needs to be set up to accept 'events' from Zenoss. NRDP is one option in getting these 'events' into Nagios so that it can alert on them. Nagios can also accept SNMP traps as another way of getting them in. You can find more information about these at:
https://assets.nagios.com/downloads/nag ... erview.pdf
https://support.nagios.com/kb/article/n ... t-599.html
https://assets.nagios.com/downloads/nag ... ios_XI.pdf
Both of these are considered passive checks(clients send data to XI). The other type of check is active(XI actively queries/pulls the client for information). If your able to script something using Splunk's API to get the information you want, then it should be pretty easy to convert that into active check/plugin that XI can work with -
http://nagios-plugins.org/doc/guidelines.html.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Wed Dec 04, 2019 4:33 am
by solarmon
@cdienger
Thank you.
We feed logs in to Splunk (not Zenoss) and that perform queries to generate alerts based on the criteria set in the query, which then sends the alert/event to Zenoss.
Without reading all the documents link, I think we may still have the following challenges.
We don't want to tie an alert/event with a specific service. If the alert/event was for a host, how would multiple alerts/events be handled? (I suppose this also applies for alerts/events for a service too).
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Wed Dec 04, 2019 10:56 am
by cdienger
How would you want each alert/event handled? XI would be able to alert and notify each unique alert/event from Splunk using a single service if the is_volatile option is set.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Wed Dec 04, 2019 11:13 am
by solarmon
@cdienger
What we basically don't want is to have to create a service for every thing that needs to be alerted on from Splunk.
What we have is a telecoms solution where the resources are not (or rather, cannot be) monitored as a service on the host in question within Nagios XI. The logs from the host are sent to Splunk and if the Splunk query deems it to be an alert/event we want it to be sent to Nagios XI.
The solution we have on our Zenoss system effectively creates a 'component' for the resource (that Splunk has sent and event/alert from) on the fly, using their API. We do not need to manually create these components/services/resources up front.
We just need for Nagios XI to be able to alert us when it receives a unique event/alert from Splunk for the host in question.
I'm not aware of this 'is_volatile' option - please could you point me to more details about it.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Wed Dec 04, 2019 12:48 pm
by mbellerue
Here is the documentation on Volatile Service.
https://assets.nagios.com/downloads/nag ... vices.html
And you might also be interested in State Stalking as well.
https://assets.nagios.com/downloads/nag ... lking.html
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Thu Dec 05, 2019 4:28 am
by solarmon
@mbellerue
Thank you.
It looks like Volatile Services is what we are looking for - as we would want to both log and alert.
We will have to create a single Volatile service just for Slunk alerts on the host in Nagios, which is fine.
Re: Injecting alerts/events from Splunk to Nagios XI
Posted: Thu Dec 05, 2019 10:54 am
by mbellerue
Perfect! Is there anything else we can help with, or should I close this thread?