Nagios - Elasticserach
Nagios - Elasticserach
Hi,
We are using ElasticSearch to manage our logs.
We would like run a query on the elasticsearch DB on a specific host and if the query will show s specific parameter, then we would like Nagios to alert.
How can we implement that kind of configuration ?
Thanks
We are using ElasticSearch to manage our logs.
We would like run a query on the elasticsearch DB on a specific host and if the query will show s specific parameter, then we would like Nagios to alert.
How can we implement that kind of configuration ?
Thanks
Re: Nagios - Elasticserach
There's quite a plethora of Elasticsearch based Nagios plugins available in this repository:
https://github.com/HariSekhon/nagios-plugins
I don't think any of them are able to run queries, but they could serve as a skeleton for building out a plugin that does that. Here are the development guidelines if you wanted to build your own Nagios plugin:
https://nagios-plugins.org/doc/guidelines.html
The official Nagios Log Server configuration wizard for Nagios XI can run a query and check the returned record count, but it does this through the Nagios Log Server API and not through the native Elasticsearch API (though they are very similar). However, it could probably be modified to fit this use case:
https://support.nagios.com/kb/article.php?id=75
A link to download this wizard, which will include the plugin for doing this:
https://assets.nagios.com/downloads/nag ... server.zip
And if you're not sure how to use custom plugins with Nagios Core, this guide is a good starting point:
https://www.howtoforge.com/tutorial/wri ... ck-plugin/
Let us know if you have additional questions. This is definitely possible.
https://github.com/HariSekhon/nagios-plugins
I don't think any of them are able to run queries, but they could serve as a skeleton for building out a plugin that does that. Here are the development guidelines if you wanted to build your own Nagios plugin:
https://nagios-plugins.org/doc/guidelines.html
The official Nagios Log Server configuration wizard for Nagios XI can run a query and check the returned record count, but it does this through the Nagios Log Server API and not through the native Elasticsearch API (though they are very similar). However, it could probably be modified to fit this use case:
https://support.nagios.com/kb/article.php?id=75
A link to download this wizard, which will include the plugin for doing this:
https://assets.nagios.com/downloads/nag ... server.zip
And if you're not sure how to use custom plugins with Nagios Core, this guide is a good starting point:
https://www.howtoforge.com/tutorial/wri ... ck-plugin/
Let us know if you have additional questions. This is definitely possible.
Former Nagios employee
https://www.mcapra.com/
https://www.mcapra.com/
- tacolover101
- Posts: 432
- Joined: Mon Apr 10, 2017 11:55 am
Re: Nagios - Elasticserach
going with what @mcapra mentioned, a script like this should work for your needs - https://github.com/misiupajor/check_elasticsearch
a suggestion is to look at using NLS for this, as it will make querying off dynamic data much easier.
a suggestion is to look at using NLS for this, as it will make querying off dynamic data much easier.
Re: Nagios - Elasticserach
Quite a bit of info in the last couple of posts. Thanks @mcapra and @tacolover101! Let us know if you have any further questions @amitw.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: Nagios - Elasticserach
Thanks for the info guys.
I'm trying to think of way to query ElasticSeach with HTTP GET, like this:
GET /test-log-YYYY.MM.DD/_count HTTP/1.1
Host: test.kibana.com:9200
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 132
Content-Type: application/x-www-form-urlencoded
{ "query": { "bool": { "must": [{ "match_phrase": { "message": "time out" }}, { "range": { "date": { "gte": "now-15m" } } } ] } } }
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 59
{"count":0,"_shards":{"total":5,"successful":5,"failed":0}}
where YYYY.MM.DD is the current date? Then it should raise alarm in case the returned count value is greater than zero, and remove alarm if it returns to zero.
It should query ElasticSearch index called test-lot-YYYY.MM.DD for all events added during last 15 minutes and containing text “time out”.
Can Nagios do that?
Thanks
I'm trying to think of way to query ElasticSeach with HTTP GET, like this:
GET /test-log-YYYY.MM.DD/_count HTTP/1.1
Host: test.kibana.com:9200
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 132
Content-Type: application/x-www-form-urlencoded
{ "query": { "bool": { "must": [{ "match_phrase": { "message": "time out" }}, { "range": { "date": { "gte": "now-15m" } } } ] } } }
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 59
{"count":0,"_shards":{"total":5,"successful":5,"failed":0}}
where YYYY.MM.DD is the current date? Then it should raise alarm in case the returned count value is greater than zero, and remove alarm if it returns to zero.
It should query ElasticSearch index called test-lot-YYYY.MM.DD for all events added during last 15 minutes and containing text “time out”.
Can Nagios do that?
Thanks
Re: Nagios - Elasticserach
The best way to create this would be through a query on the dashboards and create an alert on the result.
Under dashboards run a query for:
message:"time out"
and the use the 'create an alert' button (bell in the top right) to create an alert based on queries of the current dashboard.
https://library.nagios.com/library/prod ... og-events/
Under dashboards run a query for:
message:"time out"
and the use the 'create an alert' button (bell in the top right) to create an alert based on queries of the current dashboard.
https://library.nagios.com/library/prod ... og-events/
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: Nagios - Elasticserach
Using the plugin provided by @tacolover101, you'd probably need to leverage some sort of wrapper script to get the correct index name based on the current date. Or modify the script itself.
You could also bake a date execution into your Nagios command definition, but you may need to do some clever escaping.
You could also bake a date execution into your Nagios command definition, but you may need to do some clever escaping.
Code: Select all
[root@capra_nag bin]# echo "check_elasticsearch.py --host <host> --index \"logstash-$(date '+%Y.%m.%d')\" --query <query> --warning <warning> --critical <critical>"
check_elasticsearch.py --host <host> --index "logstash-2018.05.07" --query <query> --warning <warning> --critical <critical>
Former Nagios employee
https://www.mcapra.com/
https://www.mcapra.com/
Re: Nagios - Elasticserach
Thanks for the assist, @mcapra and @tacolover!
@amitw, please let us know if you need further assistance.
@amitw, please let us know if you need further assistance.
Former Nagios employee
Re: Nagios - Elasticserach
THANKS ALOT!
will test it and update you accordingly
will test it and update you accordingly